- Email: [email protected]
Considerations for mood and emotion measures in mindfulness-based intervention research
Accepted Manuscript Title: Considerations for Mood and Emotion Measures in Mindfulness-Based Intervention Research Authors: Elizabeth A Hoge, Samantha R Philip, Carl Fulwiler PII: DOI: Reference:
S2352-250X(18)30230-6 https://doi.org/10.1016/j.copsyc.2019.02.001 COPSYC 789
To appear in:
Please cite this article as: Hoge EA, Philip SR, Fulwiler C, Considerations for Mood and Emotion Measures in Mindfulness-Based Intervention Research, Current Opinion in Psychology (2019), https://doi.org/10.1016/j.copsyc.2019.02.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Considerations for Mood and Emotion Measures in Mindfulness-Based Intervention Research Elizabeth A. Hoge M.Da , Samantha R. Philip B.Sa, Carl Fulwiler M.D.. PhD.b a
Department of Psychiatry, Georgetown University Medical center, Washington, D.C. USA Departments of Psychiatry and Medicine, University of Massachusetts Medical School, Worcester, MA 06155 USA Elizabeth Hoge, MD Associate Professor of Psychiatry Georgetown University Medical Center 2115 Wisconsin Ave NW Washington, DC 20007 Phone (202) 687-0635 Fax (202) 687-2684 Email: [email protected]
U
Corresponding Author:
SC RI PT
b
N
Abstract A large and growing body of work has examined the effects of Mindfulness-Based Interventions
A
(MBI’s), such as Mindfulness-Based Stress Reduction and Mindfulness-Based Cognitive Therapy,
M
on emotion-related outcomes, both in mental health settings and general populations. These
D
studies vary widely in the approach to measurement of emotion-related measurements after MBI’s.
TE
A systematic review of randomized clinical trials of MBIs was conducted with a focus on identifying what emotion-related assays were able to detect changes with MBI’s, including scales
EP
and instruments (both self-report and clinician-rated) on constructs such as depression, anxiety,
CC
emotion regulation, and other mood states. In this paper we reflect on these findings and discuss considerations of outcome measures in MBI research. There are previously established practices
A
for clinical trials research on emotion-related outcomes which may provide some useful methodological standards and study design options for use by the MBI research field.
Introduction Measurement of emotion is not straightforward [1]. There is general agreement that emotions can be characterized as the subjective, physiological and behavioral responses to an event [2].
Accordingly, instruments for measuring each of these components of emotion have been developed. However, there are wide variations in correlation between subjective, physiological and behavioral measures, and each approach has its weaknesses. Unfortunately, there is no “gold
SC RI PT
standard” measure of emotion [3].
Many researchers examining the effects of Mindfulness-Based Interventions (MBIs) have an interest in emotion-related changes that occur over the course of mindfulness training . These effects are assessed with a wide variety of measures which range from current mood states to
U
symptoms of emotional disorders such as depression or anxiety disorders. Measures can be
N
clinician-rated, self-report, behavioral (how a person responds in a laboratory testing paradigm) or
A
physiological (electrophysiology, hormones, or immunological markers). The type of measure
M
depends on the study population, design, and the goals of the authors.
D
A systematic review of randomized clinical trials of MBI’s was conducted with a focus on
TE
identifying these measures, or assays, used to assess outcomes related to emotion (Kimmel et al,
EP
in preparation). We found that the most widely used type of measure for emotion outcomes was self-report (over 90%), with only about 5-10% of studies using either clinician-rated or
CC
physiological measures (and none using behavioral ones).
A
Self-report measures have significant limitations. These measures may be more valid for current emotional states than for emotional traits (or symptom severity) in general, and the symptoms they measure may not be specific (for example, overlap with common symptoms of a medical illness, such as racing heart). To be considered accurate, self-reports should ideally correlate with
objective measurements (clinician-rated, behavioral, or physiological) and/or functional outcome, but they often do not, or these data are not available. Studies examining the correlation between self-report scales and physiological or behavioral outcome measures would be an important
SC RI PT
contribution to the field.
Physiological measures for assessing emotion, such as autonomic activity, facial expression and central nervous system changes, are more objective but are not well established. In our systematic review, we found that less than 10% of the studies utilized physiological measures, despite
U
evidence that these measures may correlate to psychological states or emotion-related conditions.
N
For example, several studies have shown fMRI amygdala hyper-reactivity when viewing fearful Other research has looked at
A
or angry faces in patients with anxiety disorders ([4], [5]).
M
physiological reactions to emotionally provocative stimuli such as effect of International Affective Picture System (IAPS) pictures on skin conductance [6], the Trier Social Stress Test on stress [7],
or
individually-created
D
hormones
negative
narratives
about
oneself
on
TE
electroencephalographic activation, eye-blink startle reflux, and immune response [8]. Although
EP
some aspects of emotion are measurable, research has not yet identified anything close to a specific physiological fingerprint for specific emotions. Nevertheless, given the risk of bias associated with
CC
self-report instruments, physiological measures could serve as an objective way of measuring changes in distressful emotion-related outcomes, and the development of these techniques should
A
continue to be emphasized in future studies.
In light of this complexity in measuring emotion, we aim to examine the challenges faced specifically by MBI researchers when attempting to capture emotional outcomes of these
SC RI PT
interventions and offers recommendations to strengthen the body of research in this field.
Emotion-Related Measures Used in MBIs Depends on Study Aim
When determining which emotion-related assays to use in a study, researchers must consider the population of interest and the aims of the study. In our systematic review of emotional domain
U
outcomes in MBIs (primarily MBSR and MBCT, Kimmel et al., in preparation), we found that
N
studies that measured current mood states as primary outcomes (about 25% of the studies), such
A
as with the Positive and Negative Affect Schedule (PANAS) or the Profile of Mood States,
M
(POMS), often used a population that did not undergo diagnostic assessment, such as college students, nurses, etc. In this case, the primary aim of the studies was to assess whether an
D
experimental procedure leads to an observable change in current mood states.
TE
In contrast, we found that a majority (75%) of MBI work in emotion-related outcomes was
EP
in clinical samples --individuals with a diagnosed psychiatric or medical illness, for which the aim was to test whether the MBI was associated with a reduction in symptom burden (anxiety or
CC
depression) or medical illness-related stress (such as in diabetes, cancer, chronic pain, etc). The most common psychiatric disorders were depression and anxiety disorders, with approximately
A
35% of all the MBI research on just these two categories of disorders, evincing the considerable interest there is in using MBI’s to treat mental illness. The individual scales most commonly used for anxiety and depression were the State-Trait Anxiety Inventory (STAI), the Beck Anxiety Inventory (BAI), the Beck Depression Inventory II (BDI-II), and the Center of Epidemiologic
Studies Depression Scale (CES-D). Overall, our review found that about half of the time, the MBI group had significantly more change in anxiety and depression measures than the control group.
SC RI PT
If these anxiety and depression measures were originally designed for use with psychiatric patients; what are the implications of using them in non-psychiatric populations? The interpretation of results may be less clear. For example, if no change in depression symptoms is observed in an MBI trial, is this because the MBI was not effective in reducing depression symptoms, or because, if the study sample is from the general population, the symptoms were too low at baseline to have a
U
detectable change (“floor effect”)? Secondly, are items from scales designed for a disordered
N
population relevant for people with no psychiatric illness (e.g., what percent of the population has
A
“inappropriate guilt which may be delusional”? [9]. A study sample that contains unidentified
M
depressed and non-depressed individuals will be have greater variability and a more complicated analysis, and depression scales may be assessing symptoms mistakenly ascribed to depression (but
TE
D
are really due to a medical condition or some other cause).
EP
MBIs on Mood States and Emotion Regulation
CC
As mentioned earlier, many MBI studies are interested in how mindfulness interventions affect mood states. In our review, 15 studies assessed mood states and specific emotions, with 70% using
A
scales such as the POMS and the PANAS. The POMS, for example, assesses 6 different dimensions of mood (e.g., tension or anxiety, anger of hostility, vigor or activity, etc.), while the PANAS consists of two scales measuring positive affect and negative affect. Findings revealed that, over half of the time, the MBIs led to significantly more change in emotion and mood states
than the control group. More specifically, MBIs were generally associated with significant increased positive affect and significantly decreased negative affect, tension and anxiety, fatigue,
SC RI PT
anger, and total mood disturbance compared to the control group (Kimmel et al., in preparation).
Another common emotion-related construct of interest in MBI research is emotion regulation: an individual’s ability to adjust the intensity and/or duration of an emotion, or the presence of emotional states [10]. Poor emotion regulation can lead to worse health outcomes, as it impairs one’s ability to engage in self-regulating health behaviors related to diet, exercise, smoking, etc
U
[11, 12]. Thus, greater emotion regulation may facilitate more self-regulation behaviors and, in
N
turn, improved health outcomes. In our systematic review, 4 studies utilized emotion regulation
A
measures. The most commonly used measure was the Difficulties in Emotion Regulation Scale
M
(DERS), which assesses one’s self-reported ability to cope with negative emotions. The findings demonstrated that MBIs outperformed the control in almost 70% of the emotion regulation
TE
D
comparisons.
EP
Research with Clinical Populations: Depression & Anxiety
CC
Methodological Considerations
Because of the intense interest in MBIs for psychiatric populations, it is important that researchers
A
utilize rigorous clinical trial methodology that will ensure that findings are taken seriously by the medical community, health insurance payers and disease management companies.
Despite
hundreds of published trials on MBSR with promising results, the vast majority of health insurance companies do not cover this treatment.
General considerations for study design for behavioral interventions such as MBI’s will vary depending on the overarching aim of the study and the stage of research for the intervention.
SC RI PT
According to the Stage Model for Behavioral Intervention Development [13]), in Stage I of intervention development, researchers work towards generating a new intervention or modifying existing interventions and running pilot trials. In this stage, the aim could be to test the feasibility of the intervention and proof-of-concept, and a control group is not required. If no change in the outcome measure is detected after the intervention, researchers may need to go back to
U
development and modification of the intervention. Alternatively, the aim could be to test patient
N
acceptability of an intervention, the outcome measures would focus on treatment acceptability,
M
A
adherence, and satisfaction.
If a “signal” is detected (a change associated with the treatment), researchers can feel more
D
confident about moving into more advanced stages of methodological design, such as formally
TE
testing efficacy of the MBI and using an active control group. Efficacy studies are most robust
EP
when they use an active control group or placebo group to test if the effect of the intervention is due to the actual treatment or the non-specific (placebo) effects of treatment. Once sufficient
CC
evidence of efficacy is established, the next step to improve external validity could be to use a comparative effectiveness approach to test the intervention. Thus, the choice of study design
A
should be carefully selected based on the developmental stage of intervention.
MBI researchers conducting treatment trials for anxiety and depression should be aware of previously established standards in order to make their data relevant, accepted, and impactful. For
example, the US Food and Drug Administration demands the highest rigor in methodological standards from drug companies seeking an indication for their drug. Some of these practices are relevant for MBI research and it is therefore worthwhile to be aware of these established standards
SC RI PT
when considering design and procedures of MBI randomized, controlled trials (RCT’s) in order to make an impact in wider arenas including payors and managed care. In our systematic review of MBI research on emotion (Kimmel et al, in preparation), a significant number (more than 50%) of the included studies had design elements that suggested a high risk for bias, and less than half of the studies utilized an active control group. MBI researchers should be aware of the expectation
N
U
in the field for clinical trials.
M
A
Diagnostic Screening
In the above section we discuss the risks associated with not conducting formal diagnostic
D
screening for samples of individuals with anxiety and depression. In our systematic review of
TE
MBI research and emotion-related outcomes, nearly 60% of the studies using emotion-related
EP
measures did not utilize a standardized, diagnostic instrument designed to assess the presence of a depressive or anxiety disorder, despite the fact that many of them included anxiety and depression
CC
measures. Often, authors simply measure anxiety and depression symptom levels in a population that might be expected to experience these, such as university students, nurses, or those with a
A
chronic medical problem.
Authors should consider carefully whether the aim of the study is to correct an abnormal symptom, such as in a clinical population, or decrease negative emotions in a normal population, with whom
many of the depression and anxiety scales were not designed to be used. Authors need to think about which type of population they are targeting, so that they can use the most appropriate scales.
SC RI PT
Looking at standard clinical trials procedures in the medical literature, we see that the FDA guidance document for depression (2018) treatment trials states that: “diagnosis should be confirmed via a semi-structured interview such as the current Structured Clinical Interview for DSM or MINI International Neuropsychiatric Interview.” [9].
U
A related concern is that trials in which depression symptoms are measured, sometimes do not
N
screen for, or adequately address, suicidality. Several questionnaires do contain suicidality items,
A
such as the BDI-II and the Patient Health Questionnaire (PHQ-9), but not all do (such as the
M
CESD). Researchers who suspect depression symptoms or a depressive disorder in their populations should specifically ask about suicidal thinking, ideation, and planning, and provide
TE
D
appropriate clinical follow-up care and referral if risk is detected.
EP
The Problem of Comparison Groups
CC
One methodological issue that remains problematic is the question of a comparison group for an RCT on MBSR or MBCT. Although some have argued that traditional RCT study designs are not
A
relevant or possible for use with complementary and alternative medicine (CAM) therapies [15], wider audiences remain skeptical of outcomes without this trial component. Looking at industry standards, we see that the FDA guidance document for depression treatment trials, for example, states that “a placebo group is necessary to ensure that observed effects are not the result of
spontaneous improvement, expectation bias, attention from health care professionals involved in the trial, regression to the mean, or other factors not related to the activity of the study drug.” [14].
SC RI PT
Criticism has been raised concerning inadequate control groups in MBI research (for example see review for the Agency for Healthcare Research and Quality (AHRQ): [16, 17, 18, 19, 20] but suggested attention control conditions are problematic. For example, an effort to develop “plausible and therapeutic comparison conditions” [20] such as the health enhancement program (HEP) disqualifies that very comparison group because it has become too therapeutic for the
U
detection of a contrast. For example, HEP contains aerobic exercise, which has already been
N
shown to decrease anxiety, stress and depression symptoms [21, 22, 13]. If a hypothetical study
A
compares two active (therapeutic) treatments, such as HEP and MBSR, and both are found to be
M
mildly helpful (which we might expect given that they both have therapeutic elements), we still don’t know whether MBSR is effective—we can’t tell from this design whether both work, or both
D
don’t work and other factors resulted in the findings. If researchers want to compare two active
EP
TE
treatments, the most appropriate study design is non-inferiority.
CC
Non-inferiority
An alternative to utilizing an attention control comparison group is to use a therapeutic comparator
A
that has already been established to be effective, or a “gold standard” of treatment. This approach would also address the US Agency for Healthcare Research and Quality's concern that “special attention should be paid to developing studies that provide a more accurate assessment of the efficacy and effectiveness of meditation practices, both against standard therapies and against
each other” [24]. A 2-arm non-inferiority design could be considered when it would be unethical to have a placebo group receiving no treatment.
SC RI PT
In this alternative approach, the statistical comparison would be looking for ‘equivalence’ or, more accurately, ‘non-inferiority,’ rather than superiority to control (as in a typical RCT). Noninferiority trials are used when—despite the fact that there is already a proven treatment for a disorder—a new treatment is hypothesized to be useful (e.g., fewer side effects, faster onset of action). The non-inferiority design is a valid approach to prove efficacy. For example, it is included
U
as an acceptable design for companies seeking formal FDA approval for a drug [25]. In this design,
N
the statistical test concerns whether the new treatment is worse than the standard treatment by
A
some pre-determined ‘non-inferiority margin.’ [26, 27]. This is in contrast to a traditional,
NOT indicate equivalence.
M
superiority trial, in which finding no statistically significant difference between treatments does An example of how this approach could be used in mindfulness
D
research would be to compare mindfulness training (a newer treatment), with an established, gold-
TE
standard treatment (such as medication or psychotherapy) for major depressive disorder or anxiety
EP
disorders.
CC
The non-inferiority margin is calculated and declared before the start of the study using data from an earlier, gold-standard treatment’s efficacy study, thus incorporating that earlier study’s placebo
A
group data to determine this margin (that the non-inferiority margin is not too close to the placebo outcome). In this way, the new treatment is determined to be not significantly worse than the goldstandard treatment. Non-inferiority analyses utilize a separate set of statistical procedures and
typically a larger study sample is needed, which could be achieved with multiple study sites (for more information see [8, 25, 29]).
SC RI PT
Dose-Response Relationship
Another alternative to use of a placebo group is a dose-response trial. An example of a drug doseresponse trial would be this: a drug at 100 mg is found to have twice the clinical effect as the same drug at 50 mg, showing that the change is likely due to the treatment. This design has been used
U
in medication studies and with psychosocial treatment trials when the use of placebo is unfeasible
N
or unethical [30, 31, 32]. In mindfulness studies, this approach could be used in a design in which
A
twice the amount of training was compared to half that amount of time; if the effect is large in the
M
longer group and medium in the shorter group, this suggests that the training had an effect on the
D
outcome.
EP
TE
Clinician Ratings vs. Self-Report Ratings in Clinical Populations
Another important consideration in clinical trials is the type of symptom severity rating used as
CC
the primary outcome variable, as briefly discussed above. Turning to traditional standards-ofpractice, we see that the FDA guidance document for depression treatment trials states that
A
“clinician-rated outcome measures are the current standard for assessing efficacy in antidepressant trials. To date, the FDA has accepted the following as primary endpoints in phase 3 medication studies to support an MDD indication: Hamilton Depression Rating Scale (typically the 17-item version) Montgomery Asberg Depression Rating Scale.” In trials of anxiety disorders, the most
widely recognized clinician-rated scales, which are also used in FDA registry trials, are the Hamilton Anxiety Scale for generalized anxiety disorder (HAM-A, or the scripted version: Structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A) [33]; the Panic
SC RI PT
Disorder Severity Scale for panic disorder [34], and the Liebowitz Social Anxiety Scale for social anxiety disorder [35]. Utilizing traditional, widely-used instruments allows for comparison of findings with published trials across decades.
The preference for clinician ratings over self-report ratings of depression is supported by several
U
reviews and meta-analyses [36, 37]. For example, a meta-analysis of 48 studies including a total
N
of 2462 participants found that “clinician-rated instruments resulted in a significantly higher effect
A
size than self-report instruments from the same studies” [36]. Clinician ratings may assess global
M
disorder severity more accurately because of experience and training; in contrast, a patient’s perspective is in relation only to his/her own experience. There are other limitations of self-report
D
instruments, for example, a patient may have trouble completing instruments due to diminished
TE
attention or cognition. Or, ratings may be affected by social desirability influences or disease states
EP
which may distort perceptions or judgment. Despite strong evidence for the use of clinician-rated outcomes, our systematic review found that less than 10% of MBI studies with clinical samples
CC
used clinician-rated outcomes. MBI studies should aim to employ more clinician-rated outcomes in order to be in line with FDA guidance and to be taken seriously by the scientific community.
A
However, clinician-rated instruments should be carefully selected, as some can be problematic; some of them are decades old (for example, the HAM-D was originally developed in 1960) and the diagnostic criteria for anxiety and depression have changed with successive versions of the
DSM. Some have missing symptom categories, such as hypersomnia, or include symptoms that are no longer diagnostic features of the disorders.
SC RI PT
Self-report instruments also have an important role in clinical research [37]. For example, patientrated questionnaires can better assess the impact of symptoms on overall functioning, work impairment, and social functioning. Self-report measures also benefit from being free from biases from the clinician. Often, clinical trials use both clinician-rated and patient-rated instruments.
U
A recent NIH initiative to improve self-report measures used recent advances in information
N
technology, psychometrics, and qualitative, cognitive, and health survey research to produce the
A
NIH Toolbox (www.nihtoolbox.org) and the Patient Reported Outcomes Measurement
M
Information System (PROMIS; www.nihpromis.org). Given the known limitations of the selfreport instruments most widely used in the mindfulness literature, and the advances in assessment
D
represented by the PROMIS measures, we encourage MBI researchers to utilize these instruments
TE
for symptom assessment, and to continue exploring objective and functional to overcome the
EP
overcome the limitations of symptom assessment.
CC
Cost and Blinding
A
A few other considerations can help researchers choose between clinician-ratings and patient-rated instruments. One consideration—cost—can influence decisions about whether a primary measure, whether it is clinician-rated or self-rated. The Beck instruments (self-report Beck Depression Inventory and Beck Anxiety Inventory) require payment, whereas the MADRS (clinician rated
form) is free. Lastly, the use of clinician raters allows for some aspect of data collection to remain blinded (also called “masked”); that is, clinician raters are kept “blind” to treatment allocation and thus expectation effects are kept to a minimum. Given that MBI trials are impossible to conduct
SC RI PT
using standard double-blind procedures, having blinded clinical assessors provides some protection against expectation bias and could be considered a “single-blind” design.
Conclusion
U
There are many aspects to consider when designing a MBI study when emotion-related measures
N
are the primary outcome. For populations with a disorder, researchers should use diagnostic
A
instruments to ascertain presence of the disorder and consider clinician-rated instruments to
M
measure symptom change over time. Given the advances in self-report assessment represented by the PROMIS measures, we encourage MBI researchers to utilize PROMIS instruments for
D
symptom assessment when self-report outcomes are desired. Lastly, researchers should continue
TE
to explore objective, physiological measures to overcome the limitations of questionnaire-based
EP
symptom assessment.
A
CC
Author Declarations: No declarations. There are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all three named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing, we confirm that we have followed the regulations of our institutions concerning intellectual property. We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). We are responsible for submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible for the Corresponding Author.
D
TE
EP
CC
A
SC RI PT
U
N
A
M
References [1] Barrett, L. F. Navigating the science of emotion. In Meiselman H. L. (Ed.), Woodhead Publishing. (2016), pp. 31-63. doi://doi.org/10.1016/B978-0-08-100508-8.00002-3 [2] Nairne, J. S. Psychology: The Adaptive Mind. 2nd Ed. Wadsworth, 2000. p. 444.
pp. 209-237. doi:10.1080/02699930802204677.
SC RI PT
[3] Mauss, I. B., & Robinson, M. D. Measures of emotion: A review. Cognition & Emotion, 23 (2009),
[4] Fonzo, G. A., Ramsawh, H. J., Flagan, T. M., Sullivan, S. G., Letamendi, A., Simmons, A. N., . . . Stein, M. B. Common and disorder-specific neural responses to emotional faces in generalised anxiety, social anxiety and panic disorders. The British Journal of Psychiatry: The Journal of Mental Science,
N
U
206 (2015), pp. 206-215. doi:10.1192/bjp.bp.114.149880.
A
[5] Etkin, A., & Wager, T. D. Functional neuroimaging of anxiety: A meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. The American Journal of
M
Psychiatry, 164 (2007), pp.1476-1488. doi:164/10/1476.
visual,
auditory
and
haptic
stimuli. Scientific
Data,
5
(2018),
pp.
180120.
TE
to
D
[6] Gatti, E., Calzolari, E., Maggioni, E., & Obrist, M. Emotional ratings and skin conductance response
EP
doi:10.1038/sdata.2018.120.
[7] Hoge, E. A., Bui, E., Palitz, S. A., Schwarz, N. R., Owens, M. E., Johnston, J. M., . . . Simon, N. M.
CC
The effect of mindfulness meditation training on biological acute stress responses in generalized anxiety disorder (2018). doi://doi.org/10.1016/j.psychres.2017.01.006.
A
[8] Rosenkranz, M. A., Jackson, D. C., Dalton, K. M., Dolski, I., Ryff, C. D., Singer, B. H., . . . Davidson, R. J. Affective style and in vivo immune response: Neurobehavioral mechanisms. Proceedings of the National Academy of Sciences of the United States of America, 100 (2003), pp. 11148-11152. doi:10.1073/pnas.1534743100.
[9] American Psychiatric Association., American Psychiatric Association. DSM-5 Task Force.,. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Arlington, VA: American Psychiatric Association.
SC RI PT
[10] Gross, J. J. The emerging field of emotion regulation: An integrative review. Reviews of General Psychology. 2 (1998), pp. 271-299.
[11] Smyth, J. M., & Arigo, D. Recent evidence supports emotion-regulation interventions for improving health in at-risk and clinical populations. Curr Opin Psychiatry. 22(2009), pp. 205-
U
210. doi:10.1097/YCO.0b013e3283252d6d
N
[12] Wallis, D. J., & Hetherington, M. M. Emotions and eating. Self-reported and experimentally
M
doi:10.1016/j.appet.2008.11.007
A
induced changes in food intake under stress. Appetite. 52(2009), pp. 355-362.
D
[13] Onken, L., Carroll, K., Shoham, V., Cuthbert, B., & Riddle, M. Reinvisioning clinical science:
TE
Unifying the discipline to improve the public health. Clinical Psychological Science, 2 (2014), pp.
EP
22–34. doi: 10.1177/2167702613497932. [14] Food Drug Administration Center for Drugs Evaluation Research (2018). Major Depressive Disorder:
CC
Developing Drugs for Treatment: Guidance for (FDA Maryland).
A
[15] Forbes, B. Yoga and managed care: A cautionary tale. International Journal of Yoga Therapy, 20 (2010), pp. 22-25. doi:10.17761/ijyt.20.1.t873831731361401.
[16] Ospina, M. B., Bond, K., Karkhaneh, M., Tjosvold, L., Vandermeer, B., Liang, Y., . . . Klassen, T. P. Meditation practices for health: State of the research. Evidence Report/Technology Assessment, (155) (2007), pp.1-263.
[17] Ospina, M. B., Bond, K., Karkhaneh, M., Buscemi, N., Dryden, D. M., Barnes, V., . . . ShannahoffKhalsa, D. Clinical trials of meditation practices in health care: Characteristics and quality. Journal of Alternative and Complementary Medicine (New York, N.Y.), 14 (2008), pp. 1199-
SC RI PT
1213. doi:10.1089/acm.2008.0307. [18] Farias, M., Wikholm, C., & Delmonte, R. What is mindfulness-based therapy good for? The Lancet. Psychiatry, 3 (2016), pp. 1012-1013. doi: S2215-0366(16)30211-5.
[19] Davidson, R. J., & Kaszniak, A. W. Conceptual and methodological issues in research on mindfulness
and
meditation.
American
Psychologist,
(2015),
pp.
581-592.
U
doi:10.1037/a0039512.
70
N
[20] Goldberg, S. B., Tucker, R. P., Greene, P. A., Simpson, T. L., Kearney, D. J., & Davidson, R. J. Is
M
(2017). doi: 10.1371/journal.pone.0187298.
A
mindfulness research methodology improving over time? A systematic review. PloS One, 12
D
[21] MacCoon, D. G., Imel, Z. E., Rosenkranz, M. A., Sheftel, J. G., Weng, H. Y., Sullivan, J. C., . . . Lutz, A. The validation of an active control intervention for mindfulness based stress reduction
TE
(MBSR). Behaviour Research and Therapy, 50 (2012), pp. 3-12. doi: 10.1016/j.brat.2011.10.011.
EP
[22] Harvey, S. B., Overland, S., Hatch, S. L., Wessely, S., Mykletun, A., & Hotopf, M. Exercise and the prevention of depression: Results of the HUNT cohort study. The American Journal of Psychiatry,
CC
175 (2018), pp. 28-36. doi: 10.1176/appi.ajp.2017.16111223.
A
[23] Asmundson, G. J., Fetzner, M. G., Deboer, L. B., Powers, M. B., Otto, M. W., & Smits, J. A. Let's get physical: A contemporary review of the anxiolytic effects of exercise for anxiety and its disorders. Depression and Anxiety, 30 (2013), pp. 362-373. doi:10.1002/da.22043.
[24] Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, Publication No. 07-E010 (2007).
[25] 21 Code of Federal Regulations, 314.126. [26] Streiner, D. L. Alternatives to placebo-controlled trials. The Canadian Journal of Neurological Sciences.Le Journal Canadien Des Sciences Neurologiques, 34 Suppl 1 (2007), pp. 37.
SC RI PT
[27] Leon, A. C. Evolution of psychopharmacology trial design and analysis: Six decades in the making. The Journal of Clinical Psychiatry, 72 (2011), pp. 331-340. doi:10.4088/JCP.10r06669.
[28] D'Agostino RB, S., Massaro, J. M., & Sullivan, L. M. Non-inferiority trials: Design concepts and issues - the encounters of academic consultants in statistics. Statistics in Medicine, 22 (2003), pp.
U
169-186. doi:10.1002/sim.1425.
N
[29] FDA Non-Inferiority Clinical Trials to Establish Effectiveness Guidance for Industry (2016).
September
26,
1996,
Amsterdam.
European
College
of
M
meeting,
A
[30] Montgomery, S. A. Alternatives to placebo-controlled trials in psychiatry. ECNP consensus
Neuropsychopharmacology. European Neuropsychopharmacology: The Journal of the European
D
College of Neuropsychopharmacology, 9 (1999), pp.265-269. doi: S0924977X98000492.
TE
[31] Delgadillo, J., McMillan, D., Lucock, M., Leach, C., Ali, S., & Gilbody, S. Early changes, attrition, and dose-response in low intensity psychological interventions. The British Journal of Clinical
EP
Psychology, 53 (2014), pp. 114-130. doi:10.1111/bjc.12031.
CC
[32] Dinan, T. G. Lithium augmentation in sertraline-resistant depression: A preliminary doseresponse study. Acta Psychiatrica Scandinavica, 88 (1993), pp. 300-301.
A
[33] Shear, M. K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., . . . Wang, T. Reliability and validity of the panic disorder severity scale: Replication and extension. Journal of Psychiatric Research, 35 (2001), pp. 293-296. doi: S0022-3956(01)00028-0.
[34] Shear, M. K., Vander Bilt, J., Rucci, P., Endicott, J., Lydiard, B., Otto, M. W., . . . Frank, D. M. Reliability and validity of a structured interview guide for the Hamilton anxiety rating scale (SIGH-A). Depression and Anxiety, 13 (2001), pp. 166-178. doi:10.1002/da.1033.
SC RI PT
[35] Heimberg, R. G., Horner, K. J., Juster, H. R., Safren, S. A., Brown, E. J., Schneier, F. R., & Liebowitz, M. R. Psychometric properties of the liebowitz social anxiety scale. Psychological Medicine, 29 (1999), pp.199-212.
[36] Cuijpers, P., Li, J., Hofmann, S. G., & Andersson, G. Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: A meta-
U
analysis. Clinical Psychology Review, 30 (2010), pp. 768-778. doi: 10.1016/j.cpr.2010.06.001.
N
[37] Moller, H. J. Rating depressed patients: Observer- vs self-assessment. European Psychiatry: The
A
Journal of the Association of European Psychiatrists, 15 (2000), pp.160-172. doi: S0924-
Annotated References (4)
D
M
9338(00)00229-7.
TE
Etkin, A., & Wager, T. D. Functional neuroimaging of anxiety: A meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. The American Journal of
EP
Psychiatry, 164 (2007), pp.1476-1488. doi:164/10/1476.
CC
This meta-analysis reviews concepts about emotion and examines fMRI studies of emotional processing in several anxiety disorders and provides evidence for the identification of emotion-related areas of the brain. Although it is not very recent, it expertly assesses a large body of imaging research and presents key anatomical areas for further inquiry.
A
Farias, M., Wikholm, C., & Delmonte, R. What is mindfulness-based therapy good for? The Lancet. Psychiatry, 3 (2016), pp. 1012-1013. doi: S2215-0366(16)30211-5.
This short paper, from a very high impact journal, is a great example of how readers of the medical literature encounter mindfulness-based treatments. The paper points out that “most of the studies had methodological limitations” and makes suggestions for future research.
Food Drug Administration Center for Drugs Evaluation Research (2018). Major Depressive Disorder: Developing Drugs for Treatment: Guidance for (FDA Maryland).
SC RI PT
This document is a great example of how scientists from regulatory bodies work to improve the quality of clinical science. Researchers who study mindfulness interventions for depression, for example, should know that there are already published standards for clinical research, and that many issues in clinical trial design have already been discussed and best recommendations set forth.
Goyal, M. et al. Meditation programs for psychological stress and well-being: a systematic review
and
meta-analysis.
JAMA
Intern
Med.
U
10.1001/jamainternmed.2013.13018.
2014
Mar;174(3):357-68.
doi:
A
CC
EP
TE
D
M
A
N
While not specifically focusing on mindfulness meditation, this excellent meta-analysis, published in a high impact medical journal, reviews effects of clinical trials of meditation treatments for stress-related outcomes (for example anxiety, depression, eating habits, sleep, and pain). The authors use strict quality criteria (the Evidence-based Practice Center Methods Guide) and point out helpful suggestions to improve the quality of research trials.
S2352-250X(18)30230-6 https://doi.org/10.1016/j.copsyc.2019.02.001 COPSYC 789
To appear in:
Please cite this article as: Hoge EA, Philip SR, Fulwiler C, Considerations for Mood and Emotion Measures in Mindfulness-Based Intervention Research, Current Opinion in Psychology (2019), https://doi.org/10.1016/j.copsyc.2019.02.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Considerations for Mood and Emotion Measures in Mindfulness-Based Intervention Research Elizabeth A. Hoge M.Da , Samantha R. Philip B.Sa, Carl Fulwiler M.D.. PhD.b a
Department of Psychiatry, Georgetown University Medical center, Washington, D.C. USA Departments of Psychiatry and Medicine, University of Massachusetts Medical School, Worcester, MA 06155 USA Elizabeth Hoge, MD Associate Professor of Psychiatry Georgetown University Medical Center 2115 Wisconsin Ave NW Washington, DC 20007 Phone (202) 687-0635 Fax (202) 687-2684 Email: [email protected]
U
Corresponding Author:
SC RI PT
b
N
Abstract A large and growing body of work has examined the effects of Mindfulness-Based Interventions
A
(MBI’s), such as Mindfulness-Based Stress Reduction and Mindfulness-Based Cognitive Therapy,
M
on emotion-related outcomes, both in mental health settings and general populations. These
D
studies vary widely in the approach to measurement of emotion-related measurements after MBI’s.
TE
A systematic review of randomized clinical trials of MBIs was conducted with a focus on identifying what emotion-related assays were able to detect changes with MBI’s, including scales
EP
and instruments (both self-report and clinician-rated) on constructs such as depression, anxiety,
CC
emotion regulation, and other mood states. In this paper we reflect on these findings and discuss considerations of outcome measures in MBI research. There are previously established practices
A
for clinical trials research on emotion-related outcomes which may provide some useful methodological standards and study design options for use by the MBI research field.
Introduction Measurement of emotion is not straightforward [1]. There is general agreement that emotions can be characterized as the subjective, physiological and behavioral responses to an event [2].
Accordingly, instruments for measuring each of these components of emotion have been developed. However, there are wide variations in correlation between subjective, physiological and behavioral measures, and each approach has its weaknesses. Unfortunately, there is no “gold
SC RI PT
standard” measure of emotion [3].
Many researchers examining the effects of Mindfulness-Based Interventions (MBIs) have an interest in emotion-related changes that occur over the course of mindfulness training . These effects are assessed with a wide variety of measures which range from current mood states to
U
symptoms of emotional disorders such as depression or anxiety disorders. Measures can be
N
clinician-rated, self-report, behavioral (how a person responds in a laboratory testing paradigm) or
A
physiological (electrophysiology, hormones, or immunological markers). The type of measure
M
depends on the study population, design, and the goals of the authors.
D
A systematic review of randomized clinical trials of MBI’s was conducted with a focus on
TE
identifying these measures, or assays, used to assess outcomes related to emotion (Kimmel et al,
EP
in preparation). We found that the most widely used type of measure for emotion outcomes was self-report (over 90%), with only about 5-10% of studies using either clinician-rated or
CC
physiological measures (and none using behavioral ones).
A
Self-report measures have significant limitations. These measures may be more valid for current emotional states than for emotional traits (or symptom severity) in general, and the symptoms they measure may not be specific (for example, overlap with common symptoms of a medical illness, such as racing heart). To be considered accurate, self-reports should ideally correlate with
objective measurements (clinician-rated, behavioral, or physiological) and/or functional outcome, but they often do not, or these data are not available. Studies examining the correlation between self-report scales and physiological or behavioral outcome measures would be an important
SC RI PT
contribution to the field.
Physiological measures for assessing emotion, such as autonomic activity, facial expression and central nervous system changes, are more objective but are not well established. In our systematic review, we found that less than 10% of the studies utilized physiological measures, despite
U
evidence that these measures may correlate to psychological states or emotion-related conditions.
N
For example, several studies have shown fMRI amygdala hyper-reactivity when viewing fearful Other research has looked at
A
or angry faces in patients with anxiety disorders ([4], [5]).
M
physiological reactions to emotionally provocative stimuli such as effect of International Affective Picture System (IAPS) pictures on skin conductance [6], the Trier Social Stress Test on stress [7],
or
individually-created
D
hormones
negative
narratives
about
oneself
on
TE
electroencephalographic activation, eye-blink startle reflux, and immune response [8]. Although
EP
some aspects of emotion are measurable, research has not yet identified anything close to a specific physiological fingerprint for specific emotions. Nevertheless, given the risk of bias associated with
CC
self-report instruments, physiological measures could serve as an objective way of measuring changes in distressful emotion-related outcomes, and the development of these techniques should
A
continue to be emphasized in future studies.
In light of this complexity in measuring emotion, we aim to examine the challenges faced specifically by MBI researchers when attempting to capture emotional outcomes of these
SC RI PT
interventions and offers recommendations to strengthen the body of research in this field.
Emotion-Related Measures Used in MBIs Depends on Study Aim
When determining which emotion-related assays to use in a study, researchers must consider the population of interest and the aims of the study. In our systematic review of emotional domain
U
outcomes in MBIs (primarily MBSR and MBCT, Kimmel et al., in preparation), we found that
N
studies that measured current mood states as primary outcomes (about 25% of the studies), such
A
as with the Positive and Negative Affect Schedule (PANAS) or the Profile of Mood States,
M
(POMS), often used a population that did not undergo diagnostic assessment, such as college students, nurses, etc. In this case, the primary aim of the studies was to assess whether an
D
experimental procedure leads to an observable change in current mood states.
TE
In contrast, we found that a majority (75%) of MBI work in emotion-related outcomes was
EP
in clinical samples --individuals with a diagnosed psychiatric or medical illness, for which the aim was to test whether the MBI was associated with a reduction in symptom burden (anxiety or
CC
depression) or medical illness-related stress (such as in diabetes, cancer, chronic pain, etc). The most common psychiatric disorders were depression and anxiety disorders, with approximately
A
35% of all the MBI research on just these two categories of disorders, evincing the considerable interest there is in using MBI’s to treat mental illness. The individual scales most commonly used for anxiety and depression were the State-Trait Anxiety Inventory (STAI), the Beck Anxiety Inventory (BAI), the Beck Depression Inventory II (BDI-II), and the Center of Epidemiologic
Studies Depression Scale (CES-D). Overall, our review found that about half of the time, the MBI group had significantly more change in anxiety and depression measures than the control group.
SC RI PT
If these anxiety and depression measures were originally designed for use with psychiatric patients; what are the implications of using them in non-psychiatric populations? The interpretation of results may be less clear. For example, if no change in depression symptoms is observed in an MBI trial, is this because the MBI was not effective in reducing depression symptoms, or because, if the study sample is from the general population, the symptoms were too low at baseline to have a
U
detectable change (“floor effect”)? Secondly, are items from scales designed for a disordered
N
population relevant for people with no psychiatric illness (e.g., what percent of the population has
A
“inappropriate guilt which may be delusional”? [9]. A study sample that contains unidentified
M
depressed and non-depressed individuals will be have greater variability and a more complicated analysis, and depression scales may be assessing symptoms mistakenly ascribed to depression (but
TE
D
are really due to a medical condition or some other cause).
EP
MBIs on Mood States and Emotion Regulation
CC
As mentioned earlier, many MBI studies are interested in how mindfulness interventions affect mood states. In our review, 15 studies assessed mood states and specific emotions, with 70% using
A
scales such as the POMS and the PANAS. The POMS, for example, assesses 6 different dimensions of mood (e.g., tension or anxiety, anger of hostility, vigor or activity, etc.), while the PANAS consists of two scales measuring positive affect and negative affect. Findings revealed that, over half of the time, the MBIs led to significantly more change in emotion and mood states
than the control group. More specifically, MBIs were generally associated with significant increased positive affect and significantly decreased negative affect, tension and anxiety, fatigue,
SC RI PT
anger, and total mood disturbance compared to the control group (Kimmel et al., in preparation).
Another common emotion-related construct of interest in MBI research is emotion regulation: an individual’s ability to adjust the intensity and/or duration of an emotion, or the presence of emotional states [10]. Poor emotion regulation can lead to worse health outcomes, as it impairs one’s ability to engage in self-regulating health behaviors related to diet, exercise, smoking, etc
U
[11, 12]. Thus, greater emotion regulation may facilitate more self-regulation behaviors and, in
N
turn, improved health outcomes. In our systematic review, 4 studies utilized emotion regulation
A
measures. The most commonly used measure was the Difficulties in Emotion Regulation Scale
M
(DERS), which assesses one’s self-reported ability to cope with negative emotions. The findings demonstrated that MBIs outperformed the control in almost 70% of the emotion regulation
TE
D
comparisons.
EP
Research with Clinical Populations: Depression & Anxiety
CC
Methodological Considerations
Because of the intense interest in MBIs for psychiatric populations, it is important that researchers
A
utilize rigorous clinical trial methodology that will ensure that findings are taken seriously by the medical community, health insurance payers and disease management companies.
Despite
hundreds of published trials on MBSR with promising results, the vast majority of health insurance companies do not cover this treatment.
General considerations for study design for behavioral interventions such as MBI’s will vary depending on the overarching aim of the study and the stage of research for the intervention.
SC RI PT
According to the Stage Model for Behavioral Intervention Development [13]), in Stage I of intervention development, researchers work towards generating a new intervention or modifying existing interventions and running pilot trials. In this stage, the aim could be to test the feasibility of the intervention and proof-of-concept, and a control group is not required. If no change in the outcome measure is detected after the intervention, researchers may need to go back to
U
development and modification of the intervention. Alternatively, the aim could be to test patient
N
acceptability of an intervention, the outcome measures would focus on treatment acceptability,
M
A
adherence, and satisfaction.
If a “signal” is detected (a change associated with the treatment), researchers can feel more
D
confident about moving into more advanced stages of methodological design, such as formally
TE
testing efficacy of the MBI and using an active control group. Efficacy studies are most robust
EP
when they use an active control group or placebo group to test if the effect of the intervention is due to the actual treatment or the non-specific (placebo) effects of treatment. Once sufficient
CC
evidence of efficacy is established, the next step to improve external validity could be to use a comparative effectiveness approach to test the intervention. Thus, the choice of study design
A
should be carefully selected based on the developmental stage of intervention.
MBI researchers conducting treatment trials for anxiety and depression should be aware of previously established standards in order to make their data relevant, accepted, and impactful. For
example, the US Food and Drug Administration demands the highest rigor in methodological standards from drug companies seeking an indication for their drug. Some of these practices are relevant for MBI research and it is therefore worthwhile to be aware of these established standards
SC RI PT
when considering design and procedures of MBI randomized, controlled trials (RCT’s) in order to make an impact in wider arenas including payors and managed care. In our systematic review of MBI research on emotion (Kimmel et al, in preparation), a significant number (more than 50%) of the included studies had design elements that suggested a high risk for bias, and less than half of the studies utilized an active control group. MBI researchers should be aware of the expectation
N
U
in the field for clinical trials.
M
A
Diagnostic Screening
In the above section we discuss the risks associated with not conducting formal diagnostic
D
screening for samples of individuals with anxiety and depression. In our systematic review of
TE
MBI research and emotion-related outcomes, nearly 60% of the studies using emotion-related
EP
measures did not utilize a standardized, diagnostic instrument designed to assess the presence of a depressive or anxiety disorder, despite the fact that many of them included anxiety and depression
CC
measures. Often, authors simply measure anxiety and depression symptom levels in a population that might be expected to experience these, such as university students, nurses, or those with a
A
chronic medical problem.
Authors should consider carefully whether the aim of the study is to correct an abnormal symptom, such as in a clinical population, or decrease negative emotions in a normal population, with whom
many of the depression and anxiety scales were not designed to be used. Authors need to think about which type of population they are targeting, so that they can use the most appropriate scales.
SC RI PT
Looking at standard clinical trials procedures in the medical literature, we see that the FDA guidance document for depression (2018) treatment trials states that: “diagnosis should be confirmed via a semi-structured interview such as the current Structured Clinical Interview for DSM or MINI International Neuropsychiatric Interview.” [9].
U
A related concern is that trials in which depression symptoms are measured, sometimes do not
N
screen for, or adequately address, suicidality. Several questionnaires do contain suicidality items,
A
such as the BDI-II and the Patient Health Questionnaire (PHQ-9), but not all do (such as the
M
CESD). Researchers who suspect depression symptoms or a depressive disorder in their populations should specifically ask about suicidal thinking, ideation, and planning, and provide
TE
D
appropriate clinical follow-up care and referral if risk is detected.
EP
The Problem of Comparison Groups
CC
One methodological issue that remains problematic is the question of a comparison group for an RCT on MBSR or MBCT. Although some have argued that traditional RCT study designs are not
A
relevant or possible for use with complementary and alternative medicine (CAM) therapies [15], wider audiences remain skeptical of outcomes without this trial component. Looking at industry standards, we see that the FDA guidance document for depression treatment trials, for example, states that “a placebo group is necessary to ensure that observed effects are not the result of
spontaneous improvement, expectation bias, attention from health care professionals involved in the trial, regression to the mean, or other factors not related to the activity of the study drug.” [14].
SC RI PT
Criticism has been raised concerning inadequate control groups in MBI research (for example see review for the Agency for Healthcare Research and Quality (AHRQ): [16, 17, 18, 19, 20] but suggested attention control conditions are problematic. For example, an effort to develop “plausible and therapeutic comparison conditions” [20] such as the health enhancement program (HEP) disqualifies that very comparison group because it has become too therapeutic for the
U
detection of a contrast. For example, HEP contains aerobic exercise, which has already been
N
shown to decrease anxiety, stress and depression symptoms [21, 22, 13]. If a hypothetical study
A
compares two active (therapeutic) treatments, such as HEP and MBSR, and both are found to be
M
mildly helpful (which we might expect given that they both have therapeutic elements), we still don’t know whether MBSR is effective—we can’t tell from this design whether both work, or both
D
don’t work and other factors resulted in the findings. If researchers want to compare two active
EP
TE
treatments, the most appropriate study design is non-inferiority.
CC
Non-inferiority
An alternative to utilizing an attention control comparison group is to use a therapeutic comparator
A
that has already been established to be effective, or a “gold standard” of treatment. This approach would also address the US Agency for Healthcare Research and Quality's concern that “special attention should be paid to developing studies that provide a more accurate assessment of the efficacy and effectiveness of meditation practices, both against standard therapies and against
each other” [24]. A 2-arm non-inferiority design could be considered when it would be unethical to have a placebo group receiving no treatment.
SC RI PT
In this alternative approach, the statistical comparison would be looking for ‘equivalence’ or, more accurately, ‘non-inferiority,’ rather than superiority to control (as in a typical RCT). Noninferiority trials are used when—despite the fact that there is already a proven treatment for a disorder—a new treatment is hypothesized to be useful (e.g., fewer side effects, faster onset of action). The non-inferiority design is a valid approach to prove efficacy. For example, it is included
U
as an acceptable design for companies seeking formal FDA approval for a drug [25]. In this design,
N
the statistical test concerns whether the new treatment is worse than the standard treatment by
A
some pre-determined ‘non-inferiority margin.’ [26, 27]. This is in contrast to a traditional,
NOT indicate equivalence.
M
superiority trial, in which finding no statistically significant difference between treatments does An example of how this approach could be used in mindfulness
D
research would be to compare mindfulness training (a newer treatment), with an established, gold-
TE
standard treatment (such as medication or psychotherapy) for major depressive disorder or anxiety
EP
disorders.
CC
The non-inferiority margin is calculated and declared before the start of the study using data from an earlier, gold-standard treatment’s efficacy study, thus incorporating that earlier study’s placebo
A
group data to determine this margin (that the non-inferiority margin is not too close to the placebo outcome). In this way, the new treatment is determined to be not significantly worse than the goldstandard treatment. Non-inferiority analyses utilize a separate set of statistical procedures and
typically a larger study sample is needed, which could be achieved with multiple study sites (for more information see [8, 25, 29]).
SC RI PT
Dose-Response Relationship
Another alternative to use of a placebo group is a dose-response trial. An example of a drug doseresponse trial would be this: a drug at 100 mg is found to have twice the clinical effect as the same drug at 50 mg, showing that the change is likely due to the treatment. This design has been used
U
in medication studies and with psychosocial treatment trials when the use of placebo is unfeasible
N
or unethical [30, 31, 32]. In mindfulness studies, this approach could be used in a design in which
A
twice the amount of training was compared to half that amount of time; if the effect is large in the
M
longer group and medium in the shorter group, this suggests that the training had an effect on the
D
outcome.
EP
TE
Clinician Ratings vs. Self-Report Ratings in Clinical Populations
Another important consideration in clinical trials is the type of symptom severity rating used as
CC
the primary outcome variable, as briefly discussed above. Turning to traditional standards-ofpractice, we see that the FDA guidance document for depression treatment trials states that
A
“clinician-rated outcome measures are the current standard for assessing efficacy in antidepressant trials. To date, the FDA has accepted the following as primary endpoints in phase 3 medication studies to support an MDD indication: Hamilton Depression Rating Scale (typically the 17-item version) Montgomery Asberg Depression Rating Scale.” In trials of anxiety disorders, the most
widely recognized clinician-rated scales, which are also used in FDA registry trials, are the Hamilton Anxiety Scale for generalized anxiety disorder (HAM-A, or the scripted version: Structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A) [33]; the Panic
SC RI PT
Disorder Severity Scale for panic disorder [34], and the Liebowitz Social Anxiety Scale for social anxiety disorder [35]. Utilizing traditional, widely-used instruments allows for comparison of findings with published trials across decades.
The preference for clinician ratings over self-report ratings of depression is supported by several
U
reviews and meta-analyses [36, 37]. For example, a meta-analysis of 48 studies including a total
N
of 2462 participants found that “clinician-rated instruments resulted in a significantly higher effect
A
size than self-report instruments from the same studies” [36]. Clinician ratings may assess global
M
disorder severity more accurately because of experience and training; in contrast, a patient’s perspective is in relation only to his/her own experience. There are other limitations of self-report
D
instruments, for example, a patient may have trouble completing instruments due to diminished
TE
attention or cognition. Or, ratings may be affected by social desirability influences or disease states
EP
which may distort perceptions or judgment. Despite strong evidence for the use of clinician-rated outcomes, our systematic review found that less than 10% of MBI studies with clinical samples
CC
used clinician-rated outcomes. MBI studies should aim to employ more clinician-rated outcomes in order to be in line with FDA guidance and to be taken seriously by the scientific community.
A
However, clinician-rated instruments should be carefully selected, as some can be problematic; some of them are decades old (for example, the HAM-D was originally developed in 1960) and the diagnostic criteria for anxiety and depression have changed with successive versions of the
DSM. Some have missing symptom categories, such as hypersomnia, or include symptoms that are no longer diagnostic features of the disorders.
SC RI PT
Self-report instruments also have an important role in clinical research [37]. For example, patientrated questionnaires can better assess the impact of symptoms on overall functioning, work impairment, and social functioning. Self-report measures also benefit from being free from biases from the clinician. Often, clinical trials use both clinician-rated and patient-rated instruments.
U
A recent NIH initiative to improve self-report measures used recent advances in information
N
technology, psychometrics, and qualitative, cognitive, and health survey research to produce the
A
NIH Toolbox (www.nihtoolbox.org) and the Patient Reported Outcomes Measurement
M
Information System (PROMIS; www.nihpromis.org). Given the known limitations of the selfreport instruments most widely used in the mindfulness literature, and the advances in assessment
D
represented by the PROMIS measures, we encourage MBI researchers to utilize these instruments
TE
for symptom assessment, and to continue exploring objective and functional to overcome the
EP
overcome the limitations of symptom assessment.
CC
Cost and Blinding
A
A few other considerations can help researchers choose between clinician-ratings and patient-rated instruments. One consideration—cost—can influence decisions about whether a primary measure, whether it is clinician-rated or self-rated. The Beck instruments (self-report Beck Depression Inventory and Beck Anxiety Inventory) require payment, whereas the MADRS (clinician rated
form) is free. Lastly, the use of clinician raters allows for some aspect of data collection to remain blinded (also called “masked”); that is, clinician raters are kept “blind” to treatment allocation and thus expectation effects are kept to a minimum. Given that MBI trials are impossible to conduct
SC RI PT
using standard double-blind procedures, having blinded clinical assessors provides some protection against expectation bias and could be considered a “single-blind” design.
Conclusion
U
There are many aspects to consider when designing a MBI study when emotion-related measures
N
are the primary outcome. For populations with a disorder, researchers should use diagnostic
A
instruments to ascertain presence of the disorder and consider clinician-rated instruments to
M
measure symptom change over time. Given the advances in self-report assessment represented by the PROMIS measures, we encourage MBI researchers to utilize PROMIS instruments for
D
symptom assessment when self-report outcomes are desired. Lastly, researchers should continue
TE
to explore objective, physiological measures to overcome the limitations of questionnaire-based
EP
symptom assessment.
A
CC
Author Declarations: No declarations. There are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all three named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing, we confirm that we have followed the regulations of our institutions concerning intellectual property. We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). We are responsible for submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible for the Corresponding Author.
D
TE
EP
CC
A
SC RI PT
U
N
A
M
References [1] Barrett, L. F. Navigating the science of emotion. In Meiselman H. L. (Ed.), Woodhead Publishing. (2016), pp. 31-63. doi://doi.org/10.1016/B978-0-08-100508-8.00002-3 [2] Nairne, J. S. Psychology: The Adaptive Mind. 2nd Ed. Wadsworth, 2000. p. 444.
pp. 209-237. doi:10.1080/02699930802204677.
SC RI PT
[3] Mauss, I. B., & Robinson, M. D. Measures of emotion: A review. Cognition & Emotion, 23 (2009),
[4] Fonzo, G. A., Ramsawh, H. J., Flagan, T. M., Sullivan, S. G., Letamendi, A., Simmons, A. N., . . . Stein, M. B. Common and disorder-specific neural responses to emotional faces in generalised anxiety, social anxiety and panic disorders. The British Journal of Psychiatry: The Journal of Mental Science,
N
U
206 (2015), pp. 206-215. doi:10.1192/bjp.bp.114.149880.
A
[5] Etkin, A., & Wager, T. D. Functional neuroimaging of anxiety: A meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. The American Journal of
M
Psychiatry, 164 (2007), pp.1476-1488. doi:164/10/1476.
visual,
auditory
and
haptic
stimuli. Scientific
Data,
5
(2018),
pp.
180120.
TE
to
D
[6] Gatti, E., Calzolari, E., Maggioni, E., & Obrist, M. Emotional ratings and skin conductance response
EP
doi:10.1038/sdata.2018.120.
[7] Hoge, E. A., Bui, E., Palitz, S. A., Schwarz, N. R., Owens, M. E., Johnston, J. M., . . . Simon, N. M.
CC
The effect of mindfulness meditation training on biological acute stress responses in generalized anxiety disorder (2018). doi://doi.org/10.1016/j.psychres.2017.01.006.
A
[8] Rosenkranz, M. A., Jackson, D. C., Dalton, K. M., Dolski, I., Ryff, C. D., Singer, B. H., . . . Davidson, R. J. Affective style and in vivo immune response: Neurobehavioral mechanisms. Proceedings of the National Academy of Sciences of the United States of America, 100 (2003), pp. 11148-11152. doi:10.1073/pnas.1534743100.
[9] American Psychiatric Association., American Psychiatric Association. DSM-5 Task Force.,. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Arlington, VA: American Psychiatric Association.
SC RI PT
[10] Gross, J. J. The emerging field of emotion regulation: An integrative review. Reviews of General Psychology. 2 (1998), pp. 271-299.
[11] Smyth, J. M., & Arigo, D. Recent evidence supports emotion-regulation interventions for improving health in at-risk and clinical populations. Curr Opin Psychiatry. 22(2009), pp. 205-
U
210. doi:10.1097/YCO.0b013e3283252d6d
N
[12] Wallis, D. J., & Hetherington, M. M. Emotions and eating. Self-reported and experimentally
M
doi:10.1016/j.appet.2008.11.007
A
induced changes in food intake under stress. Appetite. 52(2009), pp. 355-362.
D
[13] Onken, L., Carroll, K., Shoham, V., Cuthbert, B., & Riddle, M. Reinvisioning clinical science:
TE
Unifying the discipline to improve the public health. Clinical Psychological Science, 2 (2014), pp.
EP
22–34. doi: 10.1177/2167702613497932. [14] Food Drug Administration Center for Drugs Evaluation Research (2018). Major Depressive Disorder:
CC
Developing Drugs for Treatment: Guidance for (FDA Maryland).
A
[15] Forbes, B. Yoga and managed care: A cautionary tale. International Journal of Yoga Therapy, 20 (2010), pp. 22-25. doi:10.17761/ijyt.20.1.t873831731361401.
[16] Ospina, M. B., Bond, K., Karkhaneh, M., Tjosvold, L., Vandermeer, B., Liang, Y., . . . Klassen, T. P. Meditation practices for health: State of the research. Evidence Report/Technology Assessment, (155) (2007), pp.1-263.
[17] Ospina, M. B., Bond, K., Karkhaneh, M., Buscemi, N., Dryden, D. M., Barnes, V., . . . ShannahoffKhalsa, D. Clinical trials of meditation practices in health care: Characteristics and quality. Journal of Alternative and Complementary Medicine (New York, N.Y.), 14 (2008), pp. 1199-
SC RI PT
1213. doi:10.1089/acm.2008.0307. [18] Farias, M., Wikholm, C., & Delmonte, R. What is mindfulness-based therapy good for? The Lancet. Psychiatry, 3 (2016), pp. 1012-1013. doi: S2215-0366(16)30211-5.
[19] Davidson, R. J., & Kaszniak, A. W. Conceptual and methodological issues in research on mindfulness
and
meditation.
American
Psychologist,
(2015),
pp.
581-592.
U
doi:10.1037/a0039512.
70
N
[20] Goldberg, S. B., Tucker, R. P., Greene, P. A., Simpson, T. L., Kearney, D. J., & Davidson, R. J. Is
M
(2017). doi: 10.1371/journal.pone.0187298.
A
mindfulness research methodology improving over time? A systematic review. PloS One, 12
D
[21] MacCoon, D. G., Imel, Z. E., Rosenkranz, M. A., Sheftel, J. G., Weng, H. Y., Sullivan, J. C., . . . Lutz, A. The validation of an active control intervention for mindfulness based stress reduction
TE
(MBSR). Behaviour Research and Therapy, 50 (2012), pp. 3-12. doi: 10.1016/j.brat.2011.10.011.
EP
[22] Harvey, S. B., Overland, S., Hatch, S. L., Wessely, S., Mykletun, A., & Hotopf, M. Exercise and the prevention of depression: Results of the HUNT cohort study. The American Journal of Psychiatry,
CC
175 (2018), pp. 28-36. doi: 10.1176/appi.ajp.2017.16111223.
A
[23] Asmundson, G. J., Fetzner, M. G., Deboer, L. B., Powers, M. B., Otto, M. W., & Smits, J. A. Let's get physical: A contemporary review of the anxiolytic effects of exercise for anxiety and its disorders. Depression and Anxiety, 30 (2013), pp. 362-373. doi:10.1002/da.22043.
[24] Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, Publication No. 07-E010 (2007).
[25] 21 Code of Federal Regulations, 314.126. [26] Streiner, D. L. Alternatives to placebo-controlled trials. The Canadian Journal of Neurological Sciences.Le Journal Canadien Des Sciences Neurologiques, 34 Suppl 1 (2007), pp. 37.
SC RI PT
[27] Leon, A. C. Evolution of psychopharmacology trial design and analysis: Six decades in the making. The Journal of Clinical Psychiatry, 72 (2011), pp. 331-340. doi:10.4088/JCP.10r06669.
[28] D'Agostino RB, S., Massaro, J. M., & Sullivan, L. M. Non-inferiority trials: Design concepts and issues - the encounters of academic consultants in statistics. Statistics in Medicine, 22 (2003), pp.
U
169-186. doi:10.1002/sim.1425.
N
[29] FDA Non-Inferiority Clinical Trials to Establish Effectiveness Guidance for Industry (2016).
September
26,
1996,
Amsterdam.
European
College
of
M
meeting,
A
[30] Montgomery, S. A. Alternatives to placebo-controlled trials in psychiatry. ECNP consensus
Neuropsychopharmacology. European Neuropsychopharmacology: The Journal of the European
D
College of Neuropsychopharmacology, 9 (1999), pp.265-269. doi: S0924977X98000492.
TE
[31] Delgadillo, J., McMillan, D., Lucock, M., Leach, C., Ali, S., & Gilbody, S. Early changes, attrition, and dose-response in low intensity psychological interventions. The British Journal of Clinical
EP
Psychology, 53 (2014), pp. 114-130. doi:10.1111/bjc.12031.
CC
[32] Dinan, T. G. Lithium augmentation in sertraline-resistant depression: A preliminary doseresponse study. Acta Psychiatrica Scandinavica, 88 (1993), pp. 300-301.
A
[33] Shear, M. K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., . . . Wang, T. Reliability and validity of the panic disorder severity scale: Replication and extension. Journal of Psychiatric Research, 35 (2001), pp. 293-296. doi: S0022-3956(01)00028-0.
[34] Shear, M. K., Vander Bilt, J., Rucci, P., Endicott, J., Lydiard, B., Otto, M. W., . . . Frank, D. M. Reliability and validity of a structured interview guide for the Hamilton anxiety rating scale (SIGH-A). Depression and Anxiety, 13 (2001), pp. 166-178. doi:10.1002/da.1033.
SC RI PT
[35] Heimberg, R. G., Horner, K. J., Juster, H. R., Safren, S. A., Brown, E. J., Schneier, F. R., & Liebowitz, M. R. Psychometric properties of the liebowitz social anxiety scale. Psychological Medicine, 29 (1999), pp.199-212.
[36] Cuijpers, P., Li, J., Hofmann, S. G., & Andersson, G. Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: A meta-
U
analysis. Clinical Psychology Review, 30 (2010), pp. 768-778. doi: 10.1016/j.cpr.2010.06.001.
N
[37] Moller, H. J. Rating depressed patients: Observer- vs self-assessment. European Psychiatry: The
A
Journal of the Association of European Psychiatrists, 15 (2000), pp.160-172. doi: S0924-
Annotated References (4)
D
M
9338(00)00229-7.
TE
Etkin, A., & Wager, T. D. Functional neuroimaging of anxiety: A meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. The American Journal of
EP
Psychiatry, 164 (2007), pp.1476-1488. doi:164/10/1476.
CC
This meta-analysis reviews concepts about emotion and examines fMRI studies of emotional processing in several anxiety disorders and provides evidence for the identification of emotion-related areas of the brain. Although it is not very recent, it expertly assesses a large body of imaging research and presents key anatomical areas for further inquiry.
A
Farias, M., Wikholm, C., & Delmonte, R. What is mindfulness-based therapy good for? The Lancet. Psychiatry, 3 (2016), pp. 1012-1013. doi: S2215-0366(16)30211-5.
This short paper, from a very high impact journal, is a great example of how readers of the medical literature encounter mindfulness-based treatments. The paper points out that “most of the studies had methodological limitations” and makes suggestions for future research.
Food Drug Administration Center for Drugs Evaluation Research (2018). Major Depressive Disorder: Developing Drugs for Treatment: Guidance for (FDA Maryland).
SC RI PT
This document is a great example of how scientists from regulatory bodies work to improve the quality of clinical science. Researchers who study mindfulness interventions for depression, for example, should know that there are already published standards for clinical research, and that many issues in clinical trial design have already been discussed and best recommendations set forth.
Goyal, M. et al. Meditation programs for psychological stress and well-being: a systematic review
and
meta-analysis.
JAMA
Intern
Med.
U
10.1001/jamainternmed.2013.13018.
2014
Mar;174(3):357-68.
doi:
A
CC
EP
TE
D
M
A
N
While not specifically focusing on mindfulness meditation, this excellent meta-analysis, published in a high impact medical journal, reviews effects of clinical trials of meditation treatments for stress-related outcomes (for example anxiety, depression, eating habits, sleep, and pain). The authors use strict quality criteria (the Evidence-based Practice Center Methods Guide) and point out helpful suggestions to improve the quality of research trials.