- Email: [email protected]
Investigating the latent structure of distress intolerance
Author’s Accepted Manuscript Investigating the latent structure of distress intolerance Kimberly T. Stevens, Sarah J. Kertz, Thröstur Björgvinsson, R. Kathryn McHugh www.elsevier.com/locate/psychres
PII: DOI: Reference:
S0165-1781(17)30382-7 http://dx.doi.org/10.1016/j.psychres.2017.09.036 PSY10847
To appear in: Psychiatry Research Received date: 2 March 2017 Revised date: 7 August 2017 Accepted date: 12 September 2017 Cite this article as: Kimberly T. Stevens, Sarah J. Kertz, Thröstur Björgvinsson and R. Kathryn McHugh, Investigating the latent structure of distress intolerance, Psychiatry Research, http://dx.doi.org/10.1016/j.psychres.2017.09.036 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Investigating the latent structure of distress intolerance Kimberly T. Stevensa*, Sarah J. Kertza, b, Thröstur Björgvinssonb, & R. Kathryn McHughb a
Psychology Department, Southern Illinois University, Carbondale, IL, USA b
McLean Hospital/Harvard Medical School, Belmont, MA, USA
*Corresponding Author: Kimberly T. Stevens, Southern Illinois University, Mail code 6502 Carbondale IL 62901. E-mail: [email protected]. Phone: 618.453.3572.
Abstract Distress intolerance (DI) is defined as a perceived or actual inability to withstand distressing emotional or somatic states, which motivates the use of avoidance strategies. Despite widespread interest in DI, key questions about its underlying structure remain unanswered. The current study evaluated the latent structure of DI in two large samples using four-indicators and three taxometric procedures (MAMBAC, MAXEIG, and L-Mode). Data interpretation relied primarily on the Comparison Curve Fit Indices (CCFI). Overall, results from the three nonredundant procedures suggested that DI was more accurately characterized by a dimensional rather than a categorical conceptualization. Implications for assessment and conceptual models of DI are discussed.
Keywords: taxometric, distress intolerance, latent structure, dimensional models
1. Introduction Accumulating evidence suggests that many psychological constructs are dimensional in nature (Carleton et al., 2012; Holm-Denoma, Richey, & Joiner, 2010; Liu, Jones, & Spirito, 2015; Olatunji, Broman-Fulks, Bergman, Green, & Zlomke, 2010; Timpano et al., 2013). Accordingly, much attention is shifting to dimensional systems of psychological disorder classification (e.g., T. A. Brown & Barlow, 2009; Widiger, Livesley, & Clark, 2009) and the study of mental illness across a range of functional dimensions (Insel et al., 2010). Despite a movement towards dimensional models, the field currently relies on categorical classification systems such as the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5; American Psychiatric Association, 2013) and the International Statistical Classification of Diseases, 10th revision (ICD-10 World Health Organization, 1992) for characterizing mental illness. Statistically, there are costs and benefits of using a categorical versus dimensional classifications systems; however, dimensional prediction outperforms categorical in most cases (Grove, 1991). Understanding the latent structure of psychological constructs is critical to understanding whether they are optimally conceptualized according to dimensions or categories and therefore may inform measurement and assessment in both clinical and research settings. Distress intolerance (DI), defined as a perceived or actual inability to withstand distressing emotional or somatic states, motivates the use of avoidance strategies (Simons & Gaher, 2005; Trafton & Gifford, 2011). DI is elevated among those with psychological disorders relative to those without (e.g., Corstorphine, Mountford, Tomlinson, Waller, & Meyer, 2007; Gratz, Rosenthal, Tull, Lejuez, & Gunderson, 2006; McHugh & Otto, 2011; Schmidt, Richey, & Fitzpatrick, 2006), and predicts poorer outcomes within clinical samples, such as a higher likelihood of relapse or treatment discontinuation among those with substance use disorders (R.
A. Brown, Lejuez, Kahler, & Strong, 2002; Daughters, Lejuez, Bornovalova, et al., 2005; Daughters, Lejuez, Kahler, Strong, & Brown, 2005). Additionally, DI is associated with maladaptive behaviors even in unselected populations (e.g., Anestis, Selby, Fink, & Joiner, 2007; Beck, Daughters, & Ali, 2013; Harrington, 2005a), highlighting its applicability across the range of psychological functioning. In addition to its role in predicting poor outcomes and characterizing clinical samples, the reduction of DI may present as an important mechanism of treatment for a range of disorders. DI has long been conceptualized as a target of psychological treatments (Hayes, Wilson, Gifford, Follette, & Strosahl, 1996; Linehan, 1993), and can be successfully reduced in treatment, with greater reductions associated with corresponding improvement in clinical outcomes (Bornovalova, Gratz, Daughters, Hunt, & Lejuez, 2012; McHugh et al., 2014; Norr, Allan, Macatee, Keough, & Schmidt, 2014). Despite widespread interest in DI, key questions about its underlying structure remain unanswered. Studies examining the latent factor structure of DI have suggested either a single latent factor or a single higher-order factor, which subsumes distress-specific DI factors (e.g., Bernstein, Zvolensky, Vujanovic, & Moos, 2009; Leyro, Bernstein, Vujanovic, McLeish, & Zvolensky, 2011; McHugh & Otto, 2012). Most studies have operated under the assumption that DI is a dimensional construct characterized by quantitative, rather than qualitative, differences along a continuum; however, there have been no empirical studies to date to support this position and much of the literature on DI is housed in separate disorder-specific literatures, despite evidence of transdiagnostic utility (McHugh & Otto, 2011). Further, though there is an assumption of dimensionality, some measurement strategies have continued to employ dichotomous scoring (e.g., Ehrlich, Cassidy, Gorka, Lejuez, & Daughters, 2013; Tull & Gratz,
2013). Better understanding the underlying structure of DI may enhance rationale for modifying the assessment of this important construct, and may improve conceptualization and intervention efforts across diagnostic categories. Although there is a growing body of evidence for the dimensionality of many psychological constructs, studies of the latent structure of some psychological constructs have yielded mixed results. For example, suicidality is often theorized as a symptom appearing on a continuum. One study using a sample of depressed adolescents (Liu et al., 2015) found support for a dimensional conceptualization; however, another study in a sample of adults, found that high risk for suicide was categorically distinct from low risk for suicide (Witte, Holm-Denoma, Zuromski, Gauthier, & Ruscio, 2017). In addition, findings for the latent structure of depression and bipolar symptoms are mixed with some supporting a dimensional structure (Liu, 2016; Prisciandaro & Tolliver, 2015), and others finding taxonic structure (Ahmed, Green, Clark, Stahl, & McFarland, 2011). Lastly, social anxiety disorder symptoms have suggested a dimensional structure in some studies (Kollman, Brown, Liverant, & Hofmann, 2006), but taxonic structure in others (Weeks, Carleton, Asmundson, McCabe, & Antony, 2010). These mixed findings support the need for empirical evaluation, even for constructs that are theoretically conceptualized as dimensional. There are also existing taxometric investigations of factors related to DI, or distressspecific DI factors (e.g., intolerance of uncertainty and anxiety sensitivity). For example, intolerance of uncertainty has demonstrated a dimensional latent structure in an initial assessment (Carleton et al., 2012). However, existing evidence for anxiety sensitivity has been mixed across adult and youth samples. Some work has supported a taxonic structure (e.g., Bernstein, Zvolensky, Kotov, et al., 2006; Bernstein, Zvolensky, Stewart, & Comeau, 2007;
Bernstein, Zvolensky, Weems, Stickle, & Leen-Feldner, 2005; Bernstein, Zvolensky, Stewart, Nancy Comeau, & Leen-Feldner, 2006), while other work has supported a dimensional structure (Broman-Fulks et al., 2008, 2010; Taylor, Rabian, & Fedoroff, 1999). Discrepant findings for the latent structure of constructs related to DI provide additional justification for examining the underlying structure of this presumed dimensional construct. Simons and Gaher’s (2005) proposed four dimensions of DI: tolerability/aversiveness, appraisal/acceptability, absorption of attention/disruption of functioning, and the action tendency to escape/avoid distress. Thus, this definition captures both cognitive (perception/interpretation of distress) and behavioral (tendency to escape/avoid distress) elements of this construct. In the current study, our aim was to examine the latent structure of DI using indicators representative of four DI dimensions (based on empirical and theoretical considerations). Indicators were drawn from validated self-report measures and measured in two large samples. 2. Methods 2.1 Participants 2.1.1 Sample 1. Participants included 392 patients from the [hospital name removed for masked review] who received treatment between July 2010 and July 2012. The [hospital name removed for masked review] provides individual and group cognitive behavioral therapy for patients with anxiety and/or mood disorders. The study sample was primarily female (57.7%, n = 226) and the average age was 30.0 years (SD = 14.2). The sample was primarily non-Hispanic White (89.8%), with 4.6% Asian, 2.8% Hispanic/Latino/a, 1.5% Black/African American, 0.8% American Indian/Alaskan Native, and 0.5% Native Hawaiian/Pacific Islander; 3.6% declined to answer. The sample was highly educated, with 35.6% reporting some college education, 31.7% post-undergraduate education, 23.7% a 4-year college degree, 8.0% high school graduate, 0.8%
some high school, and 0.3% 8th grade or less. Most participants were never married (63.7%); however, 24.3% were married, 8.4% separated/divorced, 1.8% widowed, and 1.8% living with a partner. 2.1.2. Sample 2. Participants were recruited from community-based resources (e.g., Craigslist, flyers in public areas) and an urban outpatient clinic specializing in anxiety disorder treatment (using flyers in clinic common spaces). The sample consisted of 300 unselected participants (i.e., adults from the community) and 100 participants recruited from an outpatient clinic to obtain a range of psychiatric functioning. Of note, taxonicity can be detected in sample sizes as low as 100; however, sample sizes of at least 300 are recommended for taxometric procedures (Ruscio, Walters, Marcus, & Kaczetow, 2010). The study sample was primarily female (72.3%) and the average age was 34.5 years (SD = 13.5). The sample was primarily Caucasian (84.0%), with 5.8% Asian, 5.0% African American, 4.3% other ethnicity, and 0.3% Native Hawaiian or Pacific Islander; 0.8% declined to answer. The sample was highly educated, with 43.3% with 4-year college degrees, 27.5% with graduate degrees, 24.5% with some college education, and 4.0% high school graduates; 0.8% declined to answer. 2.2. Measures 2.2.1. Distress Intolerance Index (McHugh & Otto, 2012). Participants in Sample 1 completed the DII, a 10-item self-report content-independent measure of DI. The measure derived items from the Distress Tolerance Scale (Simons & Gaher, 2005), Frustration Discomfort Scale (Harrington, 2005b); and Anxiety Sensitivity Index (Peterson & Reiss, 1992). Items were rated on a scale from 0 (very little) to 4 (very much). The DII has demonstrated excellent internal consistency, strong reliability, and initial concurrent validity with self-report and behavioral measures of DI (McHugh & Otto, 2012). Four DII items adopted from the
Distress Tolerance Scale were selected as indicators for the current study. The indicators evidenced strong internal consistency (α = .82). 2.2.2. Distress Tolerance Scale (DTS; Simons & Gaher, 2005). Sample 2 completed the DTS, a 15-item self-report measure of distress tolerance. The DTS has demonstrated strong internal consistency, reliability, and validity in unselected and clinical samples (Simons & Gaher, 2005). The same four items from Sample 1 were chosen as indicators, which also demonstrated strong internal consistency in Sample 2 (α = .82). 2.3. Procedure 2.3.1. Sample 1. The current study was approved by the hospital’s Institutional Review Board. Participants provided informed consent before participating, and all data were handled in accordance with the ethical guidelines provided by the American Psychological Association. Each participant completed an initial intake assessment including a demographics survey, battery of self-report questionnaires (also completed upon discharge), and the MINI structured diagnostic interview (Sheehan et al., 1998). 2.3.2. Sample 2. This study was approved by the university’s Institutional Review Board. All procedures were conducted electronically via a web-based survey tool. Participants provided informed consent within the web-based program, and those who agreed to participate then completed a brief battery of questionnaires assessing DI and related constructs. 2.4. Data analytic plan Taxometric and comparison data simulations were computed using Ruscio and Wang’s (2017) taxometric programs for R (available at https://cran.rproject.org/web/packages/RTaxometrics/index.html). Extensive support materials are available at http://ruscio.pages.tcnj.edu/taxometrics/. Three taxometric procedures were used to provide
non-redundant, consistent evidence of the latent structure of DI (Ruscio et al., 2010). The current study computed the Mean Above Minus Below a Cut (MAMBAC; Meehl & Yonce, 1994), maximum eigenvalue (MAXEIG; Waller & Meehl, 1998), and latent-mode factor analysis (LMode; Waller & Meehl, 1998) procedures. We used the following code: RunTaxometrics(data, seed = 1, n.pop = 1e+05, n.samples = 100, reps = 1,MAMBAC = TRUE, assign.MAMBAC = 1, n.cuts = 50, n.end = 25, MAXEIG =TRUE, assign.MAXEIG = 1, windows = 50, overlap = 0.9, LMode = TRUE, mode.l =-0.001, mode.r = 0.001, MAXSLOPE = FALSE) Interpretation of results relied primarily on the Comparison Curve Fit Indices (CCFI). CCFI scores were developed to provide objective estimates of fit between the simulated data sets and the research data (Ruscio, Ruscio, & Meron, 2007). CCFI values range from 0 to 1. A score of 0 strongly indicates support for a dimensional model and a score of 1 strongly supports a categorical model. A score of .5 represents ambiguous latent structure, so values between .45 and .55 should be cautiously interpreted. Underlying structure is best identified at CCFI values greater than .55 or less than .45 with an accuracy of .90 (Ruscio et al., 2010). Several simulation studies have indicated that the CCFI distinguishes between categorical and dimensional data, even when data are less than ideal (Ruscio, Ruscio, & Carney, 2011). Mean CCFI scores across the three procedures were used to evaluate the underlying structure. Graphical representations of dimensional and categorical simulated comparison data sets were also computed for each sample in order to visualize the MAMBAC and MAXEIG curves (Ruscio, Haslam, & Ruscio, 2006a). 2.4.1. MAMBAC taxometric method. The MAMBAC procedure is used to detect taxonicity through the use of quantitative indicators examining the means above and below a sliding cut score (Ruscio et al., 2006a). MAMBAC relies on the assumption that if a construct
has a taxonic latent structure, then there will be a cut score that distinguishes groups. Cut scores are identified across the range of the input variable as a function of the second indicator (output variable). Thus, the MAMBAC identifies a group above and below the cut score for a series of values of the input variable. The mean of cases falling below the cut score is subtracted from the mean of cases falling above the cut score. Mean differences are graphed, resulting in curved distributions. Fifty equally spaced cuts were used along the input indicator beginning at 25 cases from either extreme. If the construct has a taxonic structure, the MAMBAC curves will appear peaked at the point that best distinguishes the two groups. A construct with a dimensional latent structure yields flat or concave shaped MAMBAC curves, which suggests that there is no cut score that classifies cases. Default settings (including variables used in all input-output pairings [assign.MAMBAC = 1], cuts starting and ending at 25 points from either extreme [n.end = 25], and 50 total cuts [n.cuts = 50]) were used in the analyses. 2.4.2. MAXEIG taxometric method. The MAXEIG is a multivariate generalization of the maximum covariance (MAXCOV; Meehl, 1973) procedure. MAXEIG is conceptually and mathematically similar to MAXCOV but differs in several ways (see Ruscio, Haslam, & Ruscio, 2006b for details). The procedure provides an estimate of the conditional eigenvalues between two or more output indicators for a series of subsamples ordered along the range of a third indicator (input variable; Ruscio et al., 2006a), which is then graphed in order along the input variable, resulting in curved distributions. A smooth distribution indicates a dimensional structure, whereas a distribution with a peaked curve suggests taxonic structure. The input variable in the current study was the sum of the remaining two indicators. Analyses used 50 windows with a 0.9 overlap and n = 66 per window. Default settings (each variable serving as an
input variable once [assign.MAXEIG = 1] and overlapping windows set at .90 [overlap = .90]) were used. 2.4.3. L-Mode taxometric method. The L-Mode procedure aggregates indicators using exploratory factor analysis on the covariance between indicators (Ruscio et al., 2006a). The LMode graph produces a factor score density plot with factor scores along the x-axis and frequency along the y-axis. Dichotomous latent structure yields a bimodal L-Mode distribution, whereas a dimensional latent structure yields a unimodal distribution. Default settings (searching for the left mode beyond -.001 [mode.l = -.001] and searching for the right mode beyond .001 [mode.r = .001]) were used in analyses. 2.4.4. Indicator selection and adequacy. As described above, four indicators were used in the analyses. Items included "I can't handle feeling distressed or upset,” "My feelings of distress or being upset scare me,” “I'll do anything to stop feeling distressed or upset,” and “When I feel distressed or upset, I cannot help but concentrate on how bad the distress actually feels." These items were selected to represent the relevant facets of DI (Simons & Gaher, 2005) and to reduce within-group indicator correlation, based on Ruscio, Haslam, and Ruscio (2006a)’s suggestions. Although a number of studies have suggested some variability across types of distress, studies have consistently suggested a single latent higher-order DI factor (Bernstein et al., 2009; Leyro et al., 2011; McHugh & Otto, 2012), and thus our selection of indicators was informed by efforts to capture the relevant facets of DI in a way that is general to distress type (i.e., distress or upset rather than a specific emotion). Further, Walters and Ruscio (2009) recommended selecting indicators that have four or more ordered categories. The range of the indicators selected in the current study (1-5) is appropriate for traditional taxometric analyses.
The adequacy of the indicators was assessed using validity estimates, indicator distributions, and indicator correlations in the full sample as well as the putative taxon and complement groups. Across each procedure, the indicators demonstrated appropriate validity, Cohen’s d > 1.25 (Meehl, 1995; see Table 1). Indicator distributions suggested tolerable levels of skew, Mskew = -.27 (SD = .22), Mskew = .53 (SD = .27), and kurtosis, Mkurtosis = -.93 and Mkurtosis = -.87, respectively for each sample. It is also recommended that correlations between indicators be around, r < .30 (Meehl, 1995), and be higher in the full sample compared to the putative taxon and complement groups. In both samples, indicator correlations were larger in the full sample than in the taxon or complement groups (see Table 1 for a summary of indicator correlations). Within-group correlations for each group were within the recommended range. Overall, estimates indicated that the indicators were valid, their distributions were roughly normal, and indicator correlations were higher in the full sample compared to the groups. Therefore, the indicators were judged as appropriate and adequate for further taxometric analyses. 3. Results Overall results for Sample 1 from the three procedures suggested a dimensional structure. The CCFI was .20 for the MAMBAC procedure, .32 for the MAXEIG procedure, and .47 for LMode. The mean CCFI across procedures was .33. Results from Sample 2 were also indicative of dimensional structures. The CCFI was .16 for the MAMBAC procedure, .36 for MAXEIG, and .34 for L-Mode. The mean CCFI across the three procedures was .29. The panel of curves are presented in Figure 1 (Sample 1) and Figure 2 (Sample 2) for desriptive purposes. The panel on the left represents siimulated categorical comparison data and the panel on the right represents simulated dimensional comparison data. Averaged indicator curve values from the current study are illustrated by the bolded black line. The thin black lines
represent the minimum and maximum simulated values at each data point and the gray band represents the middle 50% of simulated values. For Sample 1, the average curve appeared more consistent with the simulated dimensional curve compared to the categorical comparison curve. Similarly, the average curve for Sample 2 appeared more consistent with the simulated dimensional comparison data. 4. Discussion Although DI is a well-established transdiagnostic vulnerability factor, there has been relatively little research focused on its underlying structure. The current study used taxometric analyses to investigate the extent to which the construct is better understood as dimensional or categorical in nature, using both mixed (i.e., both unselected and clinical) and clinical samples. Overall, results from three non-redundant analyses suggested that DI was more accurately characterized by a dimensional rather than a categorical conceptualization. These findings are consistent with a growing body of evidence supporting a dimensional latent structure of psychological symptoms (Carleton et al., 2012; Holm-Denoma et al., 2010; Liu et al., 2015; Olatunji et al., 2010; Timpano et al., 2013). Although this is the first known study to examine the latent structure of DI, most empirical studies have assumed a dimensional underlying structure by measuring DI continuously and testing its linear associations with outcomes (e.g., Banducci, Bujarski, BonnMiller, Patel, & Connolly, in press; McHugh et al., 2014). The exception is with behavioral measures of DI, in which quantitative outcomes have been dichotomized in some studies (e.g., Gratz et al., 2006; Nugent, Chiappelli, Rowland, Daughters, & Hong, 2014). Behavioral measures of DI employ goal-driven tasks that induce distress and measure the time to discontinuation as an index of DI. These measures can be scored either continuously (time to
discontinuation) or dichotomously (persist until the maximum time or discontinue prior to that time). Findings from the current study suggest that a dimensional conceptualization and corresponding continuous measurement may more accurately reflect the underlying construct. However, self-report and behavioral measures of DI are not consistently highly correlated, and thus the degree to which these approaches measure separate constructs has been debated (McHugh et al., 2011). Future studies utilizing multiple methods—including both self-report and behavioral measures—may improve understanding of the latent structure of DI. This may further elucidate whether these methods capture different constructs, or different facets of DI. Results from the current study have implications for the assessment of DI. These results suggest that optimal measures of DI will be continuous. Interestingly, a recent meta-analysis found that continuous measures of psychopathology were on average 15% more reliable and 37% more valid than discrete measures, regardless of construct type (Markon, Chmielewski, & Miller, 2011). The authors proposed that unless there is a specific rationale for using discrete measures, continuous measures of psychopathology are more likely to show greater associations with the construct of interest. As such, current self-report measures of DI, including the Distress Tolerance Scale and the Distress Intolerance Index, appear to be well suited to assess the construct. Dichotomized behavioral measures, however, may be improved by using a continuous indicator (e.g., number of seconds of distress endured). Given evidence for ceiling effects or truncated ranges of scores in these measures, which at times necessitates dichotomous scoring (Ehrlich et al., 2013; Tull & Gratz, 2013), consideration of how to enhance the range of scores on behavioral measures is needed. Nonetheless, studies examining the underlying structure of DI using a multi-method approach is needed to better understand the construct captured by behavioral measures of DI.
Understanding DI as a dimensional construct also has important implications for developmental models. Dimensional constructs are thought to develop from interactive and additive effects of multiple factors (Haslam, Holland, & Kuppens, 2012; Meehl, 1977). Identifying factors that contribute to the development of DI may provide insight into its underlying mechanisms and potentially allow for early intervention or prevention efforts to alter negative trajectories during critical developmental periods. Unfortunately, studies of putative risk or causal factors for the development of DI are lacking. Such factors are likely important as they would contribute directly to increased DI, but may also serve as proxy risk factors for the development of other psychiatric symptoms or disorders. Proxy risk factors are those that may not have a direct influence on the outcome variable (e.g., substance misuse, self-injurious behavior, etc.) but increase the likelihood of the outcome by their influence on the causal risk factor (i.e., DI). Thus, given the transdiagnostic relevance of DI, factors associated with increased likelihood of developing high DI may also serve as proxy risk factors for a wide range of symptoms and maladaptive behaviors. The current study had several limitations, which may have influenced findings or limit generalizability to other samples. First, the current study focused on DI for negative emotions. Results may not generalize to other facets of DI (e.g., uncertainty, pain, etc.). Additional study of other forms of DI in various samples is needed. Second, the study samples were not fully representative of demographic variability. Future work including participants from a broader range of racial and ethnic backgrounds and socioeconomic status would provide further support for the generalizability of the findings. Third, the data were limited to self-report measures of perceived DI and did not include any behavioral indices of DI. Although it was previously thought that that the nature of the data submitted to taxometric analyses may influence outcome
(i.e., self-report data may be more likely to result in taxonic structure), a recent review found no effect of data type on rates of taxonic compared to dimensional structure (Haslam et al., 2012). Additional studies incorporating both behavioral and self-report data would be beneficial. Finally, several studies have suggested that DI may vary based on the type of distress (McHugh & Otto, 2011; Sirota, Rohsenow, Dolan, Martin, & Kahler, 2013), and thus studies also examining several types of distress may further elucidate the nature of the DI construct. Overall, results suggest that DI is better conceptualized as a dimensional, rather than categorical, construct. Conceptual models of the development of DI may be informed by these results and suggest that several smaller factors likely interact in an additive way to result in poorer tolerance of distress. In addition, continuous assessment measures are likely to provide the optimal assessment of this construct.
References Ahmed, A. O., Green, B. A., Clark, C. B., Stahl, K. C., & McFarland, M. E. (2011). Latent structure of unipolar and bipolar mood symptoms. Bipolar Disorders, 13(5–6), 522–536. http://doi.org/10.1111/j.1399-5618.2011.00940.x American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (Fifth). American Psychiatric Publishing. http://doi.org/10.1176/appi.books.9780890425596.744053 Anestis, M. D., Selby, E. A., Fink, E. L., & Joiner, T. E. (2007). The multifaceted role of distress tolerance in dysregulated eating behaviors. International Journal of Eating Disorders, 40(8), 718–726. http://doi.org/10.1002/eat.20471 Banducci, A. N., Bujarski, S. J., Bonn-Miller, M. O., Patel, A., & Connolly, K. M. (n.d.). The impact of intolerance of emotional distress and uncertainty on veterans with co-occurring
PTSD and substance use disorders. Journal of Anxiety Disorders. http://doi.org/10.1016/j.janxdis.2016.03.003 Beck, K. H., Daughters, S. B., & Ali, B. (2013). Hurried driving: Relationship to distress tolerance, driver anger, aggressive and risky driving in college students. Accident Analysis and Prevention, 51, 51–55. http://doi.org/10.1016/j.aap.2012.10.012 Bernstein, A., Zvolensky, M. J., Kotov, R., Arrindell, W. A., Taylor, S., Sandin, B., … Schmidt, N. B. (2006). Taxonicity of anxiety sensitivity: A multi-national analysis. Journal of Anxiety Disorders, 20(1), 1–22. http://doi.org/10.1016/j.janxdis.2004.11.006 Bernstein, A., Zvolensky, M. J., Stewart, S., & Comeau, N. (2007). Taxometric and factor analytic models of anxiety sensitivity among youth: Exploring the latent structure of anxiety psychopathology vulnerability. Behavior Therapy, 38(3), 269–283. http://doi.org/10.1016/j.beth.2006.08.005 Bernstein, A., Zvolensky, M. J., Stewart, S. H., Nancy Comeau, M., & Leen-Feldner, E. W. (2006). Anxiety sensitivity taxonicity across gender among youth. Behaviour Research and Therapy, 44(5), 679–698. http://doi.org/10.1016/j.brat.2005.03.011 Bernstein, A., Zvolensky, M. J., Vujanovic, A. A., & Moos, R. (2009). Integrating anxiety sensitivity, distress tolerance, and discomfort intolerance: a hierarchical model of affect sensitivity and tolerance. Behavior Therapy, 40(3), 291–301. http://doi.org/10.1016/j.beth.2008.08.001 Bernstein, A., Zvolensky, M. J., Weems, C., Stickle, T., & Leen-Feldner, E. W. (2005). Taxonicity of anxiety sensitivity: An empirical test among youth. Behaviour Research and Therapy, 43(9), 1131–1155. http://doi.org/10.1016/j.brat.2004.07.008 Bornovalova, M. A., Gratz, K. L., Daughters, S. B., Hunt, E. D., & Lejuez, C. W. (2012). Initial
RCT of a distress tolerance treatment for individuals with substance use disorders. Drug and Alcohol Dependence, 122(1–2), 70–76. http://doi.org/10.1016/j.drugalcdep.2011.09.012 Broman-Fulks, J. J., Deacon, B. J., Olatunji, B. O., Bondy, C. L., Abramowitz, J. S., & Tolin, D. F. (2010). Categorical or dimensional: A reanalysis of the anxiety sensitivity construct. Behavior Therapy, 41(2), 154–171. http://doi.org/10.1016/j.beth.2009.02.005 Broman-Fulks, J. J., Green, B. A., Olatunji, B. O., Berman, M. E., Arnau, R. C., Deacon, B. J., & Sawchuk, C. N. (2008). The latent structure of anxiety sensitivity-revisited. Assessment, 15(2), 188–203. http://doi.org/10.1177/1073191107311284 Brown, R. A., Lejuez, C. W., Kahler, C. W., & Strong, D. R. (2002). Distress tolerance and duration of past smoking cessation attempts. Journal of Abnormal Psychology, 111(1), 180– 185. http://doi.org/10.1037/0021-843X.111.1.180 Brown, T. A., & Barlow, D. H. (2009). A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment, 21(3), 256–271. http://doi.org/10.1037/a0016608 Carleton, R. N., Weeks, J. W., Howell, A. N., Asmundson, G. J. G., Antony, M. M., & McCabe, R. E. (2012). Assessing the latent structure of the intolerance of uncertainty construct: An initial taxometric analysis. Journal of Anxiety Disorders, 26(1), 150–157. http://doi.org/10.1016/j.janxdis.2011.10.006 Corstorphine, E., Mountford, V., Tomlinson, S., Waller, G., & Meyer, C. (2007). Distress tolerance in the eating disorders. Eating Behaviors, 8(1), 91–97. http://doi.org/10.1016/j.eatbeh.2006.02.003 Daughters, S. B., Lejuez, C. W., Bornovalova, M. A., Kahler, C. W., Strong, D. R., & Brown, R.
A. (2005). Distress tolerance as a predictor of early treatment dropout in a residential substance abuse treatment facility. Journal of Abnormal Psychology, 114(4), 729–734. http://doi.org/10.1037/0021-843x.114.4.729 Daughters, S. B., Lejuez, C. W., Kahler, C. W., Strong, D. R., & Brown, R. A. (2005). Psychological distress tolerance and duration of most recent abstinence attempt among residential treatment-seeking substance abusers. Journal of the Society of Psychologists in Addictive Behaviors, 19(2), 208–211. http://doi.org/10.1037/0893-164X.19.2.208 Ehrlich, K. B., Cassidy, J., Gorka, S. M., Lejuez, C. W., & Daughters, S. B. (2013). Adolescent friendships in the context of dual risk: the roles of low adolescent distress tolerance and harsh parental response to adolescent distress. Emotion, 13(5), 843–851. http://doi.org/10.1037/a0032587 Gratz, K. L., Rosenthal, Z., Tull, M. T., Lejuez, C. W., & Gunderson, J. G. (2006). An experimental investigation of emotion dysregulation in borderline personality disorder. Journal of Abnormal Psychology, 115(4), 850–855. http://doi.org/10.1037/19492715.S.1.18 Grove, W. M. (1991). When is a diagnosis worth making? A statistical comparison of two prediction strategies. Psychological Reports, 69(1), 3–17. Harrington, N. (2005a). It’s too difficult! Frustration intolerance beliefs and procrastination. Personality and Individual Differences, 39(5), 873–883. http://doi.org/10.1016/j.paid.2004.12.018 Harrington, N. (2005b). The frustration discomfort scale: Development and psychometric properties. Clinical Psychology & Psychotherapy, 387, 374–387. http://doi.org/10.1002/cpp.465
Haslam, N., Holland, E., & Kuppens, P. (2012). Categories versus dimensions in personality and psychopathology: A quantitative review of taxometric research. Psychological Medicine, 42(5), 903–920. http://doi.org/10.1017/S0033291711001966 Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., & Strosahl, K. D. (1996). Experimental avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. Journal of Consulting and Cinical Psychology, 64(6), 1152–1168. http://doi.org/10.1037/0022-006X.64.6.1152 Holm-Denoma, J. M., Richey, J. A., & Joiner, T. E. (2010). The latent structure of dietary restraint, body dissatisfaction, and drive for thinness: A series of taxometric analyses. Psychological Assessment, 22(4), 788–797. http://doi.org/10.1037/a0020132 Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., … Wang, P. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. http://doi.org/10.1176/appi.ajp.2010.09091379 Kollman, D. M., Brown, T. A., Liverant, G. I., & Hofmann, S. G. (2006). A taxometric investigation of the latent structure of social anxiety disorder in outpatients with anxiety and mood disorders. Depression and Anxiety, 23(4), 190–199. http://doi.org/10.1002/da.20158 Leyro, T. M., Bernstein, A., Vujanovic, A. A., McLeish, A. C., & Zvolensky, M. J. (2011). Distress Tolerance Scale: A confirmatory factor analysis among daily cigarette smokers. Journal of Psychopathology and Behavioral Assessment, 33(1), 47–57. http://doi.org/10.1007/s10862-010-9197-2. Linehan, M. M. (1993). Cognitive-behavioral Treatment of Borderline Personality Disorder. Guilford Press.
Liu, R. T. (2016). Taxometric evidence of a dimensional latent structure for depression in an epidemiological sample of children and adolescents. Psychological Medicine, 46(6), 1265– 1275. http://doi.org/10.1017/S0033291715002792 Liu, R. T., Jones, R. N., & Spirito, A. (2015). Is adolescent suicidal ideation continuous or categorical? A taxometric analysis. Journal of Abnormal Child Psychology, 43(8), 1459– 1466. http://doi.org/10.1007/s10802-015-0022-y Markon, K. E., Chmielewski, M., & Miller, C. J. (2011). The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review. Psychological Bulletin, 137(6), 1093. http://doi.org/10.1037/a0025727 McHugh, R. K., Daughters, S. B., Lejuez, C. W., Murray, H. W., Hearon, B. A., Gorka, S. M., & Otto, M. W. (2011). Shared variance among self-report and behavioral measures of distress intolerance. Cognitive Therapy and Research, 35(3), 266–275. http://doi.org/10.1007/s10608-010-9295-1 McHugh, R. K., Kertz, S. J., Weiss, R. B., Baskin-Sommers, A. R., Hearon, B. A., & Björgvinsson, T. (2014). Changes in distress intolerance and treatment outcome in a partial hospital setting. Behavior Therapy, 45(2), 232–240. http://doi.org/10.1016/j.beth.2013.11.002 McHugh, R. K., & Otto, M. W. (2011). Domain-general and domain-specific strategies for the assessment of distress intolerance. Psychology of Addictive Behaviors, 25(4), 745–749. http://doi.org/10.1037/a0025094 McHugh, R. K., & Otto, M. W. (2012). Refining the measurement of distress intolerance. Behavior Therapy, 43(3), 641–651. http://doi.org/10.1016/j.beth.2011.12.001 Meehl, P. E. (1973). MAXCOV-HITMAX: A taxonomic search method for loose genetic
syndromes. In P. E. Meehl (Ed.), Psychodiagnosis: Selected Papers (pp. 200–224). Minneapolis: University of Minnesota Press. Meehl, P. E. (1977). Specific etiology and other forms of strong influence: Some quantitative meanings. Journal of Medicine and Philosophy, 2(1), 33–53. http://doi.org/10.1093/jmp/2.1.33 Meehl, P. E. (1995). Bootstraps taxometrics. Solving the classification problem in psychopathology. The American Psychologist, 50(4), 266–275. http://doi.org/10.1037/0003066X.50.4.266 Meehl, P. E., & Yonce, L. J. (1994). Taxometric analysis: I. Detecting taxonicity with two quantitative indicators using means above and below a sliding cut (MAMBAC procedure). Psychological Reports, 74(3, Pt 2), 1059–1274. Norr, A. M., Allan, N. P., Macatee, R. J., Keough, M. E., & Schmidt, N. B. (2014). The effects of an anxiety sensitivity intervention on anxiety, depression, and worry: Mediation through affect tolerances. Behaviour Research and Therapy, 59, 12–19. http://doi.org/10.1016/j.brat.2014.05.011 Nugent, K. L., Chiappelli, J., Rowland, L. M., Daughters, S. B., & Hong, L. E. (2014). Distress intolerance and clinical functioning in persons with schizophrenia. Psychiatry Research, 220(1–2), 31–36. http://doi.org/10.1016/j.psychres.2014.07.026 Olatunji, B. O., Broman-Fulks, J. J., Bergman, S. M., Green, B. A., & Zlomke, K. R. (2010). A taxometric investigation of the latent structure of worry: Dimensionality and associations with depression, anxiety, and stress. Behavior Therapy, 41(2), 212–228. http://doi.org/10.1016/j.beth.2009.03.001 Peterson, R. A., & Reiss, S. (1992). Anxiety Sensitivity Index Revised. Worthington, OH:
International Diagnostic Systems. Prisciandaro, J. J., & Tolliver, B. K. (2015). Evidence for the continuous latent structure of mania and depression in out-patients with bipolar disorder: Results from the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD). Psychological Medicine, 45(12), 2595–2603. http://doi.org/10.1017/S0033291715000513 Ruscio, J., Haslam, N., & Ruscio, A. M. (2006a). Introduction to the taxometric method: A practical guide. Mahwah, NJ: Lawrence Erlbaum Associates. Ruscio, J., Haslam, N., & Ruscio, A. M. (2006b). Introduction to the taxometric method: A practical guide. Mahwah, NJ: Lawrence Erlbaum. Ruscio, J., Ruscio, A. M., & Carney, L. M. (2011). Performing taxometric analysis to distinguish categorical and dimensional variables. Journal of Experimental Psychopathology, 2(2), 170–196. http://doi.org/10.5127/jep.010910 Ruscio, J., Ruscio, A. M., & Meron, M. (2007). Applying the bootstrap to taxometric analysis: Generating empirical sampling distributions to help interpret results. Multivariate Behavioral Research, 42(2), 349–386. http://doi.org/10.1080/00273170701360795 Ruscio, J., Walters, G. D., Marcus, D. K., & Kaczetow, W. (2010). Comparing the relative fit of categorical and dimensional latent variable models using consistency tests. Psychological Assessment, 22(1), 5–21. http://doi.org/10.1037/a0018259 Ruscio, J., & Wang, S. (2017). RTaxometrics: Taxometric Analysis. Retrieved from https://cran.r-project.org/web/packages/RTaxometrics/index.html Schmidt, N. B., Richey, J. A., & Fitzpatrick, K. K. (2006). Discomfort intolerance: Development of a construct and measure relevant to panic disorder. Journal of Anxiety Disorders, 20(3), 263–280. http://doi.org/10.1016/j.janxdis.2005.02.002
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., … Dunbar, G. C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59, 22–33. Simons, J. S., & Gaher, R. M. (2005). The Distress Tolerance Scale: Development and validation of a self-report measure. Motivation and Emotion. http://doi.org/10.1007/s11031-005-79553 Sirota, A. D., Rohsenow, D. J., Dolan, S. L., Martin, R. A., & Kahler, C. W. (2013). Intolerance for discomfort among smokers: comparison of smoking-specific and non-specific measures to smoking history and patterns. Addictive Behaviors, 38(3), 1782–1787. http://doi.org/10.1016/j.addbeh.2012.10.009 Taylor, S., Rabian, B., & Fedoroff, I. C. (1999). Anxiety sensitivity: Progress, prospects, and challenges. In S. Taylor (Ed.), Anxiety sensitivity: Theory, research, and treatment of fear of anxiety (pp. 339–353). Mahwah, NJ: Lawrence Erlbaum. Timpano, K. R., Broman-Fulks, J. J., Glaesmer, H., Exner, C., Rief, W., Olatunji, B. O., … Schmidt, N. B. (2013). A taxometric exploration of the latent structure of hoarding. Psychological Assessment, 25(1), 194–203. http://doi.org/10.1037/a0029966 Trafton, J. A., & Gifford, E. V. (2011). Biological bases of distress tolerance. In M. J. Zvolensky, A. Bernstein, & A. A. Vujanovic (Eds.), Distress tolerance: Theory, research, and clinical applications (pp. 80–102). New York, New York: The Guilford Press. Tull, M. T., & Gratz, K. L. (2013). Major depression and risky sexual behavior among substance dependent patients: The moderating roles of distress tolerance and gender. Cognitive Therapy and Research, 37(3), 483–497. http://doi.org/10.1007/s10608-012-9490-3
Waller, N. G., & Meehl, P. E. (1998). Multivariate taxometric procedures: Distinguishing types from continua. Thousand Oaks, CA: SAGE Publications. Walters, G. D., & Ruscio, J. (2009). To sum or not to sum: Taxometric analysis with ordered categorical assessment items. Psychological Assessment, 21(1), 99–111. http://doi.org/10.1037/a0015010 Weeks, J. W., Carleton, R. N., Asmundson, G. J. G., McCabe, R. E., & Antony, M. M. (2010). “Social Anxiety Disorder carved at its joints”: Evidence for the taxonicity of social anxiety disorder. Journal of Anxiety Disorders, 24(7), 734–742. http://doi.org/10.1016/j.janxdis.2010.05.006 Widiger, T. A., Livesley, W. J., & Clark, L. A. (2009). An integrative dimensional classification of personality disorder. Psychological Assessment, 21(3), 243–255. http://doi.org/10.1037/a0016606 Witte, T. K., Holm-Denoma, J. M., Zuromski, K. L., Gauthier, J. M., & Ruscio, J. (2017). Individuals at High Risk for Suicide Are Categorically Distinct From Those at Low Risk. Psychological Assessment, 29(4), 382–393. http://doi.org/10.1037/pas0000349 World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva: World Health Organization.
Figure 1 Graphical results of MAMBAC, MAXEIG, and LMode analyses for Sample 1
Figure 2. Graphical results of MAMBAC, MAXEIG, and LMode analyses for Sample 2
Table 1. Measures of average indicator validity, correlations, and CCFI values for taxometric analyses. Indicator correlations
Sample 1
BetweenGroup Validity MAMBAC 1.80 (.31) MAXEIG 1.87 (.25) L-Mode 1.78 (.22) Mean
CCFI .20 .32 .47
Full sample
Taxon Complement
.54 (.07) .54 (.07) .54 (.07)
.21 (.07) .29 (.08) .16 (.07)
.19 (.10) .09 (.08) .21 (.10)
.33
2 MAMBAC 1.90 .16 .53 .17 .15 (.35) (.05) (.06) (.08) MAXEIG 1.94 .36 .53 .16 .20 (.34) (.05) (.07) (.06) L-Mode 1.64 .34 .53 .28 -.02 (.25) (.05) (.12) (.08) Mean .29 Note. Correlations and between-group validity (Cohen’s d) values are averaged across indicators. Skew and kurtosis are averaged across indicators in the full sample.
Highlights:
This is the first known examination of the latent structure of distress intolerance
Results indicated that distress intolerance is best characterized dimensionally
Dimensional classification and measurement may better reflect distress intolerance
PII: DOI: Reference:
S0165-1781(17)30382-7 http://dx.doi.org/10.1016/j.psychres.2017.09.036 PSY10847
To appear in: Psychiatry Research Received date: 2 March 2017 Revised date: 7 August 2017 Accepted date: 12 September 2017 Cite this article as: Kimberly T. Stevens, Sarah J. Kertz, Thröstur Björgvinsson and R. Kathryn McHugh, Investigating the latent structure of distress intolerance, Psychiatry Research, http://dx.doi.org/10.1016/j.psychres.2017.09.036 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Investigating the latent structure of distress intolerance Kimberly T. Stevensa*, Sarah J. Kertza, b, Thröstur Björgvinssonb, & R. Kathryn McHughb a
Psychology Department, Southern Illinois University, Carbondale, IL, USA b
McLean Hospital/Harvard Medical School, Belmont, MA, USA
*Corresponding Author: Kimberly T. Stevens, Southern Illinois University, Mail code 6502 Carbondale IL 62901. E-mail: [email protected]. Phone: 618.453.3572.
Abstract Distress intolerance (DI) is defined as a perceived or actual inability to withstand distressing emotional or somatic states, which motivates the use of avoidance strategies. Despite widespread interest in DI, key questions about its underlying structure remain unanswered. The current study evaluated the latent structure of DI in two large samples using four-indicators and three taxometric procedures (MAMBAC, MAXEIG, and L-Mode). Data interpretation relied primarily on the Comparison Curve Fit Indices (CCFI). Overall, results from the three nonredundant procedures suggested that DI was more accurately characterized by a dimensional rather than a categorical conceptualization. Implications for assessment and conceptual models of DI are discussed.
Keywords: taxometric, distress intolerance, latent structure, dimensional models
1. Introduction Accumulating evidence suggests that many psychological constructs are dimensional in nature (Carleton et al., 2012; Holm-Denoma, Richey, & Joiner, 2010; Liu, Jones, & Spirito, 2015; Olatunji, Broman-Fulks, Bergman, Green, & Zlomke, 2010; Timpano et al., 2013). Accordingly, much attention is shifting to dimensional systems of psychological disorder classification (e.g., T. A. Brown & Barlow, 2009; Widiger, Livesley, & Clark, 2009) and the study of mental illness across a range of functional dimensions (Insel et al., 2010). Despite a movement towards dimensional models, the field currently relies on categorical classification systems such as the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5; American Psychiatric Association, 2013) and the International Statistical Classification of Diseases, 10th revision (ICD-10 World Health Organization, 1992) for characterizing mental illness. Statistically, there are costs and benefits of using a categorical versus dimensional classifications systems; however, dimensional prediction outperforms categorical in most cases (Grove, 1991). Understanding the latent structure of psychological constructs is critical to understanding whether they are optimally conceptualized according to dimensions or categories and therefore may inform measurement and assessment in both clinical and research settings. Distress intolerance (DI), defined as a perceived or actual inability to withstand distressing emotional or somatic states, motivates the use of avoidance strategies (Simons & Gaher, 2005; Trafton & Gifford, 2011). DI is elevated among those with psychological disorders relative to those without (e.g., Corstorphine, Mountford, Tomlinson, Waller, & Meyer, 2007; Gratz, Rosenthal, Tull, Lejuez, & Gunderson, 2006; McHugh & Otto, 2011; Schmidt, Richey, & Fitzpatrick, 2006), and predicts poorer outcomes within clinical samples, such as a higher likelihood of relapse or treatment discontinuation among those with substance use disorders (R.
A. Brown, Lejuez, Kahler, & Strong, 2002; Daughters, Lejuez, Bornovalova, et al., 2005; Daughters, Lejuez, Kahler, Strong, & Brown, 2005). Additionally, DI is associated with maladaptive behaviors even in unselected populations (e.g., Anestis, Selby, Fink, & Joiner, 2007; Beck, Daughters, & Ali, 2013; Harrington, 2005a), highlighting its applicability across the range of psychological functioning. In addition to its role in predicting poor outcomes and characterizing clinical samples, the reduction of DI may present as an important mechanism of treatment for a range of disorders. DI has long been conceptualized as a target of psychological treatments (Hayes, Wilson, Gifford, Follette, & Strosahl, 1996; Linehan, 1993), and can be successfully reduced in treatment, with greater reductions associated with corresponding improvement in clinical outcomes (Bornovalova, Gratz, Daughters, Hunt, & Lejuez, 2012; McHugh et al., 2014; Norr, Allan, Macatee, Keough, & Schmidt, 2014). Despite widespread interest in DI, key questions about its underlying structure remain unanswered. Studies examining the latent factor structure of DI have suggested either a single latent factor or a single higher-order factor, which subsumes distress-specific DI factors (e.g., Bernstein, Zvolensky, Vujanovic, & Moos, 2009; Leyro, Bernstein, Vujanovic, McLeish, & Zvolensky, 2011; McHugh & Otto, 2012). Most studies have operated under the assumption that DI is a dimensional construct characterized by quantitative, rather than qualitative, differences along a continuum; however, there have been no empirical studies to date to support this position and much of the literature on DI is housed in separate disorder-specific literatures, despite evidence of transdiagnostic utility (McHugh & Otto, 2011). Further, though there is an assumption of dimensionality, some measurement strategies have continued to employ dichotomous scoring (e.g., Ehrlich, Cassidy, Gorka, Lejuez, & Daughters, 2013; Tull & Gratz,
2013). Better understanding the underlying structure of DI may enhance rationale for modifying the assessment of this important construct, and may improve conceptualization and intervention efforts across diagnostic categories. Although there is a growing body of evidence for the dimensionality of many psychological constructs, studies of the latent structure of some psychological constructs have yielded mixed results. For example, suicidality is often theorized as a symptom appearing on a continuum. One study using a sample of depressed adolescents (Liu et al., 2015) found support for a dimensional conceptualization; however, another study in a sample of adults, found that high risk for suicide was categorically distinct from low risk for suicide (Witte, Holm-Denoma, Zuromski, Gauthier, & Ruscio, 2017). In addition, findings for the latent structure of depression and bipolar symptoms are mixed with some supporting a dimensional structure (Liu, 2016; Prisciandaro & Tolliver, 2015), and others finding taxonic structure (Ahmed, Green, Clark, Stahl, & McFarland, 2011). Lastly, social anxiety disorder symptoms have suggested a dimensional structure in some studies (Kollman, Brown, Liverant, & Hofmann, 2006), but taxonic structure in others (Weeks, Carleton, Asmundson, McCabe, & Antony, 2010). These mixed findings support the need for empirical evaluation, even for constructs that are theoretically conceptualized as dimensional. There are also existing taxometric investigations of factors related to DI, or distressspecific DI factors (e.g., intolerance of uncertainty and anxiety sensitivity). For example, intolerance of uncertainty has demonstrated a dimensional latent structure in an initial assessment (Carleton et al., 2012). However, existing evidence for anxiety sensitivity has been mixed across adult and youth samples. Some work has supported a taxonic structure (e.g., Bernstein, Zvolensky, Kotov, et al., 2006; Bernstein, Zvolensky, Stewart, & Comeau, 2007;
Bernstein, Zvolensky, Weems, Stickle, & Leen-Feldner, 2005; Bernstein, Zvolensky, Stewart, Nancy Comeau, & Leen-Feldner, 2006), while other work has supported a dimensional structure (Broman-Fulks et al., 2008, 2010; Taylor, Rabian, & Fedoroff, 1999). Discrepant findings for the latent structure of constructs related to DI provide additional justification for examining the underlying structure of this presumed dimensional construct. Simons and Gaher’s (2005) proposed four dimensions of DI: tolerability/aversiveness, appraisal/acceptability, absorption of attention/disruption of functioning, and the action tendency to escape/avoid distress. Thus, this definition captures both cognitive (perception/interpretation of distress) and behavioral (tendency to escape/avoid distress) elements of this construct. In the current study, our aim was to examine the latent structure of DI using indicators representative of four DI dimensions (based on empirical and theoretical considerations). Indicators were drawn from validated self-report measures and measured in two large samples. 2. Methods 2.1 Participants 2.1.1 Sample 1. Participants included 392 patients from the [hospital name removed for masked review] who received treatment between July 2010 and July 2012. The [hospital name removed for masked review] provides individual and group cognitive behavioral therapy for patients with anxiety and/or mood disorders. The study sample was primarily female (57.7%, n = 226) and the average age was 30.0 years (SD = 14.2). The sample was primarily non-Hispanic White (89.8%), with 4.6% Asian, 2.8% Hispanic/Latino/a, 1.5% Black/African American, 0.8% American Indian/Alaskan Native, and 0.5% Native Hawaiian/Pacific Islander; 3.6% declined to answer. The sample was highly educated, with 35.6% reporting some college education, 31.7% post-undergraduate education, 23.7% a 4-year college degree, 8.0% high school graduate, 0.8%
some high school, and 0.3% 8th grade or less. Most participants were never married (63.7%); however, 24.3% were married, 8.4% separated/divorced, 1.8% widowed, and 1.8% living with a partner. 2.1.2. Sample 2. Participants were recruited from community-based resources (e.g., Craigslist, flyers in public areas) and an urban outpatient clinic specializing in anxiety disorder treatment (using flyers in clinic common spaces). The sample consisted of 300 unselected participants (i.e., adults from the community) and 100 participants recruited from an outpatient clinic to obtain a range of psychiatric functioning. Of note, taxonicity can be detected in sample sizes as low as 100; however, sample sizes of at least 300 are recommended for taxometric procedures (Ruscio, Walters, Marcus, & Kaczetow, 2010). The study sample was primarily female (72.3%) and the average age was 34.5 years (SD = 13.5). The sample was primarily Caucasian (84.0%), with 5.8% Asian, 5.0% African American, 4.3% other ethnicity, and 0.3% Native Hawaiian or Pacific Islander; 0.8% declined to answer. The sample was highly educated, with 43.3% with 4-year college degrees, 27.5% with graduate degrees, 24.5% with some college education, and 4.0% high school graduates; 0.8% declined to answer. 2.2. Measures 2.2.1. Distress Intolerance Index (McHugh & Otto, 2012). Participants in Sample 1 completed the DII, a 10-item self-report content-independent measure of DI. The measure derived items from the Distress Tolerance Scale (Simons & Gaher, 2005), Frustration Discomfort Scale (Harrington, 2005b); and Anxiety Sensitivity Index (Peterson & Reiss, 1992). Items were rated on a scale from 0 (very little) to 4 (very much). The DII has demonstrated excellent internal consistency, strong reliability, and initial concurrent validity with self-report and behavioral measures of DI (McHugh & Otto, 2012). Four DII items adopted from the
Distress Tolerance Scale were selected as indicators for the current study. The indicators evidenced strong internal consistency (α = .82). 2.2.2. Distress Tolerance Scale (DTS; Simons & Gaher, 2005). Sample 2 completed the DTS, a 15-item self-report measure of distress tolerance. The DTS has demonstrated strong internal consistency, reliability, and validity in unselected and clinical samples (Simons & Gaher, 2005). The same four items from Sample 1 were chosen as indicators, which also demonstrated strong internal consistency in Sample 2 (α = .82). 2.3. Procedure 2.3.1. Sample 1. The current study was approved by the hospital’s Institutional Review Board. Participants provided informed consent before participating, and all data were handled in accordance with the ethical guidelines provided by the American Psychological Association. Each participant completed an initial intake assessment including a demographics survey, battery of self-report questionnaires (also completed upon discharge), and the MINI structured diagnostic interview (Sheehan et al., 1998). 2.3.2. Sample 2. This study was approved by the university’s Institutional Review Board. All procedures were conducted electronically via a web-based survey tool. Participants provided informed consent within the web-based program, and those who agreed to participate then completed a brief battery of questionnaires assessing DI and related constructs. 2.4. Data analytic plan Taxometric and comparison data simulations were computed using Ruscio and Wang’s (2017) taxometric programs for R (available at https://cran.rproject.org/web/packages/RTaxometrics/index.html). Extensive support materials are available at http://ruscio.pages.tcnj.edu/taxometrics/. Three taxometric procedures were used to provide
non-redundant, consistent evidence of the latent structure of DI (Ruscio et al., 2010). The current study computed the Mean Above Minus Below a Cut (MAMBAC; Meehl & Yonce, 1994), maximum eigenvalue (MAXEIG; Waller & Meehl, 1998), and latent-mode factor analysis (LMode; Waller & Meehl, 1998) procedures. We used the following code: RunTaxometrics(data, seed = 1, n.pop = 1e+05, n.samples = 100, reps = 1,MAMBAC = TRUE, assign.MAMBAC = 1, n.cuts = 50, n.end = 25, MAXEIG =TRUE, assign.MAXEIG = 1, windows = 50, overlap = 0.9, LMode = TRUE, mode.l =-0.001, mode.r = 0.001, MAXSLOPE = FALSE) Interpretation of results relied primarily on the Comparison Curve Fit Indices (CCFI). CCFI scores were developed to provide objective estimates of fit between the simulated data sets and the research data (Ruscio, Ruscio, & Meron, 2007). CCFI values range from 0 to 1. A score of 0 strongly indicates support for a dimensional model and a score of 1 strongly supports a categorical model. A score of .5 represents ambiguous latent structure, so values between .45 and .55 should be cautiously interpreted. Underlying structure is best identified at CCFI values greater than .55 or less than .45 with an accuracy of .90 (Ruscio et al., 2010). Several simulation studies have indicated that the CCFI distinguishes between categorical and dimensional data, even when data are less than ideal (Ruscio, Ruscio, & Carney, 2011). Mean CCFI scores across the three procedures were used to evaluate the underlying structure. Graphical representations of dimensional and categorical simulated comparison data sets were also computed for each sample in order to visualize the MAMBAC and MAXEIG curves (Ruscio, Haslam, & Ruscio, 2006a). 2.4.1. MAMBAC taxometric method. The MAMBAC procedure is used to detect taxonicity through the use of quantitative indicators examining the means above and below a sliding cut score (Ruscio et al., 2006a). MAMBAC relies on the assumption that if a construct
has a taxonic latent structure, then there will be a cut score that distinguishes groups. Cut scores are identified across the range of the input variable as a function of the second indicator (output variable). Thus, the MAMBAC identifies a group above and below the cut score for a series of values of the input variable. The mean of cases falling below the cut score is subtracted from the mean of cases falling above the cut score. Mean differences are graphed, resulting in curved distributions. Fifty equally spaced cuts were used along the input indicator beginning at 25 cases from either extreme. If the construct has a taxonic structure, the MAMBAC curves will appear peaked at the point that best distinguishes the two groups. A construct with a dimensional latent structure yields flat or concave shaped MAMBAC curves, which suggests that there is no cut score that classifies cases. Default settings (including variables used in all input-output pairings [assign.MAMBAC = 1], cuts starting and ending at 25 points from either extreme [n.end = 25], and 50 total cuts [n.cuts = 50]) were used in the analyses. 2.4.2. MAXEIG taxometric method. The MAXEIG is a multivariate generalization of the maximum covariance (MAXCOV; Meehl, 1973) procedure. MAXEIG is conceptually and mathematically similar to MAXCOV but differs in several ways (see Ruscio, Haslam, & Ruscio, 2006b for details). The procedure provides an estimate of the conditional eigenvalues between two or more output indicators for a series of subsamples ordered along the range of a third indicator (input variable; Ruscio et al., 2006a), which is then graphed in order along the input variable, resulting in curved distributions. A smooth distribution indicates a dimensional structure, whereas a distribution with a peaked curve suggests taxonic structure. The input variable in the current study was the sum of the remaining two indicators. Analyses used 50 windows with a 0.9 overlap and n = 66 per window. Default settings (each variable serving as an
input variable once [assign.MAXEIG = 1] and overlapping windows set at .90 [overlap = .90]) were used. 2.4.3. L-Mode taxometric method. The L-Mode procedure aggregates indicators using exploratory factor analysis on the covariance between indicators (Ruscio et al., 2006a). The LMode graph produces a factor score density plot with factor scores along the x-axis and frequency along the y-axis. Dichotomous latent structure yields a bimodal L-Mode distribution, whereas a dimensional latent structure yields a unimodal distribution. Default settings (searching for the left mode beyond -.001 [mode.l = -.001] and searching for the right mode beyond .001 [mode.r = .001]) were used in analyses. 2.4.4. Indicator selection and adequacy. As described above, four indicators were used in the analyses. Items included "I can't handle feeling distressed or upset,” "My feelings of distress or being upset scare me,” “I'll do anything to stop feeling distressed or upset,” and “When I feel distressed or upset, I cannot help but concentrate on how bad the distress actually feels." These items were selected to represent the relevant facets of DI (Simons & Gaher, 2005) and to reduce within-group indicator correlation, based on Ruscio, Haslam, and Ruscio (2006a)’s suggestions. Although a number of studies have suggested some variability across types of distress, studies have consistently suggested a single latent higher-order DI factor (Bernstein et al., 2009; Leyro et al., 2011; McHugh & Otto, 2012), and thus our selection of indicators was informed by efforts to capture the relevant facets of DI in a way that is general to distress type (i.e., distress or upset rather than a specific emotion). Further, Walters and Ruscio (2009) recommended selecting indicators that have four or more ordered categories. The range of the indicators selected in the current study (1-5) is appropriate for traditional taxometric analyses.
The adequacy of the indicators was assessed using validity estimates, indicator distributions, and indicator correlations in the full sample as well as the putative taxon and complement groups. Across each procedure, the indicators demonstrated appropriate validity, Cohen’s d > 1.25 (Meehl, 1995; see Table 1). Indicator distributions suggested tolerable levels of skew, Mskew = -.27 (SD = .22), Mskew = .53 (SD = .27), and kurtosis, Mkurtosis = -.93 and Mkurtosis = -.87, respectively for each sample. It is also recommended that correlations between indicators be around, r < .30 (Meehl, 1995), and be higher in the full sample compared to the putative taxon and complement groups. In both samples, indicator correlations were larger in the full sample than in the taxon or complement groups (see Table 1 for a summary of indicator correlations). Within-group correlations for each group were within the recommended range. Overall, estimates indicated that the indicators were valid, their distributions were roughly normal, and indicator correlations were higher in the full sample compared to the groups. Therefore, the indicators were judged as appropriate and adequate for further taxometric analyses. 3. Results Overall results for Sample 1 from the three procedures suggested a dimensional structure. The CCFI was .20 for the MAMBAC procedure, .32 for the MAXEIG procedure, and .47 for LMode. The mean CCFI across procedures was .33. Results from Sample 2 were also indicative of dimensional structures. The CCFI was .16 for the MAMBAC procedure, .36 for MAXEIG, and .34 for L-Mode. The mean CCFI across the three procedures was .29. The panel of curves are presented in Figure 1 (Sample 1) and Figure 2 (Sample 2) for desriptive purposes. The panel on the left represents siimulated categorical comparison data and the panel on the right represents simulated dimensional comparison data. Averaged indicator curve values from the current study are illustrated by the bolded black line. The thin black lines
represent the minimum and maximum simulated values at each data point and the gray band represents the middle 50% of simulated values. For Sample 1, the average curve appeared more consistent with the simulated dimensional curve compared to the categorical comparison curve. Similarly, the average curve for Sample 2 appeared more consistent with the simulated dimensional comparison data. 4. Discussion Although DI is a well-established transdiagnostic vulnerability factor, there has been relatively little research focused on its underlying structure. The current study used taxometric analyses to investigate the extent to which the construct is better understood as dimensional or categorical in nature, using both mixed (i.e., both unselected and clinical) and clinical samples. Overall, results from three non-redundant analyses suggested that DI was more accurately characterized by a dimensional rather than a categorical conceptualization. These findings are consistent with a growing body of evidence supporting a dimensional latent structure of psychological symptoms (Carleton et al., 2012; Holm-Denoma et al., 2010; Liu et al., 2015; Olatunji et al., 2010; Timpano et al., 2013). Although this is the first known study to examine the latent structure of DI, most empirical studies have assumed a dimensional underlying structure by measuring DI continuously and testing its linear associations with outcomes (e.g., Banducci, Bujarski, BonnMiller, Patel, & Connolly, in press; McHugh et al., 2014). The exception is with behavioral measures of DI, in which quantitative outcomes have been dichotomized in some studies (e.g., Gratz et al., 2006; Nugent, Chiappelli, Rowland, Daughters, & Hong, 2014). Behavioral measures of DI employ goal-driven tasks that induce distress and measure the time to discontinuation as an index of DI. These measures can be scored either continuously (time to
discontinuation) or dichotomously (persist until the maximum time or discontinue prior to that time). Findings from the current study suggest that a dimensional conceptualization and corresponding continuous measurement may more accurately reflect the underlying construct. However, self-report and behavioral measures of DI are not consistently highly correlated, and thus the degree to which these approaches measure separate constructs has been debated (McHugh et al., 2011). Future studies utilizing multiple methods—including both self-report and behavioral measures—may improve understanding of the latent structure of DI. This may further elucidate whether these methods capture different constructs, or different facets of DI. Results from the current study have implications for the assessment of DI. These results suggest that optimal measures of DI will be continuous. Interestingly, a recent meta-analysis found that continuous measures of psychopathology were on average 15% more reliable and 37% more valid than discrete measures, regardless of construct type (Markon, Chmielewski, & Miller, 2011). The authors proposed that unless there is a specific rationale for using discrete measures, continuous measures of psychopathology are more likely to show greater associations with the construct of interest. As such, current self-report measures of DI, including the Distress Tolerance Scale and the Distress Intolerance Index, appear to be well suited to assess the construct. Dichotomized behavioral measures, however, may be improved by using a continuous indicator (e.g., number of seconds of distress endured). Given evidence for ceiling effects or truncated ranges of scores in these measures, which at times necessitates dichotomous scoring (Ehrlich et al., 2013; Tull & Gratz, 2013), consideration of how to enhance the range of scores on behavioral measures is needed. Nonetheless, studies examining the underlying structure of DI using a multi-method approach is needed to better understand the construct captured by behavioral measures of DI.
Understanding DI as a dimensional construct also has important implications for developmental models. Dimensional constructs are thought to develop from interactive and additive effects of multiple factors (Haslam, Holland, & Kuppens, 2012; Meehl, 1977). Identifying factors that contribute to the development of DI may provide insight into its underlying mechanisms and potentially allow for early intervention or prevention efforts to alter negative trajectories during critical developmental periods. Unfortunately, studies of putative risk or causal factors for the development of DI are lacking. Such factors are likely important as they would contribute directly to increased DI, but may also serve as proxy risk factors for the development of other psychiatric symptoms or disorders. Proxy risk factors are those that may not have a direct influence on the outcome variable (e.g., substance misuse, self-injurious behavior, etc.) but increase the likelihood of the outcome by their influence on the causal risk factor (i.e., DI). Thus, given the transdiagnostic relevance of DI, factors associated with increased likelihood of developing high DI may also serve as proxy risk factors for a wide range of symptoms and maladaptive behaviors. The current study had several limitations, which may have influenced findings or limit generalizability to other samples. First, the current study focused on DI for negative emotions. Results may not generalize to other facets of DI (e.g., uncertainty, pain, etc.). Additional study of other forms of DI in various samples is needed. Second, the study samples were not fully representative of demographic variability. Future work including participants from a broader range of racial and ethnic backgrounds and socioeconomic status would provide further support for the generalizability of the findings. Third, the data were limited to self-report measures of perceived DI and did not include any behavioral indices of DI. Although it was previously thought that that the nature of the data submitted to taxometric analyses may influence outcome
(i.e., self-report data may be more likely to result in taxonic structure), a recent review found no effect of data type on rates of taxonic compared to dimensional structure (Haslam et al., 2012). Additional studies incorporating both behavioral and self-report data would be beneficial. Finally, several studies have suggested that DI may vary based on the type of distress (McHugh & Otto, 2011; Sirota, Rohsenow, Dolan, Martin, & Kahler, 2013), and thus studies also examining several types of distress may further elucidate the nature of the DI construct. Overall, results suggest that DI is better conceptualized as a dimensional, rather than categorical, construct. Conceptual models of the development of DI may be informed by these results and suggest that several smaller factors likely interact in an additive way to result in poorer tolerance of distress. In addition, continuous assessment measures are likely to provide the optimal assessment of this construct.
References Ahmed, A. O., Green, B. A., Clark, C. B., Stahl, K. C., & McFarland, M. E. (2011). Latent structure of unipolar and bipolar mood symptoms. Bipolar Disorders, 13(5–6), 522–536. http://doi.org/10.1111/j.1399-5618.2011.00940.x American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (Fifth). American Psychiatric Publishing. http://doi.org/10.1176/appi.books.9780890425596.744053 Anestis, M. D., Selby, E. A., Fink, E. L., & Joiner, T. E. (2007). The multifaceted role of distress tolerance in dysregulated eating behaviors. International Journal of Eating Disorders, 40(8), 718–726. http://doi.org/10.1002/eat.20471 Banducci, A. N., Bujarski, S. J., Bonn-Miller, M. O., Patel, A., & Connolly, K. M. (n.d.). The impact of intolerance of emotional distress and uncertainty on veterans with co-occurring
PTSD and substance use disorders. Journal of Anxiety Disorders. http://doi.org/10.1016/j.janxdis.2016.03.003 Beck, K. H., Daughters, S. B., & Ali, B. (2013). Hurried driving: Relationship to distress tolerance, driver anger, aggressive and risky driving in college students. Accident Analysis and Prevention, 51, 51–55. http://doi.org/10.1016/j.aap.2012.10.012 Bernstein, A., Zvolensky, M. J., Kotov, R., Arrindell, W. A., Taylor, S., Sandin, B., … Schmidt, N. B. (2006). Taxonicity of anxiety sensitivity: A multi-national analysis. Journal of Anxiety Disorders, 20(1), 1–22. http://doi.org/10.1016/j.janxdis.2004.11.006 Bernstein, A., Zvolensky, M. J., Stewart, S., & Comeau, N. (2007). Taxometric and factor analytic models of anxiety sensitivity among youth: Exploring the latent structure of anxiety psychopathology vulnerability. Behavior Therapy, 38(3), 269–283. http://doi.org/10.1016/j.beth.2006.08.005 Bernstein, A., Zvolensky, M. J., Stewart, S. H., Nancy Comeau, M., & Leen-Feldner, E. W. (2006). Anxiety sensitivity taxonicity across gender among youth. Behaviour Research and Therapy, 44(5), 679–698. http://doi.org/10.1016/j.brat.2005.03.011 Bernstein, A., Zvolensky, M. J., Vujanovic, A. A., & Moos, R. (2009). Integrating anxiety sensitivity, distress tolerance, and discomfort intolerance: a hierarchical model of affect sensitivity and tolerance. Behavior Therapy, 40(3), 291–301. http://doi.org/10.1016/j.beth.2008.08.001 Bernstein, A., Zvolensky, M. J., Weems, C., Stickle, T., & Leen-Feldner, E. W. (2005). Taxonicity of anxiety sensitivity: An empirical test among youth. Behaviour Research and Therapy, 43(9), 1131–1155. http://doi.org/10.1016/j.brat.2004.07.008 Bornovalova, M. A., Gratz, K. L., Daughters, S. B., Hunt, E. D., & Lejuez, C. W. (2012). Initial
RCT of a distress tolerance treatment for individuals with substance use disorders. Drug and Alcohol Dependence, 122(1–2), 70–76. http://doi.org/10.1016/j.drugalcdep.2011.09.012 Broman-Fulks, J. J., Deacon, B. J., Olatunji, B. O., Bondy, C. L., Abramowitz, J. S., & Tolin, D. F. (2010). Categorical or dimensional: A reanalysis of the anxiety sensitivity construct. Behavior Therapy, 41(2), 154–171. http://doi.org/10.1016/j.beth.2009.02.005 Broman-Fulks, J. J., Green, B. A., Olatunji, B. O., Berman, M. E., Arnau, R. C., Deacon, B. J., & Sawchuk, C. N. (2008). The latent structure of anxiety sensitivity-revisited. Assessment, 15(2), 188–203. http://doi.org/10.1177/1073191107311284 Brown, R. A., Lejuez, C. W., Kahler, C. W., & Strong, D. R. (2002). Distress tolerance and duration of past smoking cessation attempts. Journal of Abnormal Psychology, 111(1), 180– 185. http://doi.org/10.1037/0021-843X.111.1.180 Brown, T. A., & Barlow, D. H. (2009). A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment, 21(3), 256–271. http://doi.org/10.1037/a0016608 Carleton, R. N., Weeks, J. W., Howell, A. N., Asmundson, G. J. G., Antony, M. M., & McCabe, R. E. (2012). Assessing the latent structure of the intolerance of uncertainty construct: An initial taxometric analysis. Journal of Anxiety Disorders, 26(1), 150–157. http://doi.org/10.1016/j.janxdis.2011.10.006 Corstorphine, E., Mountford, V., Tomlinson, S., Waller, G., & Meyer, C. (2007). Distress tolerance in the eating disorders. Eating Behaviors, 8(1), 91–97. http://doi.org/10.1016/j.eatbeh.2006.02.003 Daughters, S. B., Lejuez, C. W., Bornovalova, M. A., Kahler, C. W., Strong, D. R., & Brown, R.
A. (2005). Distress tolerance as a predictor of early treatment dropout in a residential substance abuse treatment facility. Journal of Abnormal Psychology, 114(4), 729–734. http://doi.org/10.1037/0021-843x.114.4.729 Daughters, S. B., Lejuez, C. W., Kahler, C. W., Strong, D. R., & Brown, R. A. (2005). Psychological distress tolerance and duration of most recent abstinence attempt among residential treatment-seeking substance abusers. Journal of the Society of Psychologists in Addictive Behaviors, 19(2), 208–211. http://doi.org/10.1037/0893-164X.19.2.208 Ehrlich, K. B., Cassidy, J., Gorka, S. M., Lejuez, C. W., & Daughters, S. B. (2013). Adolescent friendships in the context of dual risk: the roles of low adolescent distress tolerance and harsh parental response to adolescent distress. Emotion, 13(5), 843–851. http://doi.org/10.1037/a0032587 Gratz, K. L., Rosenthal, Z., Tull, M. T., Lejuez, C. W., & Gunderson, J. G. (2006). An experimental investigation of emotion dysregulation in borderline personality disorder. Journal of Abnormal Psychology, 115(4), 850–855. http://doi.org/10.1037/19492715.S.1.18 Grove, W. M. (1991). When is a diagnosis worth making? A statistical comparison of two prediction strategies. Psychological Reports, 69(1), 3–17. Harrington, N. (2005a). It’s too difficult! Frustration intolerance beliefs and procrastination. Personality and Individual Differences, 39(5), 873–883. http://doi.org/10.1016/j.paid.2004.12.018 Harrington, N. (2005b). The frustration discomfort scale: Development and psychometric properties. Clinical Psychology & Psychotherapy, 387, 374–387. http://doi.org/10.1002/cpp.465
Haslam, N., Holland, E., & Kuppens, P. (2012). Categories versus dimensions in personality and psychopathology: A quantitative review of taxometric research. Psychological Medicine, 42(5), 903–920. http://doi.org/10.1017/S0033291711001966 Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., & Strosahl, K. D. (1996). Experimental avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. Journal of Consulting and Cinical Psychology, 64(6), 1152–1168. http://doi.org/10.1037/0022-006X.64.6.1152 Holm-Denoma, J. M., Richey, J. A., & Joiner, T. E. (2010). The latent structure of dietary restraint, body dissatisfaction, and drive for thinness: A series of taxometric analyses. Psychological Assessment, 22(4), 788–797. http://doi.org/10.1037/a0020132 Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., … Wang, P. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. http://doi.org/10.1176/appi.ajp.2010.09091379 Kollman, D. M., Brown, T. A., Liverant, G. I., & Hofmann, S. G. (2006). A taxometric investigation of the latent structure of social anxiety disorder in outpatients with anxiety and mood disorders. Depression and Anxiety, 23(4), 190–199. http://doi.org/10.1002/da.20158 Leyro, T. M., Bernstein, A., Vujanovic, A. A., McLeish, A. C., & Zvolensky, M. J. (2011). Distress Tolerance Scale: A confirmatory factor analysis among daily cigarette smokers. Journal of Psychopathology and Behavioral Assessment, 33(1), 47–57. http://doi.org/10.1007/s10862-010-9197-2. Linehan, M. M. (1993). Cognitive-behavioral Treatment of Borderline Personality Disorder. Guilford Press.
Liu, R. T. (2016). Taxometric evidence of a dimensional latent structure for depression in an epidemiological sample of children and adolescents. Psychological Medicine, 46(6), 1265– 1275. http://doi.org/10.1017/S0033291715002792 Liu, R. T., Jones, R. N., & Spirito, A. (2015). Is adolescent suicidal ideation continuous or categorical? A taxometric analysis. Journal of Abnormal Child Psychology, 43(8), 1459– 1466. http://doi.org/10.1007/s10802-015-0022-y Markon, K. E., Chmielewski, M., & Miller, C. J. (2011). The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review. Psychological Bulletin, 137(6), 1093. http://doi.org/10.1037/a0025727 McHugh, R. K., Daughters, S. B., Lejuez, C. W., Murray, H. W., Hearon, B. A., Gorka, S. M., & Otto, M. W. (2011). Shared variance among self-report and behavioral measures of distress intolerance. Cognitive Therapy and Research, 35(3), 266–275. http://doi.org/10.1007/s10608-010-9295-1 McHugh, R. K., Kertz, S. J., Weiss, R. B., Baskin-Sommers, A. R., Hearon, B. A., & Björgvinsson, T. (2014). Changes in distress intolerance and treatment outcome in a partial hospital setting. Behavior Therapy, 45(2), 232–240. http://doi.org/10.1016/j.beth.2013.11.002 McHugh, R. K., & Otto, M. W. (2011). Domain-general and domain-specific strategies for the assessment of distress intolerance. Psychology of Addictive Behaviors, 25(4), 745–749. http://doi.org/10.1037/a0025094 McHugh, R. K., & Otto, M. W. (2012). Refining the measurement of distress intolerance. Behavior Therapy, 43(3), 641–651. http://doi.org/10.1016/j.beth.2011.12.001 Meehl, P. E. (1973). MAXCOV-HITMAX: A taxonomic search method for loose genetic
syndromes. In P. E. Meehl (Ed.), Psychodiagnosis: Selected Papers (pp. 200–224). Minneapolis: University of Minnesota Press. Meehl, P. E. (1977). Specific etiology and other forms of strong influence: Some quantitative meanings. Journal of Medicine and Philosophy, 2(1), 33–53. http://doi.org/10.1093/jmp/2.1.33 Meehl, P. E. (1995). Bootstraps taxometrics. Solving the classification problem in psychopathology. The American Psychologist, 50(4), 266–275. http://doi.org/10.1037/0003066X.50.4.266 Meehl, P. E., & Yonce, L. J. (1994). Taxometric analysis: I. Detecting taxonicity with two quantitative indicators using means above and below a sliding cut (MAMBAC procedure). Psychological Reports, 74(3, Pt 2), 1059–1274. Norr, A. M., Allan, N. P., Macatee, R. J., Keough, M. E., & Schmidt, N. B. (2014). The effects of an anxiety sensitivity intervention on anxiety, depression, and worry: Mediation through affect tolerances. Behaviour Research and Therapy, 59, 12–19. http://doi.org/10.1016/j.brat.2014.05.011 Nugent, K. L., Chiappelli, J., Rowland, L. M., Daughters, S. B., & Hong, L. E. (2014). Distress intolerance and clinical functioning in persons with schizophrenia. Psychiatry Research, 220(1–2), 31–36. http://doi.org/10.1016/j.psychres.2014.07.026 Olatunji, B. O., Broman-Fulks, J. J., Bergman, S. M., Green, B. A., & Zlomke, K. R. (2010). A taxometric investigation of the latent structure of worry: Dimensionality and associations with depression, anxiety, and stress. Behavior Therapy, 41(2), 212–228. http://doi.org/10.1016/j.beth.2009.03.001 Peterson, R. A., & Reiss, S. (1992). Anxiety Sensitivity Index Revised. Worthington, OH:
International Diagnostic Systems. Prisciandaro, J. J., & Tolliver, B. K. (2015). Evidence for the continuous latent structure of mania and depression in out-patients with bipolar disorder: Results from the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD). Psychological Medicine, 45(12), 2595–2603. http://doi.org/10.1017/S0033291715000513 Ruscio, J., Haslam, N., & Ruscio, A. M. (2006a). Introduction to the taxometric method: A practical guide. Mahwah, NJ: Lawrence Erlbaum Associates. Ruscio, J., Haslam, N., & Ruscio, A. M. (2006b). Introduction to the taxometric method: A practical guide. Mahwah, NJ: Lawrence Erlbaum. Ruscio, J., Ruscio, A. M., & Carney, L. M. (2011). Performing taxometric analysis to distinguish categorical and dimensional variables. Journal of Experimental Psychopathology, 2(2), 170–196. http://doi.org/10.5127/jep.010910 Ruscio, J., Ruscio, A. M., & Meron, M. (2007). Applying the bootstrap to taxometric analysis: Generating empirical sampling distributions to help interpret results. Multivariate Behavioral Research, 42(2), 349–386. http://doi.org/10.1080/00273170701360795 Ruscio, J., Walters, G. D., Marcus, D. K., & Kaczetow, W. (2010). Comparing the relative fit of categorical and dimensional latent variable models using consistency tests. Psychological Assessment, 22(1), 5–21. http://doi.org/10.1037/a0018259 Ruscio, J., & Wang, S. (2017). RTaxometrics: Taxometric Analysis. Retrieved from https://cran.r-project.org/web/packages/RTaxometrics/index.html Schmidt, N. B., Richey, J. A., & Fitzpatrick, K. K. (2006). Discomfort intolerance: Development of a construct and measure relevant to panic disorder. Journal of Anxiety Disorders, 20(3), 263–280. http://doi.org/10.1016/j.janxdis.2005.02.002
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., … Dunbar, G. C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59, 22–33. Simons, J. S., & Gaher, R. M. (2005). The Distress Tolerance Scale: Development and validation of a self-report measure. Motivation and Emotion. http://doi.org/10.1007/s11031-005-79553 Sirota, A. D., Rohsenow, D. J., Dolan, S. L., Martin, R. A., & Kahler, C. W. (2013). Intolerance for discomfort among smokers: comparison of smoking-specific and non-specific measures to smoking history and patterns. Addictive Behaviors, 38(3), 1782–1787. http://doi.org/10.1016/j.addbeh.2012.10.009 Taylor, S., Rabian, B., & Fedoroff, I. C. (1999). Anxiety sensitivity: Progress, prospects, and challenges. In S. Taylor (Ed.), Anxiety sensitivity: Theory, research, and treatment of fear of anxiety (pp. 339–353). Mahwah, NJ: Lawrence Erlbaum. Timpano, K. R., Broman-Fulks, J. J., Glaesmer, H., Exner, C., Rief, W., Olatunji, B. O., … Schmidt, N. B. (2013). A taxometric exploration of the latent structure of hoarding. Psychological Assessment, 25(1), 194–203. http://doi.org/10.1037/a0029966 Trafton, J. A., & Gifford, E. V. (2011). Biological bases of distress tolerance. In M. J. Zvolensky, A. Bernstein, & A. A. Vujanovic (Eds.), Distress tolerance: Theory, research, and clinical applications (pp. 80–102). New York, New York: The Guilford Press. Tull, M. T., & Gratz, K. L. (2013). Major depression and risky sexual behavior among substance dependent patients: The moderating roles of distress tolerance and gender. Cognitive Therapy and Research, 37(3), 483–497. http://doi.org/10.1007/s10608-012-9490-3
Waller, N. G., & Meehl, P. E. (1998). Multivariate taxometric procedures: Distinguishing types from continua. Thousand Oaks, CA: SAGE Publications. Walters, G. D., & Ruscio, J. (2009). To sum or not to sum: Taxometric analysis with ordered categorical assessment items. Psychological Assessment, 21(1), 99–111. http://doi.org/10.1037/a0015010 Weeks, J. W., Carleton, R. N., Asmundson, G. J. G., McCabe, R. E., & Antony, M. M. (2010). “Social Anxiety Disorder carved at its joints”: Evidence for the taxonicity of social anxiety disorder. Journal of Anxiety Disorders, 24(7), 734–742. http://doi.org/10.1016/j.janxdis.2010.05.006 Widiger, T. A., Livesley, W. J., & Clark, L. A. (2009). An integrative dimensional classification of personality disorder. Psychological Assessment, 21(3), 243–255. http://doi.org/10.1037/a0016606 Witte, T. K., Holm-Denoma, J. M., Zuromski, K. L., Gauthier, J. M., & Ruscio, J. (2017). Individuals at High Risk for Suicide Are Categorically Distinct From Those at Low Risk. Psychological Assessment, 29(4), 382–393. http://doi.org/10.1037/pas0000349 World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva: World Health Organization.
Figure 1 Graphical results of MAMBAC, MAXEIG, and LMode analyses for Sample 1
Figure 2. Graphical results of MAMBAC, MAXEIG, and LMode analyses for Sample 2
Table 1. Measures of average indicator validity, correlations, and CCFI values for taxometric analyses. Indicator correlations
Sample 1
BetweenGroup Validity MAMBAC 1.80 (.31) MAXEIG 1.87 (.25) L-Mode 1.78 (.22) Mean
CCFI .20 .32 .47
Full sample
Taxon Complement
.54 (.07) .54 (.07) .54 (.07)
.21 (.07) .29 (.08) .16 (.07)
.19 (.10) .09 (.08) .21 (.10)
.33
2 MAMBAC 1.90 .16 .53 .17 .15 (.35) (.05) (.06) (.08) MAXEIG 1.94 .36 .53 .16 .20 (.34) (.05) (.07) (.06) L-Mode 1.64 .34 .53 .28 -.02 (.25) (.05) (.12) (.08) Mean .29 Note. Correlations and between-group validity (Cohen’s d) values are averaged across indicators. Skew and kurtosis are averaged across indicators in the full sample.
Highlights:
This is the first known examination of the latent structure of distress intolerance
Results indicated that distress intolerance is best characterized dimensionally
Dimensional classification and measurement may better reflect distress intolerance