What were they thinking?

Drug and Alcohol Dependence 80 (2005) 191–200

What were they thinking? Adolescents’ interpretations of DSM-IV alcohol dependence symptom queries and implications for diagnostic validity Tammy Chung ∗ , Christopher S. Martin Western Psychiatric Institute and Clinic, University of Pittsburgh Medical Center, Pittsburgh Adolescent Alcohol Research Center, 3811 O’Hara Street, Pittsburgh, PA 15213, USA Received 7 April 2004; received in revised form 24 March 2005; accepted 24 March 2005

Abstract Objective: DSM-IV alcohol dependence criteria of tolerance to alcohol and drinking more or longer than intended have relatively high prevalence among youth, and may be vulnerable to false positive symptom assignments that degrade diagnostic validity. We conducted a methodological study of DSM-IV symptom queries used to assess alcohol tolerance and impaired control over drinking to determine potential sources of measurement error. Method: Adolescents recruited from addictions treatment participated in either a focus group (n = 9) or an individual interview (n = 41) to provide data on their interpretation of selected items contained in a semi-structured diagnostic interview. Results: When alcohol tolerance is operationally defined as a change in quantity to obtain the same effect, large individual differences in the change in quantity that represents a high level of tolerance limit the utility of this operational definition as an indicator of dependence. The symptom “drinking more or longer than intended”, includes the embedded assumption that a limit on use had been set. Teens, however, typically intended to become intoxicated, rather than to keep to a limit. Conclusions: Adolescents’ understanding of symptom queries suggests how validity of DSM-IV alcohol symptoms and diagnoses can be improved through greater attention to developmental considerations affecting assessment. © 2005 Elsevier Ireland Ltd. All rights reserved. Keywords: DSM-IV; Adolescent; Alcohol; Diagnosis; Dependence

1. Introduction DSM-IV alcohol diagnoses of abuse and dependence, which were derived from clinical experience and research with adults, have been applied to adolescents with little to no modification of the criteria. When applied to youth, DSM-IV alcohol diagnoses have shown some concurrent validity in that groups classified as having no alcohol diagnosis, abuse and dependence tend to differ on measures of alcohol and other drug involvement (Lewinsohn et al., 1996; Winters et al., 1999). However, important limitations of the diagnostic criteria when applied to adolescents also have been identified (Martin and Winters, 1998). Some limitations reflect ∗

Corresponding author. Tel.: +1 412 383 2630; fax: +1 412 246 6550. E-mail address: [email protected] (T. Chung).

0376-8716/$ – see front matter © 2005 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.drugalcdep.2005.03.023

developmental differences between adolescents and adults in typical pattern and context of alcohol use, and the type of alcohol-related consequences most commonly experienced by youthful drinkers (Deas et al., 2000). Importantly, developmental differences also may impact teens’ interpretation of and responses to questions that are used to determine the presence of DSM-IV alcohol symptoms, potentially resulting in false positive symptom assignments that degrade diagnostic validity. Epidemiologic surveys estimate that up to 14% of adolescents meet criteria for a lifetime DSM-IV alcohol diagnosis (Kessler et al., 1994; Lewinsohn et al., 1996) with 6% meeting criteria for an alcohol use disorder (AUD) in the past year (SAMHSA, 2004). Further, up to an additional 12% of youth report alcohol dependence symptoms, but do not meet criteria for a DSM-IV AUD (Chung et al., 2002). A

192

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

review of five community surveys and four clinical adolescent samples found that the symptoms reported most often by adolescents were two dependence criteria: tolerance to alcohol (Tolerance) and drinking more or longer than intended (Larger/Longer) (Chung et al., 2002). The relatively high prevalence of these two dependence criteria accounts for the majority of symptomatic cases that do not meet criteria for a DSM-IV AUD, and affects the prevalence of the physiologic subtype of alcohol dependence (i.e. presence of either tolerance or withdrawal). Variation in the rates of Tolerance and Larger/Longer across adolescent community surveys has a large impact on the estimated prevalence of alcohol abuse and dependence, and the ratio of these two diagnoses. Because abuse is generally considered a milder AUD than dependence (American Psychiatric Association, 2000), the higher prevalence of two dependence criteria relative to any abuse criterion suggests that these two dependence criteria, in particular, may be susceptible to false positive symptom assignments in adolescents. False positive symptom assignments can occur when a symptom is operationalized by a measure that does not adequately assess the presence of the clinical phenomenon, resulting in a positive symptom rating in the true absence of the phenomenon. Some research has begun to document limitations of DSM-IV Tolerance and Larger/Longer as indicators of alcohol dependence in adolescents (e.g. Chung et al., 2001; Chung and Martin, 2002). In descriptions of the alcohol dependence syndrome concept (Edwards and Gross, 1976), which influenced DSM-IV alcohol dependence criteria, tolerance to alcohol is a core feature of dependence. DSM-IV operationally defines tolerance as the subjective experience of a change in alcohol effects (i.e. “markedly increased amounts of alcohol to achieve the desired effect”). Such a definition of tolerance based on a change in quantity to achieve an effect, however, is problematic (Chung et al., 2001; O’Neill and Sher, 2000). Research with adolescents indicates large individual differences in both the initial and later drinking quantities needed to obtain the same effect, which limit the utility of DSM-IV’s change-based definition of tolerance (Chung et al., 2001). That is, individuals who report low initial quantities to become intoxicated are more likely to report larger percentage increases to obtain the same effect (e.g. increase from 2 to 5 drinks), while those with high initial quantities tend to report smaller percentage increases (e.g. increase from 6 to 9 drinks). Thus, in rating the presence of tolerance based on a “marked increase” as defined by DSM, the tolerance symptom is frequently assigned to lighter drinkers, and often is not assigned to heavier drinkers. It remains unclear whether the wide range in reported quantities, particularly initial quantities to obtain an effect, is due primarily to differences in the time frames that adolescents consider or to individual differences in initial sensitivity to alcohol. Data on the cognitive process that adolescents engage in when responding to symptom queries are critical to improving the ability to distinguish between tolerance as a normative developmental phenomenon and

as a clinically significant symptom associated with dependence. In describing impaired control over alcohol use, another core feature of dependence, Edwards and Gross (1976) focused on an underlying subjective compulsion to drink as the key clinical phenomenon. Two DSM-IV dependence criteria most directly indicate impaired control over alcohol use: “using more or longer than intended” (Larger/Longer) and “persistent desire or repeated unsuccessful efforts to quit or cut down on drinking” (Quit/Cut Down) (cf. International Classification of Diseases, 10th edition (ICD-10); World Health Organization, 1992). In adolescents, Larger/Longer and Quit/Cut Down showed some concurrent validity in that the symptoms were positively correlated with dependence severity and an independent measure of unsuccessful attempts to limit drinking (Chung and Martin, 2002). However, a number of potential false-positive and false-negative symptom assignments also were identified (Chung and Martin, 2002). In adults, a methodologic study of Larger/Longer suggested that among those with low to moderate levels of drinking, socially based reasons, rather than a compulsive pattern of drinking, explained a substantial proportion of symptom assignments (Caetano, 1999). Likewise, teens’ endorsement of Larger/Longer may reflect peer influences, inexperience with alcohol effects, poor judgment or impulsivity, rather than a compulsive pattern of use (McBride et al., 2000; Sastre et al., 2000). Further, an Australian study of youth found that many teens reported drinking to intoxication (e.g. McBride et al., 2000), suggesting that adolescent drinkers typically do not set the limits on alcohol use behavior that Larger/Longer requires. Little is known about how adolescents interpret symptom queries related to impaired control over alcohol use. Diagnostic interviews (e.g. Structured Clinical Interview for DSM-IV (SCID); First et al., 1999) incorporate DSM criteria as standardized symptom queries with few to no changes in wording to ensure that interview items correspond to diagnostic criteria. Given high rates of endorsement of Tolerance and Larger/Longer by adolescents in the context of relatively short drinking histories, increased understanding of the process by which teens report that these symptoms did or did not occur can reveal the extent to which teens and researchers differ in their understanding of an item’s intent. In some cases, respondents may transform a question that is difficult to understand into an item with a different meaning that is easier to answer (Belson, 1981; Schwarz, 1999). Such a situation may result in a respondent appearing to understand and to answer an item reliably, although the response provided may actually have little to do with the item’s actual intent (Schmidt and Room, 1999). Data on adolescents’ interpretation of DSM-IV AUD symptom items can reveal problems in communicating the intent of an item, and can provide guidelines for improving validity of symptom assessment. Research methods that utilize focus groups and cognitive interviews provide a systematic way to identify and reduce measurement error associated with the structure and con-

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

tent of interview items (Schwarz, 1999; Turner et al., 1992; Trotter, 1990). Methodologic studies have been conducted to examine adults’ interpretation of AUD symptom queries in community samples (Caetano, 1999), as well as crosscultural differences in adults’ interpretation of alcohol symptom queries (Room et al., 1996; Schmidt and Room, 1999). In research with adolescents, focus groups have been used to inform the development of an adolescent nicotine dependence questionnaire by providing data on how dependence manifests differently in adolescents and adults (Nichter et al., 2002; O’Loughlin et al., 2002). In addition, qualitative interview data have been used to better capture drinking patterns in culturally diverse groups of adolescents (Strunin, 2001). The purpose of this methodological study was to identify potential problems in the structure and wording of alcohol symptom items used to assess DSM-IV criteria of Tolerance, Larger/Longer and Quit/Cut Down. The study focused specifically on symptoms related to tolerance and impaired control over alcohol use because they represent core dependence constructs, and because the high prevalence of these particular symptoms has substantial downstream effects on diagnostic prevalence. The symptom queries under investigation came from a semi-structured interview (i.e. adapted SCID, Martin et al., 1995; Table 1) that has demonstrated good inter-rater reliability when used with teens (Martin et al., 2000). A systematic question appraisal procedure was used as a first step in identifying potential problems in question wording and structure. Next, based on problems identified in the question appraisal procedure, focus group and interview data were collected to determine how the identified problems may contribute to measurement error during

193

assessment, and to increase understanding of the process by which adolescent drinkers arrive at their responses to the alcohol symptom queries.

2. Methods 2.1. Overview Three methods were used, in sequence, to identify potential problems in the structure and wording of alcohol symptom items querying alcohol tolerance and impaired control over alcohol use: (1) question appraisal system (QAS; Willis and Lessler, 1999), (2) focus groups and (3) individual cognitive interviews. The QAS provided a preliminary systematic method of identifying potential problems in question wording and structure. QAS results were used to identify specific topics for discussion in focus groups and individual cognitive interviews. Two focus groups (ns = 5 and 4) were conducted prior to any of the individual cognitive interviews (n = 41). Cognitive interviews were used to provide detailed data on the process by which adolescents interpret and respond to SCID symptom queries related to tolerance and impaired control over alcohol use. Focus groups and cognitive interviews were audio taped. Analysis of focus group and cognitive interview data aimed to identify trends in adolescents’ interpretation of selected interview items, and methods to rectify potential sources of miscommunication between respondent and researcher (DeMaio and Rothgeb, 1996; Ghiselli et al., 1981). Informed consent was obtained prior to study participation (youth < 18 years old provided written assent and their par-

Table 1 DSM-IV criteria related to tolerance and impaired control over alcohol use and SCID symptom queries DSM-IV criterion Tolerance (1) Tolerance, as defined by either of the following:

(a) A need for markedly increased amounts of the substance to achieve intoxication or desired effect (b) Markedly diminished effect with continued use of the same amount of the substance Larger/Longer (2) The substance is often taken in larger amounts or over a longer period than was intended

Quit/Cut Down (3) There is a persistent desire or unsuccessful efforts to cut down or control substance use

SCID symptom queries

SCID “middle column” rating guidelines

Did you find that you needed to drink a lot more in order to get drunk than you did when you first started drinking? (Could you drink a lot more than most people without really getting drunk?) IF YES: How much more?

(Dep-1)(a): Need for markedly increased amounts of alcohol in order to achieve intoxication or desired effect. Additional, non-DSM-IV guidelines: At least 50% increase in quantity Subsequent quantity must be >5 drinks

IF NO: What about finding that when you drank the same amount, it had much less effect than before?

(Dep-1)(b): Markedly diminished effect with continued use of the same amount

Did you often find that when you started drinking you ended up drinking much more than you were planning to? What about drinking for a much longer period of time than you were planning to? What about drinking all day or going on multi-day drinking binges?

(Dep-3a): Alcohol often taken in larger amounts than the person intended (Dep-3b): Alcohol often taken over a longer period than the person intended

Did you try to cut down or stop drinking alcohol? IF YES: Did you ever actually stop drinking altogether? How many times did you try to cut down or stop altogether? IF UNCLEAR: Did you want to stop or cut down? IF YES: Is this something that you kept worrying about?

(Dep-4)(a): One or more unsuccessful efforts to cut down, quit, or control alcohol use (Dep-4)(b): Persistent desire to cut down, quit, or control alcohol use

194

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

ent provided written consent). Informed consent procedures included discussion of the confidentiality of the information provided, as well as limits to confidentiality (i.e. report of current suicidal or homicidal thoughts and physical or sexual abuse). Teens received gift certificates for their participation. 2.2. Participants in focus groups and cognitive interviews Adolescents and young adults (n = 50, 54% male, 89% Caucasian) were recruited from a variety of in- and outpatient addictions treatment sites. Participants’ age ranged from 15 to 21, average age was 18.7 (S.D. = 1.4). All participants reported lifetime alcohol use at least once per month for at least 6 months. The sample provided a broad range of severity in alcohol use patterns and problems among both older and younger drinkers, as well as males and females, to maximize conceptual rather than statistical generalization of results (Mays and Pope, 2000). In the total sample, almost all of the teens (98%) met criteria for a lifetime DSM-IV alcohol diagnosis (among those with an alcohol diagnosis: 45% abuse (n = 22), 55% dependence (n = 27)). Because this study focuses on two alcohol dependence symptoms, Tolerance and Larger/Longer, we describe the potential effect that these two symptoms may have on the prevalence of alcohol dependence diagnoses in the sample. Among those with a lifetime alcohol dependence diagnosis and who provided data on all seven dependence criteria (n = 251 ), 10 cases with 3 alcohol dependence symptoms would no longer meet criteria for dependence if Tolerance and/or Larger/Longer were excluded (i.e. at least 3 out of 7 dependence symptoms is needed for a dependence diagnosis) and 3 cases with 4 alcohol dependence symptoms would no longer have a dependence diagnosis if Tolerance and/or Larger/Longer were excluded. Thus, 13 of 25 cases, or about half of those with an alcohol dependence diagnosis, could potentially have their diagnostic status affected by false positive assignments of Tolerance and Larger/Longer. 2.3. Adapted SCID and Question Appraisal System 2.3.1. Adapted SCID The adapted SCID (Martin et al., 1995, 2000) closely follows the wording of DSM-IV SUD criteria. Adaptations to the SCID include breaking down more complicated criteria into components (e.g. the presence of “drinking more than intended” and “drinking for a longer time than intended” are each coded separately; Table 1), providing guidelines regarding the threshold for recurrence to assign a symptom, and the 1 Two cases did not provide data on all seven alcohol dependence symptoms due to time constraints (i.e. the minimum number of questions needed to clearly establish a diagnosis of dependence were asked for these two cases). In one case, Tolerance, difficulty quitting/cutting down, and much time spent drinking were assigned. In the other case, Tolerance, reduced activities to drink instead, and use despite psychological problems (depression) were assigned.

inclusion of follow-up probes to aid determination of the clinical significance of the symptom. 2.3.2. QAS method The QAS (Willis and Lessler, 1999) was used as a preliminary systematic method of identifying potential problems in the wording and structure of SCID alcohol symptom questions that may contribute to miscommunication regarding an item’s intent. The QAS uses eight steps to identify problems related to: (1) the ease with which an item can be read aloud to a respondent (i.e. complex question construction), (2) need for clearer introductions to questions or question instructions, (3) communicating the intent or meaning of the item, (4) assumptions that are embedded in an item, (5) whether the respondent may not know or have difficulty remembering the requested information, (6) querying sensitive topics, (7) adequacy of response options and (8) other problems. The first author used the fully structured QAS protocol to code potential problems in item wording and structure for each SCID alcohol symptom item. Problem codes provided a framework for determining topic areas to be covered in focus groups and cognitive interviews. 2.4. Focus groups Focus groups have been recommended to facilitate the development and modification of survey items (Fowler, 1995; Nichter et al., 2002; Trotter, 1990). Two focus groups were facilitated by the first author, who has extensive experience with adolescent substance users. This facilitator described the purpose of the group discussion as an opportunity for researchers to better understand how adolescents interpret certain interview items related to alcohol use. Discussion in both groups focused on the constructs of alcohol tolerance and impaired control over alcohol use, and the SCID interview items used to assess these domains. Adolescents were assured that there were no right or wrong answers. After warm-up discussion items on type of alcohol consumed, usual context of consumption and reasons for use, teens were asked to discuss their understanding of the constructs of tolerance to alcohol and difficulties in controlling alcohol use behavior. Teens also were asked to provide their feedback on SCID items used to assess these constructs. For example, participants were asked the types of experiences they thought a question was asking about, and what might make a specific item difficult or confusing to answer. Potential problems in question wording or structure that were identified using the QAS also provided a framework for guiding discussion content. Teens were asked to provide feedback on how to rephrase or modify questions that they perceived as difficult or confusing to answer. Each group lasted 2.5 h. Immediately after the group, the facilitator generated a detailed narrative summary on key themes for each of the topics covered. Focus group data, in combination with QAS results, were used to inform the topic areas to be covered in individual cognitive interviews.

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

2.5. Individual cognitive interviews 2.5.1. Procedure overview Cognitive interviewing methods were used as another means of identifying specific problems with interview items (Fowler, 1995; Schwarz, 1999). The main goals of the interview procedure are to determine what the adolescent thinks the question is asking, and what specific phrases or terms used in a question mean to the adolescent (Willis, 1994). In individual interviews, using a “think aloud procedure” (e.g. Turner et al., 1992; Willis, 1994), teens provided in-depth information on the process used to determine whether or not to endorse an item. Cognitive interviews were conducted by two clinicians who were highly experienced in SCID administration and who were trained on specific cognitive interviewing procedures (e.g. the think aloud method). Training included manual-based instruction (Willis, 1994), review of a training video associated with the manual, and consultation with Research Triangle Institute’s Cognitive Interviewing Lab. Interviews took an average of 2 h to complete. Teens were asked about the type and quantity of alcohol consumed to facilitate conversion to a “standard drink” (e.g. a 12 oz can of beer, 1.5 oz of 80-proof liquor). Because of the study’s focus on in-depth description of thought processes, not all participants provided data on all cognitive interview topics of interest due to time constraints. Approximately half of the initial cognitive interviews focused on topics related to assessment of tolerance, while the remainder focused on assessment of impaired control over alcohol use. In addition, due to the nature of respondent answers and experience, not all topics related to tolerance or impaired control over alcohol use could be covered. Thus, the number of participants who provided interview data on a given topic varied. Interviewers generated interview item ratings and a detailed narrative of the participant’s responses for each interview topic that was discussed. 2.5.2. Cognitive interview format The interviewer first explained that “we are trying to improve certain interview questions because they may be difficult to understand or confusing to answer. I’ll ask you a few questions about the kind of things that you thought about when deciding on your answer, so that we can improve the questions that we ask”. Using a semi-structured format for the think aloud procedure, the interviewer asked the respondent to (1) paraphrase his/her understanding of the question, (2) discuss the thought process he/she went through in deciding on an answer (e.g. What kinds of situations were they thinking about in coming up with an answer?), (3) identify parts of the question that were unclear or confusing and (4) define key terms in the interview item (Fowler, 1995; Turner et al., 1992). As part of the introduction to the cognitive interview, the clinician also went through an example of the think aloud procedure with the teen. The interviewer’s role was to document the teen’s responses to each of the four interview tasks for a given SCID item, and to clarify or ask the teen to elaborate on a response using

195

open-ended questions with minimal bias in phrasing that may “lead” a teen to provide a particular response (e.g. “What’s an example of that?”, “Say more about that”). 2.6. Analysis of focus group and cognitive interview data Detailed, narrative summaries produced by the focus group facilitator and cognitive interviewers and audiotapes provided data on adolescents’ interpretation of selected SCID items. The focus group summaries identified key themes and issues that were used to inform topics to be covered in cognitive interviews. Cognitive interview narratives provided detailed summaries of adolescents’ interpretation of items, and recorded specific suggestions on how to modify an item to more effectively convey its intended meaning. Data contained in cognitive interview narratives were tallied across all participants who were queried about an item. In this way, qualitative trends were identified, that is, interpretations of an item or identified problems that emerged repeatedly (Willis, 1994). Cognitive interview narratives also were used to identify relevant quotes; audiotapes were then transcribed to document a respondent’s comments. 3. Results 3.1. Question Appraisal System: identifying topic areas for discussion The main problems related to SCID items querying alcohol tolerance and impaired control over alcohol use involved communicating the intent of the question (e.g. vague language, undefined terms) and assumptions embedded in items. Due to ambiguous wording, questions could be interpreted in multiple ways. Specifically, for the tolerance item, phrases such as “a lot more”, “same effect” and “first started drinking” were not operationally defined for the respondent. Similarly, for Larger/Longer, “larger amount”, “plans to drink” and “longer time” was not operationally defined. Assumptions embedded in symptom queries also were identified as potential problems that may contribute to miscommunication between a respondent and researcher. Larger/Longer includes the embedded assumption that the respondent wants to limit or control drinking as the basis for asking the question. For both Larger/Longer and Quit/Cut Down, failed attempts and desire to cut down or stop drinking are assumed to indicate a compulsive pattern of use. Early in the drinking history, however, some youth may report failed attempts to limit or stop drinking, typically because of social activities (McBride et al., 2000), rather than as the result of a compulsive pattern of use. 3.2. Focus group and cognitive interview findings 3.2.1. Alcohol tolerance Teens generally understood the concept of alcohol tolerance. Several definitions of tolerance were identified, reflect-

196

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

ing the complexity of the construct. First, and most commonly, tolerance was defined as the “body is getting used to the alcohol, so you have to drink more to get drunk” (age 18, African–American female, alcohol dependence). A less common variation of this definition was, “able to drink more before feeling sick” (age 20, White female, alcohol dependence). Second, tolerance means “able to handle more alcohol and not get smashed. It is a good thing, means that you can hold your liquor” (age 17, White male, alcohol abuse). Third, tolerance was defined (infrequently) as, “I need to drink a lot more than other people to get the same effect as them” (age 17, White male, no alcohol diagnosis). Almost all participants endorsed the SCID item, “needed to drink a lot more to get the same effect”. When asked to “think aloud” about the change in quantity needed to obtain the same effect, some teens voiced uncertainty regarding the time frames to compare: “Do you mean when I first started drinking (7th grade) or in 9th grade? Even from 9th grade, there was a change compared to the amount that I need to get buzzed now, from about 8 beers to 12–13. In 9th grade, I would drink so much that I would puke or drink a lot and get drunk. But now, I can drink a lot and not get as drunk” (age 17, White male, alcohol abuse). Given potential differences in the time frames that teens considered when answering the tolerance item, standardized time frames (as operationally defined by the focus groups) were tested. The initial drinking period was operationally defined as when the teen “first started regular drinking” (i.e. drinking at least one day per week for at least 2 months) and “current” as past 6 months or just prior to treatment entry for those who reduced use after treatment. The “alcohol effect” to be considered was “getting drunk”. Even when considering a standardized alcohol effect and time frames, there was a relatively high degree of variability in average quantity consumed per occasion. Interview data (n = 41) indicated that initial quantities at the start of regular drinking ranged from 1 to 12 standard drinks per occasion, and later quantity ranged from 2 to 24 drinks. There was a correspondingly wide range in the change in drinking quantity thought to represent a “marked” increase (i.e. between 2 and 12 drinks). The modal increase that led a teen to endorse needing to drink “a lot more” to get the same effect was only two or three drinks. One teen that increased from 2 to 3 beers to 5 beers reported, “I guess that increasing by 2 beers is a lot” (age 20, White female, alcohol abuse). The item querying an alternative manifestation of tolerance, “feel much less of an effect at the same quantity”, was generally viewed as more difficult to answer. Several teens explained the difficulty in terms of confusion as to whether this item referred to a change in effect within a drinking episode (e.g. “the buzz wore off as the evening progressed” (age 20, White male, alcohol dependence)) or to a change in alcohol effect from a given quantity that occurred over different ages. If the question was intended to refer to effects at different ages or periods in the drinking history, then the time periods to be compared were vague (e.g. when I first started drinking com-

pared to now?). Despite their potential confusion, few teens requested clarification before providing a response. Some adolescents (11 of 24 teens) were not aware of developing a tolerance to alcohol until they were specifically asked about the phenomenon. According to one teen, “I didn’t notice that I built up a high tolerance. . . I never built up a tolerance where I can just sit around and drink all night and not throw up or anything like that” (age 20, White male, alcohol abuse; increase from 6 to 9 drinks). Other teens were aware of developing tolerance to alcohol. For example, “I was trying to build up a tolerance. I wanted to fit in with my friends. I wanted to at least keep up, and maybe drink more” (age 16, White male, alcohol dependence). Likewise, another teen reported, “Tolerance is a social thing. To be able to pound as much as other people, to drink the same as them” (age 20, White male, alcohol abuse). 3.2.2. Larger/Longer For both the “Larger” and “Longer” parts of this AUD criterion, teens need to consider at least two components: (1) one’s plans or intentions regarding consumption and (2) one’s ability to successfully follow through on their plans. Because teens typically endorsed “drinking more” (i.e. “Larger”; 21 of 34 teens who were queried about the topic) rather than “drinking for a longer time” (i.e. “Longer”; only 3 of 10 teens who were queried), results focus on teens’ interpretation of “Larger”. In general, teens typically did not report setting a relatively low limit on drinking quantity per occasion. Many teens that endorsed “Larger” (16 out of 34 who were queried) did not report setting a relatively low limit on their drinking. These teens interpreted “more than intended” as drinking more than the usual amount needed to become intoxicated. For example, “I would have enough (5–6 drinks), but would have 5–6 more than usual. Don’t know why. Because they were there. . . Usually drink with the intention of getting drunk” (age 17, White female, alcohol dependence). Thus, some teens that endorsed “Larger” did so because they drank more than the usual quantity needed to get drunk, but contrary to the intent of the question, typically expressed no desire to limit their alcohol use. One teen explained the item’s complexity: “That is hard to answer because most people I know don’t set a limit. If they do, they may say, I’ll have 1–2 or just a few, then they start to feel good, they’re at a party, and they have a few more, and drink until they’re hammered. If I said that I did have a few, I wouldn’t be very committed to keeping to the limit. I usually want to get drunk when I drink. I would set a limit because I need to get up the next morning. I can still do that if I’m over the limit. I just go to work hungover” (age 18, White male, alcohol dependence). Among teens that did set a relatively low limit on quantity to be consumed (5 out of 34 teens), 4 teens set a limit of 1–3 drinks and 1 teen set a limit of 4 drinks. However, few of the teens that set a relatively low limit for themselves reported a serious intention or commitment to keeping to the limit set. For example, “Maybe I’d say to myself that I’d have 1–2.

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

I don’t want to be drunk because I’m not in a drinking mood. But then, I have the drinks, and I want to party. Let me have another drink because now I’m in a drinking mood, and I’ll stop when the bar closes” (age 20, African–American female, alcohol abuse). Another aspect of “Larger” involves the embedded assumption that “drinking more than intended” occurred as the result of a compulsion or irresistible urge to drink. However, many teens (18 of 34 teens) reported drinking more than the usual quantity needed to become drunk due to social or circumstantial reasons (e.g. involvement in a drinking game). One teen reported, “I get caught up in the conversation, I’m having a good time, and I don’t want to go home. It’s a social thing, but I also never tested myself to see if I could stop after 1–2 drinks” (age 20, White female, alcohol abuse). Another teen noted, “I could stop. I just don’t. Isn’t not craving or feeling like I have to drink more. I just do” (age 20, African–American female, alcohol abuse). A few teens (3 of 21 teens who endorsed “Larger”) reported drinking beyond a desired effect (e.g. buzzed) because of their inexperience in gauging how factors such as beverage alcohol content (e.g. beer versus liquor) and rate of consumption affected the timing and intensity of the alcohol effect obtained. “I might drink more because the effect doesn’t hit until later” (age 18, White female, alcohol dependence). Of note, teens typically did not report that drinking quantity was constrained by limited availability of alcohol; in this sample, most teens reported no difficulty accessing enough alcohol to obtain the desired effect (e.g. intoxication).2 While misunderstanding of “Larger” may result in false positive symptom assignments, it is also important to consider possible false negatives. A subgroup of teens said “no” to “Larger” because they did not set a limit on their alcohol use, but reported relatively frequent episodes of drinking to intoxication (10 out of 34 teens). For example, one teen, who understood the “Larger” item, reported, “That’s like for someone who intends to have only 2, but they end up having 10. When I drink, I intend to get drunk” (age 19, White female, alcohol dependence). Some teens who said “no” to “Larger” engaged in episodes of heavy, apparently dyscontrolled drinking, but would not be assigned the symptom because they did not set a low limit on their alcohol use.

2 Estimation of the magnitude of measurement error for Larger/Longer. In a separate sample of teens recruited from addictions treatment (n = 74, 92% white, 67% male, age range 14–18), we recorded a teen’s response to the initial probe, and then made a clinical rating of the presence of the Larger/Longer symptom based on their responses to follow-up items to determine context and clinical significance. A majority of teens (65%) responded “yes” to the initial probe for “drinking more than intended” (Larger), however, after follow-up probes, 46% of those who endorsed the initial probe did not receive a clinical symptom rating. Fewer teens (46%) endorsed “drinking for a longer time than intended” (Longer); 53% of those who endorsed the initial probe did not receive a clinical symptom rating. For both items, almost half of the teens that initially endorsed the item did not receive a clinical symptom rating after follow-up probes.

197

3.2.3. Persistent desire or failed attempts to cut down or quit Teens typically did not want to limit their alcohol use prior to treatment. Most of the adolescents reported wanting to stop or reduce alcohol use because they were in treatment, and typically were “forced” to attend treatment either by family or the court system. Teens who reported “worry” or “persistent thoughts” related to cutting down or stopping alcohol use (2 of 27 teens queried about the topic) had several other dependence symptoms. When teens did try to cut down on their own (4 of 27), they reported reducing the frequency of drinking, but not changing the quantity consumed per occasion. Actual attempts to limit alcohol use were most common during treatment episodes. The most frequently reported reasons for limiting alcohol use included: being in treatment, concern about performance at work or school and reducing use to please a parent or significant other. 3.2.4. Impaired control over other substance use Due to the pattern of responses on items related to impaired control over alcohol use, we examined the possibility that adolescents may misunderstand these items because they have not experienced a compulsive pattern of substance use. Considering alcohol and tobacco, two commonly used substances, teens reported greater difficulty stopping use of tobacco than alcohol (26 of 29 teens who were queried about the topic and who reported lifetime use of both). Modal difficulty to quit tobacco was 10 out of 10 (10 = “almost impossible”), while modal difficulty to stop drinking was 2 out of 10. Many of these teens (24 of 26 teens who used both substances) said they did not “crave” alcohol. In contrast, almost all teens that smoked (25 out of 26) reported “craving” or “needing” cigarettes. For example, “Cigarettes would be the hardest drug to give up. I cannot live without it” (age 19, White male, alcohol and tobacco dependence). “The worst thing that could happen is if someone would take my cigarettes away” (age 20, White female, alcohol abuse and nicotine dependence). Further, a subgroup (5 of 26 teens) expressed a clear desire to reduce tobacco use on their own, repeated failed efforts to stop or cut down, and preoccupation with smoking if prevented from use. “I tried to cut down to 3 cigarettes in a day, but couldn’t because I needed to have it” (age 17, White female, alcohol and nicotine dependence). Adolescents’ “need” to smoke cigarettes was typically reported in relation to the experience of nicotine withdrawal. These data suggest that many clinical teens do experience and can report a compulsion to use in relation to tobacco (i.e. “need” to use), but not usually in relation to alcohol (i.e. “want”, but not usually “need” to use).

4. Discussion Symptoms representing dependence criteria of tolerance to alcohol and impaired control over alcohol use have high prevalence among youth, and may have a substantial effect on

198

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

estimates of the prevalence of alcohol dependence in adolescents (Chung et al., 2002). In this sample, more than half of those with an alcohol dependence diagnosis could potentially have their diagnostic status changed by false positive assignments of Tolerance and Larger/Longer. Study results suggest that the way in which these core features of dependence are operationally defined may result in erroneous symptom assignments for some teens. The item querying tolerance as a change in quantity to obtain the same effect revealed relatively large individual differences in the quantity that represented a “large increase”, and appeared to include some cases of normative tolerance development, as well as clinically significant, high levels of tolerance. For the item querying “drinking more than intended”, embedded assumptions of setting a relatively low limit on drinking and failing to keep to the limit due to a compulsion to drink may be missed by respondents, resulting in potential false positive symptom assignments. These relatively complex phenomena require a carefully constructed, developmentally appropriate sequence of items to maximize validity of assessment. Findings also suggest that interviewers need to be aware when further probing is indicated to minimize false positive symptom assignments and to verify teens’ understanding of an item’s intent. Improving question structure and symptom assessment, however, requires clear description of the phenomenon of interest, that is, its prototypical manifestations and threshold for clinical significance, as well as its boundary conditions. Tolerance represents a core feature of dependence American Psychiatric Association (2000). However, the operationalization of the tolerance symptom as a change in quantity to obtain the same effect has limitations that can result in errors in symptom assignment. Despite standardizing the effect to be obtained (e.g. “drunk”) and the time frames to be considered, there was a wide range in initial drinking quantity, as well as the quantity thought to represent “a lot more” to obtain the same effect. The modal increase to obtain the same effect was relatively low (i.e. an increase of 2 or 3 drinks). Depending on one’s initial drinking quantity, an increase of 2–3 drinks may represent normative tolerance development (i.e. increase from 2 to 4 drinks) or a relatively high level of tolerance development. When assessing adolescent drinkers, there is a need to better distinguish tolerance that occurs as a normative developmental phenomenon from a clinically significant, high level of tolerance that represents a more salient milestone in the development of dependence. Improving assessment of the tolerance symptom requires clear definition of the construct. Some have suggested that tolerance can be inferred without quantifying a specific degree of change. Interview results suggest that some teens recognize tolerance as a pattern of heavy drinking that occurs without the clear signs of intoxication that are almost inevitably observed in the non-tolerant drinker (i.e. the ability to hold one’s liquor). In DSM-IV-TR (American Psychiatric Association, 2000), blood alcohol concentration (BAC) is suggested as a method to evaluate the degree of tolerance: “an individual with a concentration of 100 mg of ethanol per

deciliter of blood who does not show signs of intoxication can be presumed to have acquired at least some degree of tolerance to alcohol. At 200 mg/dL, most non-tolerant individuals demonstrate severe intoxication” (p. 218). Alternatively, the National Council on Alcoholism (NCA, 1972) operationalized tolerance as a BAC > 150 mg/dL without gross evidence of intoxication or consumption of one-fifth of a gallon of whiskey or its equivalent daily, for more than one day, by a 180-pound individual. These alternative operational definitions of a high level of alcohol tolerance both avoid problems associated with tolerance defined as a change in quantity to obtain a certain effect. In addition, the alternatives provide more stringent guidelines for determining the clinical significance of tolerance development. However, it is likely that these operational definitions will not assign the tolerance symptom to many adolescents who have, indeed, developed a high degree of tolerance to alcohol. Further research is needed to develop and test alternative operational definitions of tolerance (e.g. non-change based definitions of tolerance), and to identify thresholds, which are appropriate for use with adolescents, that indicate when a phenomenon such as tolerance is clinically significant. Larger/Longer is also a relatively complex criterion to assess in adolescents and adults as well (e.g. Caetano, 1999). This criterion requires the respondent to consider certain embedded assumptions: (1) an intention to limit drinking and (2) difficulty in keeping to limits due to a compulsion to drink. These elements also are critical to Quit/Cut Down. Results suggest that few teens set the type of limits on alcohol use that Larger/Longer requires, and should not be assigned this symptom. In addition, rather than exceeding a predetermined limit because of a compulsion to drink, many teens “drank more” due to social factors, reasons that do not justify assignment of a symptom indicating impaired control over alcohol use. A similar finding also has been reported for adults with relatively low levels of alcohol use in a community survey who reported “drinking more than intended” (Caetano, 1999). Significantly, however, many teens did understand the subjective experience of a compulsion to use a substance (i.e. nicotine), but only rarely reported the phenomenon in relation to alcohol. To improve assessment of criteria related to impaired control over alcohol use, the criteria need to be broken down into a structured sequence of less complex, more specific questions. The question sequence would ideally include a brief introduction to symptom queries to provide a context for the questions that follow and to facilitate recall of relevant experiences. The question sequence also needs to address each component of Larger/Longer and Quit/Cut Down separately, making explicit any embedded assumptions. Specific examples of prototypical manifestations of the phenomenon of interest across a range of severity and circumstances could be provided to further facilitate recall and increase shared understanding of the meaning and intent of items (Schmidt and Room, 1999; Woody and Cacciola, 1994). Toward this end, questionnaires have been developed for use with youth

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

that assess subjective compulsion to drink based on the phenomenology of obsessive–compulsive disorder (e.g. Deas et al., 2000). In clinical adolescents, items related to preoccupation with drinking and craving (e.g. “needing” a drink) predicted worse alcohol outcomes (Deas et al., 2000), and may provide useful starting points for developing alternative, adolescent-appropriate alcohol symptom queries. Although this study focused on false positive symptom assignments, results also highlight the possibility of false negative assignments of certain symptoms. A particular irony of DSM-IV dependence is that an individual may cross the threshold for the number of criteria needed for diagnosis after they attempt to cut down or limit use, that is, when they make initial efforts at behavior change. For example, some adolescents do not show impaired control over alcohol use (as defined in DSM-IV) because they reported no desire or efforts to limit drinking. However, their pattern of heavy, daily or almost daily drinking appears to be dyscontrolled based on quantity and frequency of consumption, suggesting a manifestation of dependence that may be observed prior to any desire or attempts to limit use. Some of these teens may not meet criteria for a DSM-IV dependence diagnosis because they have not attempted to limit their use, or may finally meet criteria for dependence when their use becomes more intermittent (i.e. failed attempts to quit or cut down). In this regard, ICD-10 dependence (World Health Organization, 1992) includes “strong desire or sense of compulsion to take the substance”, a symptom of impaired control over substance use behavior that does not involve setting a limit on use, although it may be subject to other limitations (e.g. Sayette et al., 2000). In addition, false negative assignments of Tolerance, when defined as a change in quantity consumed, may occur among teens who report high initial levels of use, but no marked increase in quantity consumed per occasion. Certain study limitations warrant comment. Participants were primarily Caucasian, and all were drawn from clinical sources. Recruitment focused on teens and young adults from addictions treatment, rather than a representative sample of community adolescents, because clinical teens are more likely to have actually experienced the dependence constructs of interest. However, differences in the interpretation of interview items by community and clinical respondents have been documented in research with adults (Caetano, 1999), and merit attention in youthful samples. Because this study utilized a clinical sample of adolescents and young adults, estimation of the magnitude and direction of measurement error and its effect on diagnostic prevalence in the adolescent population remain to be determined. Results focused on the three DSM-IV alcohol dependence criteria that may be most susceptible to false positive symptom assignments, and that have the most impact in determining whether an adolescent meets DSM-IV criteria for an alcohol dependence diagnosis. Further research is needed to identify potential sources of measurement error for other DSM-IV AUD criteria and for other drugs. In addition, gender and ethnic differences in drinking pattern that may affect symptom assignment were

199

not addressed, and constitute an important area for future investigation. Although DSM-IV AUD diagnoses tend to show some concurrent validity in adolescents, the high prevalence of Tolerance and Larger/Longer early in the drinking history suggests the possibility of false positive symptom assignments among youth that can reduce diagnostic validity. Given that these constructs represent core features of dependence, study findings emphasize the importance of ensuring that respondents and researchers have a common understanding of an item’s meaning and intent, and the need for clear guidelines to determine the clinical significance of a phenomenon. As Robins (1989) noted, the development of diagnostic systems and assessment methods, particularly item structure and wording (i.e. “diagnostic grammar”), is symbiotic. Improving the validity of diagnostic assessment at the symptom level involves focusing attention on clear description of the prototypical manifestations of a given phenomenon, and how the presentation of that phenomenon may differ across age, gender, culture and illness severity.

Acknowledgements This research and the preparation of this manuscript were supported by National Institute on Alcohol Abuse and Alcoholism K01 AA00324 and K02 AA00249. The authors thank Dr. Judith Lessler and Rachel Caspar for their consultation on the project. Portions of this project were presented as a poster at the 2002 annual meeting of the Research Society on Alcoholism, San Francisco, CA.

References American Psychiatric Association, 2000. Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV), text revision. American Psychiatric Association, Washington, DC. Belson, W.R., 1981. The Design and Understanding of Survey Questions. Gower Press, Aldershot, England. Caetano, R., 1999. The identification of alcohol dependence criteria in the general population. Addiction 94, 255–267. Chung, T., Martin, C.S., 2002. Concurrent and discriminant validity of DSM-IV symptoms of impaired control over alcohol consumption in adolescents. Alcohol: Clin. Exp. Res. 26, 485–492. Chung, T., Martin, C.S., Armstrong, T.D., Labouvie, E.W., 2002. Prevalence of DSM-IV alcohol diagnoses and symptoms in adolescent community and clinical samples. J. Am. Acad. Child Adolesc. Psychiatry 41, 546–554. Chung, T., Martin, C.S., Winters, K.C., Langenbucher, J.W., 2001. Assessment of alcohol tolerance in adolescents. J. Stud. Alcohol 62, 687–695. Deas, D., Riggs, P., Langenbucher, J., Goldman, M., Brown, S., 2000. Adolescents are not adults: developmental considerations in alcohol users. Alcohol: Clin. Exp. Res. 24, 232–237. DeMaio, T.J., Rothgeb, J.M., 1996. Cognitive interviewing techniques: in the lab and in the field. In: Schwarz, N., Sudman, S. (Eds.), Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research. Jossey-Bass/Pfeiffer, San Francisco, CA.

200

T. Chung, C.S. Martin / Drug and Alcohol Dependence 80 (2005) 191–200

Edwards, G., Gross, M., 1976. Alcohol dependence: provisional description of a clinical syndrome. Br. Med. J. 1, 1058–1061. First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., 1999. Structured Clinical Interview for DSM-IV Axis I Disorders, Research Version Non-patient Edition (SCID-I/NP). New York State Psychiatric Institute, New York. Fowler, F., 1995. Improving Survey Questions: Design and Evaluation. Sage Publications, Thousand Oaks, CA. Ghiselli, E.E., Campbell, J.P., Zedeck, S., 1981. Measurement Theory for the Behavioral Sciences. W.H. Freeman, San Francisco, CA. Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hugher, M., Eshleman, S., Wittchen, H.-U., Kendler, K.S., 1994. Lifetime and 12month prevalence of DSM-III-R psychiatric disorders in the United States: results from the National Comorbidity Survey. Arch. Gen. Psychiatry 51, 8–19. Lewinsohn, P.M., Rohde, P., Seeley, J.R., 1996. Alcohol consumption in high school adolescents: frequency of use and dimensional structure of associated problems. Addiction 91, 375–390. Martin, C.S., Winters, K.C., 1998. Diagnosis and assessment of alcohol use disorders among adolescents. Alcohol Health Res. World 22, 95–105. Martin, C.S., Kaczynski, N.A., Maisto, S.A., Bukstein, O.G., Moss, H.B., 1995. Patterns of DSM-IV alcohol abuse and dependence symptoms in adolescent drinkers. J. Stud. Alcohol 56, 672–680. Martin, C.S., Pollock, N.K., Bukstein, O.G., Lynch, K.G., 2000. Inter-rater reliability of the SCID alcohol and substance use disorders section among adolescents. Drug Alcohol Depend. 59, 173–176. Mays, N., Pope, C., 2000. Assessing quality in quantitative research. Br. Med. J. 320, 50–52. McBride, N., Farringdon, F., Midford, R., 2000. What harms do young Australians experience in alcohol-use situations. Aust. N. Z. J. Public Health 24, 54–59. National Council on Alcoholism, 1972. Criteria for the diagnosis of alcoholism. Am. J. Psychiatry 129, 127–135. Nichter, M., Nichter, M., Thompson, P.J., Shiffman, S., Moscicki, A.-B., 2002. Using qualitative research to inform survey development on nicotine dependence among adolescents. Drug Alcohol Depend. 68, S41–S56. O’Loughlin, J., Kishchuk, N., DiFranza, J., Tremblay, M., Paradis, G., 2002. The hardest thing is the habit: a qualitative investigation of adolescent smokers’ experience of nicotine dependence. Nicotine Tob. Res. 4, 201–209. O’Neill, S.E., Sher, K.J., 2000. Physiological alcohol dependence symptoms in early adulthood: a longitudinal perspective. Exp. Clin. Psychopharmacol. 8, 493–508.

Robins, L.N., 1989. Diagnostic grammar and assessment: translating criteria into questions. Psychol. Med. 19, 57–68. Room, R., Janca, A., Bennett, L.A., Schmidt, L., Sartorius, N., 1996. WHO cross-cultural applicability research on diagnosis and assessment of substance use disorders: an overview of methods and selected results. Addiction 91, 199–220. Sastre, M.T.M., Mullet, E., Sorum, P.C., 2000. Self-assessment of inebriation from external indices. Addict. Behav. 25, 663– 681. Sayette, M.A., Shiffman, S., Tiffany, S.T., Niaura, R.S., Martin, C.S., Shadel, W.G., 2000. The measurement of drug craving. Addiction 95, S189–S210. Schmidt, L., Room, R., 1999. Cross-cultural applicability in international classifications and research on alcohol dependence. J. Stud. Alcohol 60, 448–462. Schwarz, N., 1999. Self-reports: how the questions shape the answers. Am. Psychol. 54, 93–105. Strunin, L., 2001. Assessing alcohol consumption: developments from qualitative research methods. Soc. Sci. Med. 53, 215–226. Substance Abuse and Mental Health Services Administration (SAMHSA), 2004. Results from the 2003 National Survey on Drug Use and Health: Detailed Tables [database on the Internet], c2004 [cited 2004 Oct 14], available from: http://www.oas.samhsa.gov. Trotter, R., 1990. Cross-cultural Applicability Research Study Focus Group Manual. University of AZ, Flagstaff, Arizona. Turner, C.F., Lessler, J.T., Gfroerer, J.C. (Eds.), 1992. Survey measurement of drug use: Methodological studies. D.H.H.S. Publication No. (ADM) 92-1929. U.S. Deptartment of Health and Human Services, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, Washington, DC. Willis, G., 1994. Cognitive interviewing and questionnaire design: a training manual, Working Paper #7, National Center for Health Statistics. Research Triangle Institute, Raleigh, NC. Willis, G., Lessler, J., 1999. Question Appraisal System Manual. Research Triangle Institute, Raleigh, NC. Winters, K.C., Latimer, W., Stinchfield, R.D., 1999. The DSM-IV criteria for adolescent alcohol and cannabis disorders. J. Stud. Alcohol 60, 337–344. Woody, G.E., Cacciola, J., 1994. Severity of dependence. In: Widiger, T.A., Frances, A.J., Pincus, H.A., First, M.B., Ross, R., Davis, W. (Eds.), DSM-IV Sourcebook, 1. American Psychiatric Association, Washington, DC. World Health Organization, 1992. The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. World Health Organization, Geneva, Switzerland.