Problems in the application of the Addiction Severity Index (ASI) in rural substance abuse services

journal of Substance Abuse, 7(2), 17.L 188 (1995)

Problems in the Application of the Addiction Severity Index (ASI) in Rural Substance Abuse Services Jennifer S. Wertz Bonnie L. Cleaveland Robert S. Stephens Virginia

Tech

The aim of this research was to test the utility of the Addiction Severity Index (ASI) in a rural community substance abuse outpatient treatment center and to predict future alcohol and drug problems based on ASI information. Substance abuse counselors used the ASI to assess the problems of 89 adult clients at intake, and research staff assessed drug use outcomes 3 months later. There were significant improvements between intake and follow-up on the scores on the alcohol, drug, legal, and family sections of. the ASI. Scores on the alcohol abuse, drug abuse, and medical sections of the ASI predicted some of the variance (16%) in drug and alcohol use outcomes, but psychiatric, medical, legal, and employment problems were not significant predictors. There was some difficulty in obtaining adequate reliability in counselors’ severity ratings for alcohol problems. The relationship of counselors’ severity ratings to the more objective composite scores varied substantially by subsection of the ASI. The improvements in alcohol and drug use indices appeared to be more related to subjective appraisals of problems related to use rather than due to changes in drug-using behavior. Implications of these findings and recommendations for further research using the ASI are discussed.

The aim of this research was to examine the utility of the Addiction Severity Index (ASI; McLellan, Luborsky, Woody, & O’Brien, 1980) incorporated into intake procedures in substance abuse programs offered in a rural community substance abuse treatment center. There are several advantages of having standardized, comprehensive assessment in a treatment facility. If the same information is collected on every client, predicting outcome and matching clients to treatment becomes possible. Standardized assessment, particularly when it is comprehensive, may increase therapists’ knowledge of the full range of client

We would like to acknowledge and thank the counselors and staff at New River Valley Substance Abuse Services. We would also like to thank Curtis Greaves for help in data management and Cindy Koziol for help in manuscript preparation. This work was funded by a student research award to Jennifer Wertz and Bonnie Cleaveland 1.rom the Department of Mental Health, Mental Retardation, and Substance Abuse Services of Virginia. Correspondence and requests for reprints should be sent to Robert S. Stephens, Department of Psychology, Virginia Tech, Blacksburg, VA 24061-0436. 175

J.S. Wertz, B.L. Cleaveland, and R.S. Stephens

176

and strengths.

difficulties

propriate o.b.jective cxions. The

This

knowledge

can be used to provide

for individual needs. Standardized measurement of treatment efficacy,

AS1 has been

used with a variety

assessment information

of populations

assessment of seven areas of functioning stance abuse clients, specifically medical,

made by the interviewers

apfol de-

and is a comprehensive

relevant for treatment-seeking employment, alcohol abuse,

abuse, legal, family, and psychiatric functioning. from each section: severity scores and composite judgments

treatment

also is important crucial to funding

to provide

subdrug

Two types of scores are derived scores. The severity scores are an immediate,

reliable,

valid, and

efficient summary of the client’s problem in each area. However, McLellan and colleagues (McGahan, Griffith, 8c McLellan, 1986) argued that the subjectivity of the severity Therefore,

scores makes them a poor criteria for measuring composite scores were mathematically derived

treatment outcome. from the client’s an-

swers to several questions within each subsection and were intended to provide a more objective measure of severity of problems. Higher scores on both of these types of ASI summary scores indicate greater severity of problems in that area. McLellan and colleagues (McLellan et al., 1985; McLellan, McLellan, Luborsky, Woody, O’Brien, & Druley, 1983; McLellan, borsky, O’Brien, & Druley, 1983) used the AS1 to predict outcome client-to-treatment have been used

matching. The interviewer severity scores at intake often to predict composite scores at follow-up, thereby assessing the

ability of more subjective clinical judgments to predict at follow-up (McCahan et al., 1986). The ASI has been valid across El-Guebaly,

et al., 19%; Woody, LLIand to stud)

clients 1992;

objective problem levels fo;nd to be reliable and

of varying demographic features and problems McLellan et al., Stoffehnayr, 1985; Benishek,

(Hodgins 8c Humphreys,

Lee, SC Mavis, 1989). In general, McLellan, Luborsky, et al. (1983) and McLellan et al. (1986) found that psychiatric severity emerged as one of the strongest predictors of outcome across the treatment programs and that when clients were matched to treatment based on the psychiatric severity score, correctly matched clients had better outcomes at follow-up (McLellan, Woody, et al., 1983). Two subsequent studies by different researchers have not found as much support fog the importance of psychiatric severity. Kosten, Kounsaville, and Kleber, (1987) found in a group of opioid addicts that only ASI medical severity ratings predicted posttreatment alcohol composite scores and that employment ratings predicted drug composite scores posttreatment. Alterman, Kushner, and Holahan (1990) found that higher alcohol and drug severity ratings at baseline were associated with improved alcohol problem composite scores at 1 and 6 months posttreatment in a sample of alcoholics. Psychiatric severiry was less consistently related to alcohol problem outcome. In both studies, however, the ASI gave useful predictive information. In this study, the AS1 was administered as part of the regular intake protocol by existing treatment staff. This procedure differs from most studies that have used research technicians and specially trained counselors to conduct the ASI interviews. For instance, McLellan and others (McLellan et al., 1985; McLellan et al., 1980; McLellan, Luborsky, et al., 1983; McLellan, O’Brien, Kron, Alterman,

Addiction

Severity

Index in Rural Settings

177

& Druley, 1980) used research technicians with bachelor’s degrees and/or counselors with master’s degrees to establish the reliability of the severity ratings and to predict outcomes after treatment. The greater consistency and control gained by using research-specific staff is important for establishing the initial utility of a standardized assessment. However, the generalizability of assessment procedures developed under more controlled conditions to their actual use in clinical situations is less clear. This study was designed in part to test the feasibility and utility of using the AS1 as a standard part of substance abuse treatment operations. In this study, a rural subject population from southwestern Virginia was examined. The need for research on mental health care in rural populations is clear. DeLeon, Wakefield, Schultz, Williams, and Vanden Bos (1989) cited the increased rates of alcohol abuse in the rural population as one reason that the rural population needs more health care. It is, therefore, important to evaluate the effectiveness of treatment programs currently in place and assess the predictors of outcome in this population. The use of the AS1 introduced practical, standardized assessment and allowed for an examination of client characteristics predictive of outcome in an understudied population of substance abusers.

METHOD Subjects

Subjects were 89 adults over age 18 seeking services or who were referred for alcohol and/or drug usage problems at a publicly funded rural substance abuse services center in southwest Virginia. Clients who were in prison or who were seen in the local community mental health center were not included in the population sampled. These 89 subjects were obtained out of 138 sequential intakes, of, which 22 refused to participate, 25 could not be reached for follow-up, and 2 for whom treatment participation data were not available. Seventy-five percent of the sample were male. The average age of the subjects was 31.28 (SD = 9.76) ranging from 19 to 61 years of age. The majority of the sample was white (91%). Subjects had a mean of 11.94 (SD = 2.72) years of education. Fifty-one percent of the sample were never married, and 21% were currently married. The percentage of subjects on probation or parole was 34%, and 43% of the subjects were referred by the criminal justice system. Subjects reported working an average of 8.47 (SD = 9.73) days in the past 30 days, with 64% of the subjects currently employed. Forty-six percent of the sample were classified as primarily alcohol abusers based on a question within the AS1 concerning substance of abuse. Marijuana and polydrug abusers accounted for 13% and 16% of the remaining sample, respectively. Analyses of intake data showed that subjects who were not able to be reached for follow-up did not differ significantly from subjects who completed follow-up interviews on age, race, gender, marital status, years of employment, years of education, days of work in the past 30 days, legal status, or on any composite or severity scores of the ASI.

J.S. Wertz, B.L. Cleaveland, and R.S. Stephens

178

Procedures All clients

over the age of 18 referred

to substance

abuse

services

were asked

by the substance abuse counselor if they would participate in a study assessing the outcomes of treatment programs. They were informed that participation was not mandatory, would not affect their treatment, and that they could discontinue participation at any time. They were also told that they would be paid $20 to complete a follow-up interview 3 months later, which would take approximately one-half hour. Clients who indicated that they would like to participate were given a consent form to sign. Each subject was then assessed using the AS1 interview and additional routine intake information by one of the counselors at the substance abuse services center. Subjects were assigned to treatment

based

on the judgments

of the substance

abuse counselors who were not told of prior findings regarding ASI predictors of outcome. Treatment options included outpatient individual, tamily, and group counseling of variable intensity and duration. Treatment participation data were collected by file review and assessed as the total number of hours of therapist contact regardless of treatment format. Clients’ participation in multiple types of treatment and the overlap of treatment content made it impractical to distinguish differences in the type of treatment received. For example, 3Z3%’of the subjects attended intensive outpatient treatment, 63% received individual counseling, 22’$ were in an education program, 1 1% attended an extended care group, and 5% partici pated in a relapse prevention group. ?‘he percentage of subjects in each treatment totals more than 100% because many subjects were involved in more than one treatment. A follow-up assessment session was conducted 3 months afier the intake ASI and consisted of a readministration mended by Fureman, Parikh, Bragg,

of a subset of the ASI questions recomand McLellan (1990). The follow-up inter-

views were conducted at the substance abuse offices by two of the authors (B.C. 8c -1.W.) who were unaware of the sub$ects’ intake status and treatment participation. Those subjects unable to attend follow-up interviews were assessed by phone using the same interview (see McLellan et al., 1985). Several steps were taken to increase the validity and reliability of subjects’ selfreports (see Labor, Stephens, 8c Marlatt, 1987). Subjects were guaranteed confidentiality in the consent form, and the interviewers, well-trained in the use of the assessment tools, provided clear instructions. The subjects also were told that their reports on the follow-up assessment would not be shared with their counselors, and the follow-up interviews were conducted by separate research staff. rather than counselors, further reducing any incentives to misreport drug or alcohol use and related problems. Finally, the payment of a $20 incentive to complete the follow-up interview was clearly stated to be independent, of reported substance use at follow-up. Measures The complete Fifth Edition ASI was administered AS1 items recommended by Fureman et al. (1990)

at intake and a subset of the as measures of outcomes in

Addiction

Severity

Index in Rural Settings

179

each of the assessed areas was administered at follow-up. The AS1 is a structured interview that has been reported to take 45 minutes to 1 hour to administer. It assesses problem severity in seven areas: medical condition, employment, drug use, alcohol use, illegal activity, family relations, and psychiatric condition. Objective questions asked in each symptom area involve the number, duration, and extent of the problem both in the past 30 days and over the client’s lifetime. The client then uses a 5-point scale ranging from 0 (not at aU) to 4 (extreme&) to report on the severity and importance of the problem area for him or her. It is noteworthy that using the AS1 as part of the initial intake resulted in longer administration time, often stretched across two sessions. The need to obtain additional treatment-center-specific data as well as establishing rapport with clients in distress contributed to the length of time needed. Two different types of measures are derived from the AS1 interview. Severity ratings were subjective judgments made by the trained interviewer using both objective and subjective information collected in the interview. First, the interviewer narrowed his or her preliminary rating of severity to a P-point range on a lo-point scale ranging from 0 (no real problem) to 9 (extreme problem) based on a specific subset of items in each section. After the client gave his or her subjective rating of severity, the interviewer then modified the preliminary rating by choosing either the higher or lower value of the previously established range. Composite scores were recommended to be used in outcome assessments (McLellan et al., 1985); these were created by standardizing and summing sets of interrelated items within each problem area. Each score was adjusted for the range of possible answers so that each question within the composite was equally weighted, and the total score was adjusted for the number of questions comprising the composite measure. According to McLellan et al. (1985), the composite scores correlate .88 on average with the interviewer’s severity ratings but provide a “more empirical evaluation of patient change and treatment effectiveness” (p. 413). McLellan et al. (1985) performed test-retest reliability analyses using a paired t test procedure on the composite scores over a 3-day period. The results were nonsignificant, indicating that composite scores remained stable over the 3-day period. Additional discriminative validity was established by comparing the AS1 area severity and composite scores with questionnaires rating similar problems (McLellan et al., 1985).

Interviewer

Training

Ten counselors were trained in the use of the ASI according to the training packet provided by its authors (Fureman et al., 1990). The counselors represented a variety of educational levels and years of experience. For instance, the highest educational degree achieved by the counselors ranged from the attainment of a GED (n = 1) to the receipt of a master’s degree in a counseling-related field (n = 4). All but one counselor had some college experience. The mean number of years of experience as a substance abuse counselor was 6.35 with a range from 1 to 14 years. All counselors had or were working toward certiflca-

180

J.S. Wertz, B.L. Cleaveland,

and R.S. Stephens

tion as a substance abuse counselor, which required 3 years of direct abuse work and 400 hours in substance abuse education.

substance

Each counselor was given a notebook with the AS1 manuals, copies of the AS1 form, and additional abbreviated summaries of instructions for each section. The training packet also included instruction manuals and guidelines for COIIducting quently authors

role plays. Counselors were asked to read all materials and were subsetrained by the authors over six group sessions, totaling 10.5 hours. .l‘he role played the administration of each section of the AS1 during group

training sessions, answered questions, of the items. After they had gained

and facilitated discussion about the intent some familiarity with the ASI, counselors

practiced role plays in pairs, observed by and obtaining feedback from the authors. Counselors also practiced administering the AS1 to actual clients before the beginning of data collection. Counselors gained significant practice in scoring the AS1 by watching and rating eight tapes of full-length AS1 interviews created imately

to establish reliability. In addition 1.5 hours of individual feedback

to the group and training

training sessions, approxwere given to each coun-

selor. Interrater In order

to assess the reliability

Reliability

of the severity

ratings

and composite

scores,

interviewers rated 12 videotapes of actual (n = 8) and role-played (n = 4) AS1 interviews. These tapes were rated by all counselors and the two follow-up assessment interviewers to determine interrater reliability. Reliability of composite scores was computed for the 10 counselors who completed intakes for the study and for the two authors who conducted the follow-up interviews. Reliability for severity scores was computed only for the counselors and not the research staff, as severity ratings were not made at follow-up. ‘liable 1 shows the reliability coefficients for composite and severity scores in each section

of the ASI.

Interrater

reliability

was computed

in two ways across

the 12 AS1 interview tapes, as suggested by McLellan et al. (1985). The Spearman-Brown formula (Winer, 1962) assessed the variability between coun-

Table

1.

Interrater

Reliability

on the AS1

Severity Ratings SpearmanASI Subsection

Brown

Mean Correlation

Composite SpearmanBrown

Score Mean Correlation

Addiction

Severity

181

Index in Rural Settings

selors’ ratings across all tapes taking into account the number of counselors. Spearman-Brown reliability coefficients for severity scores ranged from .85 for the alcohol section to .98 for the medical section including only the counselors (see Table 1, Column 1). Spearman-Brown reliability coefficients for composite scores ranged from .98 for the employment section to .99 for the legal section, including the counselors A more conservative

and the follow-up interviewers (see Table reliability analysis also was performed

1, Column 3). following the

model of McLellan et al. (1985). Each counselor’s severity rating across tapes was correlated with each other counselor’s rating for each section of the ASI. These correlations were then averaged to yield the mean interrater reliability for each section. A similar analysis was performed with the composite scores based on each counselor’s scoring of the ASI. McLellan and colleagues (1985) noted that such ratings should be uniformly lower than Spearman-Brown correlations and that the two methods may represent the upper (Spearman-Brown) and lower (correlational) bounds of reliability. The mean interrater reliability for severity scores using this second method ranged from .41 (Alcohol) to .83 (Medical; see Table 1, Column 2). For composite scores, the range was from .83 (Employment) to .97 (Legal;

see Table

1, Column

4). RESULTS

Description

of Sample at Intake

The mean composite scores and severity ratings at intake are presented in Table 2. The highest severity scores were in the alcohol, drug, and family problems areas, whereas employment, family, and alcohol problems attained the highest composite scores. These data suggest that the areas that the interviewers rated as most troublesome for subjects differed somewhat from those identified by the relatively objective items used to compute the composite scores. To further examine the relationship between the interviewers’ severity ratings and the mathematically

Table 2.

Medical Employment Alcohol Drug Legal Family Psychiatric

derived

composite

scores,

the correlations

between

Mean Intake Severity Scores and Intake and Follow-up

Composite

composite

Scores

Intake Severity

Intake Composite

M

SD

M

SD

M

SD

1.90 2.44 4.90 3.62

2.63 2.64 2.63 2.88 2.2.5 2.57 2.46

.22 .56 .28 .09 .20 .32 .21

.30 .30 .22 .07 .22 .18 .21

.23 .53 ,17*** .06*** .13** .27* .16*

.29 .32 .18 .06 .I7 .16 .I8

1.84 3.81 2.96

Note. N = 89. SD indicates standard deviation. intake to follow-up are noted with asterisks. *p <.05. **p <.Ol. ***p <.OOl.

Significant

changes

Follow-up Composite

in composite

scores from

J.S. Wertz, B.L. Cleaveland, and R.S. Stephens

182

Table 3.

Correlations

and Follow-up

of Intake Severity Ratings With Intake

Composite

Scores Intake Severity Ratings

Composite Score

Medical

Employment

Alcohol

Drug

Legal

.0-i .I I .lti .Ol Ti*“”

Family

Psychiatric

IutcikP

Medical

.x7***

Imlploynlrtlt Alrohol Drug LegA Family I’svchiatric

.I7

~

- .oli .I0 .04 .1x* ,y j**

.(I4

93”

I r,

9,;“”

.I’)”

.:35***

.22*

,r,,j*w:

-.I

?Gr,i”“*

.?I” ~. 1tj -.I4 .24*

.25* .o?l I6 -.lO

IVOlP. A\’ = 89. *p < .(I.‘. **p < .(I I. ***p

.12 ‘,‘j* .A .2-l*

I

.“(I

.OO .24” ,‘~(j”i’ .I I .(j5*“”

.2:<* .OH

,yc,**

,#)“$:”

IO

.24” .:w** .oi .‘Lti** ,i”“:“*

< .OOl

scores and severity ratings at intake were computed (see upper Diagonal correlations represent agreement between severity

half of Table 3). and composite

scores for various sections of the ASI. Although all diagonal correlations are significant, the average agreement of the two indices of problem severity was only .58. The correlations between severity scores and composite scores for the employment,

Comparison

drug,

and alcohol

of Problems

sections

were particularly

at Intake and Follow-Up

In order to detect changes in functioning measures multivariate analysis of variance intake and follow-up composite scores for analysis revealed a significant multivariate .O 1. Repeated measures univariate analyses nificant reductions in composite scores for p < .OO1, drug problems, F( 1, X7) = 18.8 1, 9.99, p < ,002, family problems, F( 1, 88) = lems, F (1, 87) = 4.20, p < .04, f ram intake

low.

Subject Characteristics

in each of the AS1 areas, a repeated (MANOVA) was performed on the the seven scales (see I’able 2). This effect of time, F(7, 79) = 4.39, p < of variance (ANOVAs) showed sigalcohol problems, F( 1, 88) = 23.33, p < .OO1, legal problems, F( 1, 8X) = 5.48, p < .02, and psychiatric probto follow-up.

The items that were used to compute the composite scores for the alcohol and drug abuse sections of the ASI were compared between intake and follow-up using a repeated measures ANOVA. .I‘he items composing the alcohol composite scores are presented in Table 4. In the alcohol section, the number of days sub-

Addiction Severity Index in Rural Settings

183

Intake and Follow-up Composite Score Items

Table 4. Comparisons Between in the Alcohol Section

Intake

Item

M

Days used alcohol Days used alcohol

to intoxication

Days experienced

alcohol

How much

alcohol

How much

subject

Dollars

sDent

problems

problems thinks

bother

he or she

subject

needs treatment

on alcohol

SD

standard

M

SD

6.26

8.01

5.05

7.64

4.29

6.41

3.87

6.70

4.93

8.5X

2.56

6.73*

1.35

1.46

0.69

1.07**

2.21

1.73

1.11

42.76

N&J. N = 89. SD indicates */, < .0.5. **p < .OOl.

Follow-Up

114.34

1.54**

20.60

43.57

deviation

jects reported having problems due to alcohol, the degree to which they were troubled by the problems, and how much they wanted treatment decreased significantly. However, the number of days of use and the frequency of use to intoxication did not change significantly between intake and follow-up. Similarly, Table 5 shows treatment for follow-up but of any drugs.

that the degree to which subjects were bothered by and wanted drug problems was significantly decreased between intake and there were no significant reductions in the number of days of use Abstinence rates between intake (25.8%) and follow-up (31.5%)

were compared using the McNemar test for related samples. Although abstinence rates generally increased, there were no significant differences in abstinence rates between intake and follow-up. Prediction In order problem variables

to explore

client

variables

of Outcomes that were predictive

of drug

and alcohol

outcomes, zero-order relationships between predictor and outcome were computed. As can be seen in the lower panel of Table 3, medical

Table 5. Comparison Between Intake and Follow-Up Composite Score Items in the Drug Section Follow-up

Intake

Item

Mean

Days used drugs Days experienced How much

drug

How much

subject

drug

problems

problems thinks

bother

subject

he or she needs

treatment

SD

Mf3Wl

SD

3.73

7.91

4.84

10.20

2.64

6.56

2.17

6.66

0.90

1.40

.47

1.04*

1.62

1.81

.74

1.31**

No/r. h’ = 89. Although days of drug use were assessed individually for a variety of drugs, a composite index is presented in the interest of space. There were no significant differences between baseline and follow-up drug use for any of the individual drugs. SD indicates standard deviation. *p < .Ol. **1 < .OOl.

J.S.Wertz, B.L. Cleaveland,

184

and alcohol

intake

severity

scores

correlated

significantly

and R.S. Stephens

with follow-up

alcohol

composite scores. Alcohol and drug intake severity scores correlated significantly with drug follow-up composite scores. In order to test the utility of the intake variables as predictors of outcomes, follow-up alcohol and drug composite scores were regressed on blocks of predictors using stepwise multiple regression. ‘l‘he order of variable entry into the equation was patterned after McLellan, Luborsky, et al. (1983). Kelated variables were grouped in blocks and the stepwise procedure selected all significant predictors from a given block before considering variables in subsequent blocks. First, in order to adjust for any pretreatment differences in client characteristics, the demographic variables of age, gender, race, and years of education were entered in a block along with the nunlber of previous drug and alcohol treamerits. .l‘he second block entered included severity ratings in the seven AS1 areas (medical, employment, alcohol, drug, legal, family, and psychiatric problems). The third block included the number of hours of treatment received. The only variables that entered the equation predicting alcohol were the medical, F(l, 87) = 7.27, p < .008, and alcohol, F( 1, 86) = .006, severity scores, explaining a total of 16% of the variance in F (2, 86) = 7.91, 1 < ,001. More severe alcohol problems at baseline

outcomes 7.97, f) < outcome, predicted

more severe alcohol problems at follow-up, and more severe medical problems at baseline predicted less severe alcohol problems at follow-up. In the regression analyses predicting drug composite scores at follow-up, the only variables that entered the equation were the number of previous drug treatments, F (1, 86) = 6.04, p < ,016, and the alcohol severity score at intake, F (1, 85) = 9.58, p < ,003. Together, those variables explained lci%, of the variance in drug outcomes, F (2, 85) = 8.1 1, p < .OO1. A greater nunlber of previous drug drug

treatments outcomes

and more se\Ter-e alcohol at follow-up.

problem

at baseline

predicted

worse

DISCUSSION This study examined the practical application of the AS1 in a mostly rural substance-abusing population by integrating the ASI into the regular intake protocol of an outpatient substance abuse counseling center. As such, the stucly differs from prior research using primarily urban and suburban samples and more research-specific interview procedures (Kosten et al., 1987; McLellan et al., 1992; McLellan et al., 19%). The AS1 severity ratings were able to account for some of the posttreatment improvement on measures of alcohol and clrugrelated problems, but some problems with the reliability and validity of severity ratings and composite scores were revealed. Interrater reliability coefficients indicated that counselors were able to agree on the more objective information elicited by the ASI and reflected in the composite scores. However, agreement by interviewers on the more subjective judgments of problem severity made in each section of the ASI was somewhat less reliable. Most troubling was the particularly low average interrater correlation for the alcohol problem severity measure. ‘I‘he difficulty in obtaining adequate

Addiction

Severity

Index in Rural Settings

reliability

among

ty scores

for either

the alcohol research

severity

185

ratings

or clinical

undermines

purposes.

Our

the utility of the severitraining

in how to make

judgments concerning severity of problems was based on the instruction manual of the AS1 and followed the protocol described in the McLellan et al. (1985) article. Although the McLellan et al. (1985) study described a 4-day training and supervision course and our training was spread out over several weeks, the actual amount and type of training was comparable. Because the interviewers in this study were the treatment staff of the facility rather than researchers, it is possible that the intake counselors in this study were more diverse in theoretical orientation or opinions concerning what constitutes severity than the McLellan et al. (1985) interviewers and that this diversity led to decreased agreement between counselors concerning the severity of problem areas. Furthermore, although counselors were given feedback concerning their ratings, we could not eliminate any counselors from being interviewers if they did not make reliable ratings (as suggested integrate

by McLellan the instrument

Further

evidence

et al., 1992) into regular

because clinical

of the lack of consistency

the purpose work. in severity

of the study ratings

was to

can be seen in

the low to moderate correlations between intake severity ratings and intake composite scores for the same subsections of the ASI. Agreement between severity and composite scores was substantially lower than reported by McLellan et al. (1985). Counselors were instructed to make initial severity ratings after reviewing responses to several items in each section that also formed the basis of the respective composite scores. The high reliability of the composite scores indicates that counselors were able to agree on the specific responses to these items, but the lower correlations between severity and composite scores suggest that counselors differentially weighted these items in arriving at the severity ratings. Thus, these data question the validity of the severity ratings and it seems likely that more specific guidelines for determining severity ratings will be needed if they are to be useful outside of the research context. For instance, current instructions for making severity ratings do not tie specific scale scores (i.e., O-9) to specific numbers of days of intoxication, specific numbers or types of problems experienced, or specific frequencies or types of drugs used. Counselors must rely on their own internal norms when assigning a specific severity score to a client in each area assessed. Grissom tion about liability of information samples of information

and Bragg (199 1) suggested that raters be given normative informathe AS1 scores for various populations in order to increase the reseverity ratings. Recently, McLellan et al. (1992) provided normative on composite scores and severity ratings for several different subpeople who might seek treatment for alcohol or drug problems. If concerning constellations of scores that would indicate relative se-

verity of problems were available and used by interviewers, interviewer severity ratings might become more reliable and, therefore, more clinically useful. Another possibility would be to train counselors to compute the more reliable composite scores and use them, rather than severity ratings, to predict outcome. However, other findings in this study questioned the validity of the composite scores. Significant reductions in composite scores for alcohol, drug, legal, family,

J.S. Wertz, B.L. Cleaveland, and R.S. Stephens

186

anti psychiatric problems suggested that subjects experiencetl fewer problems at follow-up than at intake. However, on examination of‘ the individual items cornprising the alcohol and drug composite scores, it appeared. that the majority of change was clue to subjects’ ratings of‘ the f’requency and severity of problems associated with alcohol and drug use and their own need for treatment rathel than because of‘ significant changes in substance use. It is possible that the ASI composite

scores for alcohol and drug abuse tap more into the negative conseassociated with substance use rather than use itself’. ‘I‘hese data highlight the fact that cornpusite scores are not completely objective measures and consist quences

of‘ subjects’ perceptions of their situations and the relationships between use and tlrug-related problems. By combining more subjective inf’orrnation more

objective

inf’ormation,

it becomes

._

difficult

to identif’y

exactly

drug with

what changes

with treatment. Although previous studies (e.g., McLellan et al., 19%) have reported corresponding changes in substance use rates and composite scores, it is important that f‘uture research not rely solely on the composite scores as indicators of‘ problem severity. Validation of‘ composite scores in relation to collateral reports or biological iritiicators of substarice use would be preferable. Only the alcohol and drug severity ratings at intake anti the number of previous tlrug treatments predicted alcohol and drug composite score outcomes. ‘l‘hose individuals with the least severe problems at intake reported the least sever problems 3 months later. Although the proportion of’ variance in outcomes that was explained was small (16%), the ASI severity ratings outperf’ormecl backgrountl demographic and drug treatment variables. .l‘he alcohol ancf chug abuse severity ratings had some utility for identifying subjects with poorer prognoses despite the problems with reliability anal validity not&l earlier. training that result in higher reliability fill- severity

Irrlpl-o\,ements in counselor ratings are likely to f’urther

increase their predictive power. A similar list of predictor variables accountecl between 38 and 39%’ of the \zarianc-e in a study by R/IcLellan et al. (1986). ‘l-lie findings did not support previous research showing that psychiatric

for

se-

verity predicts outcome (McLellan. Luborsky, et al., 1983; McLellan et al., 19X(i). In tl;is study, the range of‘ psychiatric severity may have been restricted by the exclusion of‘ clients seen for substance use in the mental health center and thus preveutetl f‘intling relaGonships with outcome. I Iowever, others have f’ailecl to replicate the psychiatric severity fintliugs and, instead, have fountl itliosyricratic constellations of‘predictors (Alterman et al., 1990; Kosten et al., 1987). It stwuetl likely that pretlictors of‘outcorue will vary across populations that also \‘ar-y in the degree and type of‘ relatetl problems that affect substance abuse. For instance, the subjects in this study appearecl to have a longer history of’ alcohol ancl mariju& abuse, were more likely to by on parole or probation, were less likel) to be unerriployecl, and hat1 a history of‘ fewci- psychiatric treatments co~ii~~~ii-ec~ to other studies using the ASI (McCMlati et al., 1986). ‘l‘hcse dif‘terenccs tray have inipactetl the ability of. various subsections of the AS1 to cotitribute to the prediction of oci~cotnc‘s. It is important, theref’ore, that tt-eatnnent agencies using the AS1 or other forms of standardized assessment concluct their own research stuthes to understand the preclictors of treatnient-oLitcoliie specific to the popw lations they serve.

Addiction

Severity

Index in Rural Settings

187

There are several limitations to this study. The low level of change from intake to follow-up, for instance, may be due to a lower rate of reporting of substance use at the beginning of treatment. It is possible that subjects minimized their report of substance use to their counselors at the beginning of treatment more than they did to the researchers at the end of treatment. On the other hand, the opposite temporal pattern of bias is more typically predicted and the problem of getting valid self-reports about substance abuse from clients is inherent in the clinical assessment process. The use of biological markers or collateral reports to substantiate self-reported drug and alcohol use both before and after treatment would have helped establish the reliability and validity of both the severity and composite scores. Although these limitations reduce the strength of the conclusions that can be drawn, this study adds to the information on the generalizability and utility of the ASI. The ability of the ASI to predict outcomes when integrated into daily intake procedures is promising, but more work needs to be done. Standardized assessment of all clients will aid program evaluation and development and potentially allow service providers to understand client characteristics predictive of positive outcomes with particular types of treatments. However, the utility of counselor severity ratings is suspect until protocols for arriving at a consensus across diverse counselors and subject populations can be developed. Similarly, our data question the validity of composite scores and the general tendency of the AS1 to rely on subjective evaluations of the extent of problems by clients or counselors.

REFERENCES Alterman, A.I., Kushner, H., & Holahan, J.M. (1990). Cognitive functioning and treatment outcome in alcoholics. ,]ournnl of Nervotr.c and Mrntnl Dimm, 178, 494-499. Babor, T.F., Stephens, R.S., & Marlatt, <;.A. (1987). Verbal report methods in clinical research on alcoholism: Response bias and its minimization. Journnl of‘Situdies on Alcohol, 48, 410-424. DeLeon, P.H., Wakefield, M., Schultz, A.J., Williams, J., & VandenBos, G.R. (1989). Rural America: Unique opportunities for health care delivery and health services research. American P.sychologist, 44, 1298-1306. Fureman, B., Parikh, G., Bragg, A., & McLellan A.T. (1990). Addiction Smerity Index A guide to traznzng and su@mi\ing ASI mtrrviews based on the past ten years (5th ed.). Philadelphia: The University of Pennsylvania/Veterans Administration Center for Studies on Addiction. Grissom, G.R., & Bragg, A. (1991). Addiction Severity Index: Experience in the field. International Journnl of thr Addictions, Z(l), 55-64. Hodgins, D.C., & El-Guebaly, N. (1992). More data on the Addiction Severity Index. journal of Nervous and Mental Diseaw, 180, 197-20 I. Kosten, T.R., Rounsaville, B.J., & Kleber, H.D. (1987). Multidimensionality and prediction of treatment outcome in opioid addicts: 2.5.year follow-up. Comprehensiur Psychiatry, 28(l), 3-13. Mc(;ahan, P., Griffith, J., & McLellan, A.T. (1986). Corn@& scorc.~ from the Addiction Severity Indrx: Manual. Philadelphia: Veterans Administration Press. McLellan, A.T., Jushner, H., Metrger, D., Peters, R., Smith, I., Grissom, G., Pettinati, H., & Argeriou, M. (1992). The fifth edition of the Addiction Severity Index. Journal ofSub&nce Abuw Treat?aent, 9, 199-213. McLellan, A.T., Luborsky, L., Caccila, J., Griffith, B.A., Evans, R., Barr, H.L., & O’Brien, C.P. (1985). New data from the Addiction Severity Index: Reliability and validity in three centers.Journ& nf Nwwus and Mental Disease, 173, 4 12-423.

188

J.S. Wertz, B.L. Cleaveland,

and R.S. Stephens

McLellan, A.‘T., Luborsky, L., O’Brien, C.P., Barr, H.L., &aEvans, K. (IYX6). Alcohol and drug abuse treatment in three different populations: Is tbel-e itnprovement and is it predictable? Am~ritrc,r Juuml of l)tq