Methodological issues in the evaluation of treatment of drug dependence

Adv. Behw. Res. Ther. Vol. Il. pp. 13%150. 1989. Printed in Great Britain. AU rights reserved.

0

014&64fl2/89 %u.o0+.5u 1989 Pergamon Press plc.

METHODOLOGICAL ISSUES IN THE EVALUATION OF TREATMENT OF DRUG DEPENDENCE Garth W. Martin* *Addiction

and D. Adrian Wilkinson?

Research Foundation, 33 Russell Street, Toronto, Ontario. M5S 2Sl. Canada tYork University, Ontario, Canada

Abstract - Studies of treatment outcome for drug dependence are methodologicaffy inferior to those for alcohol dependence, probably because scientific interest in drug use is more recent. Also, the behavior of drug dependence is very complex, since multiple drug use is normative for psychoactive drugs. Compared to clients in alcohol and tobacco studies, those in studies of drug dependence are more heterogeneous in their drug use. It is recommended that research priority be given to the identification of homogeneous subsets of the drug user population, including users of alcohol and tobacco. Several methods of measuring drug use have been advocated, but all have limitations, and little is known about their impact on the interpretation of treatment outcome. Recently the quality of designs in this area of research has improved, but avoidance of randomized trials continues to retard the acquisition of knowledge about the relative effectiveness of treatment procedures. Variability in the duration of follow-ups also impedes interpretation of findings; a minimum of six months post treatment is recommended. It is concluded that measures of drug use should be the principal outcome in evaluations of treatment effectiveness, that the role of therapist variables deserves research attention, and that criteria be developed for the interpretation of nonabstinent outcomes, which are the norm of clients treated for drug dependence.

INTRODUCTION In North America, the problem of drug use among youth emerged as a social issue in the late 1960s and quickly burgeoned (Beschner & Friedman, 1979). A rapid proliferation of treatment interventions followed, and in this haste scientific evaluation was frequently compromised (Lavenhar, 1979). Since that time, there has been increasing interest in the measurement and comparison of treatment outcomes. Progress has been substantial (Jaffe, 1984), but methodological weaknesses have characterized many of the studies which are, in general, regarded as methodologically inferior to studies of alcohol abuse (Goldstein, Surber, & Wilner, 1984). Furthermore, methodological issues have emerged from research advances identifying complexities in the definition and measurement of the problem behaviors. This paper will examine current methodological issues in the evaluation of treatments of drug abuse. Its principal foci will be on issues related to the heterogeneity of the drug user population, limitations in the design of outcome studies, problems in the measurement of the target behaviors, and inconsistencies in the conceptualization of outcome measures. JAERT 11:3-o

134

G.W. Martin and D.A. Wilkinson

DEFINING

THE DRUG

ABUSE POPULATION

Problems of definition hamper attempts to identify and describe the “drug abuser” population. The clients in studies of the treatment of drug dependence are much more heterogeneous in regard to their substance use than those involved in studies of alcohol or tobacco dependence. This leads to difficulties in the specification of the target behavior to be treated and the comparability of clients among studies of treatment outcome. The normative pattern of use of psychoactive substances is multiple drug use (Farley, Santo, & Speck, 1979; Simpson & Sells, 1974), even among heroin addicts (Craddock, Bray, & Hubbard, 1985; Shaffer, Nurco, Ball, & Kinlock, 1986). Increasingly, clients entering treatment exhibit patterns of use of a wide variety of psychoactive substances (Hubbard, Rachel, Craddock, & Cavanaugh, 1984), and these patterns frequently include use of alcohol and tobacco (Wilkinson, Leigh, Cordingley, Martin, & Lei, 1987; Kozlowski et al., 1989). This trend has highlighted the need for development of methods for classification of multiple drug users that differ from the conventional practice of categorization on the basis of “primary” drug of abuse (Hubbard et al., 1986). Two general approaches to classification have been attempted: one based on patterns of drug use, and the other emphasizing nondrug variables of potential clinical relevance. Typologies of Drug Users Drug users can be classified on the basis of conceptually derived or empirically specified patterns of drug use. The empirical approach (e.g., Wilkinson et al., 1987) involves the identification of typologies of multiple drug users by means of multivariate analysis of measures of frequency of drug use. This classification procedure has the advantage of reflecting the full extent of drug use reported by the client. However, the approach is purely descriptive, and tends to result in a number of drug use measures, thus necessitating larger samples or more extensive exclusion criteria in the evaluation of treatment outcome. The utility of this procedure depends upon the identification of typologies that: (1) reflect clinically discrete subsets of the drug abuser population, (2) can be consistently replicated, and (3) have predictive validity. Only three studies have applied this method to clinical samples of drug users (Braught, Kirby, &Berry, 1978; Hubbard et al., 1986; Wilkinson et al., 1987). In each, clinically interpretable typologies were identified and various psychosocial measures were found to be differentially related to them. There were also similarities in the typologies identified among the studies despite differences in the methods of cluster analysis used and in the clinical populations represented (heroin addicts were underrepresented in the Wilkinson et al. [1987] study).

Drug Dependence

135

Analytical procedures for the empirical derivation of typologies are in their infancy but appear to hold considerable promise in attempts to identify more homogeneous subsets of the drug users (Skinner, 1980). It is noteworthy that in the one study in which it was feasible to test the predictive validity of the typologies (Hubbard et al., 1986) the three empirically derived typologies were ignored in favor of a system of seven conceptually developed classifications for outcome evaluation purposes. Thus, the predictive validity of these typologies remains unreported. Classification by Nondrug

Variables

The second approach to classification involves the categorization of drug users on nondrug variables. This method is compatible with the employment of drug use categories, but typically it has substituted for, rather than complemented, such efforts. Two current practices exemplify this approach: one based on untested assumptions, one empirically derived. The former refers to the growing emphasis on “adolescent” drug use. This age-based classification procedure appropriately draws attention to the clinical importance of developmental factors in the treatment of teenage drug users, but in so doing typically obscures the heterogeneity of their drug use. Age differences among typologies of drug users have been observed, but they do not reflect an adolescent cluster (Martin, Wilkinson, Cordingley, & Leigh, 1987; Wilkinson et al., 1987). Indeed, age-related differences in treatment outcome may be as great among adolescents (Harrison & Hoffman, 1987) as they are in comparisons of adolescents with adult drug users (Holland & Griffen, 1984; Deleon, 1985). A more promising approach to classification on the basis of nondrug variables is exemplified in the work of McLellan and colleagues (1983, 1986) and Rounsaville and colleagues (1982a, 1986) in which drug users have been classified on the basis of coincident psychopathology. Two distinct procedures have been employed. In .one, a global measure of Psychiatric Severity incorporated within the Addiction Severity Index (McLellan, Luborsky, Woody, & O’Brien, 1980a) was found to predict treatment outcome. Stratification of the client group on the basis of their Psychiatric Severity rating revealed the most pronounced client-programme matching effects were for subjects in the “mid-severity” range (McLellan, Luborsky, Woody, O’Brien, & Druley, 1983; McLellan, Luborsky, & O’Brien, 1986). A high level of interrater reliability has been established for these severity ratings (McLellan, Luborsky, Woody, & O’Brien, 1980). The second procedure has involved use of DSM-III diagnostic criteria to assess psychopathology (Rounsaville, Weissman, Kleber, & Wilber, 1982).

136

G.W. Martin and D.A. Wilkinson

In these studies, the presence of diagnosed psychiatric disorder was predictive of poorer treatment outcome (Rounsaville, Tierney, Crits-Christoph, Weissman, & Kleber, 1982; Rounsaville, Kosten, Weissman, & Kleber 1986), but successful client-programme matching did not appear to be associated with particular diagnostic categories (Rounsaville & Kleber, 1986).

MEASUREMENT

OF DRUG

USE

It has been said that the nature of drug dependence is like the unicorn, often described but seldom captured (Johns, 1988). Efforts to arrive at consensus on procedures for measuring drug use have eluded researchers because of the enormous complexity of the behavior, and its changing character (Craddock et al., 1985). The traditional emphasis on the primary substance of abuse is no longer sufficient in the light of growing evidence of patterns of multiple drug use among clients presenting for treatment (Wilkinson et al., 1987). Complexities of Multiple

Drug Use

Several issues complicate the quantification of multiple drug use. There is no standard method of classifying all of the variety of abused substances because they vary independently on a number of dimensions, such as pharmacological properties, social acceptability, typical patterns of use and availability. These dimensions tend to be confounded in systems of classification. For example, if codeine and heroin are both classified as opiates, use of the two drugs is undifferentiated and important information related to pattern and severity of drug use may be lost. Within pharmacologically defined drug classes it is virtually impossible to avoid the common classification of drugs of unequal potency or risk. However, adding drug classes in the interest of greater descriptive specificity must be done parsimoniously because, ultimately, they increase the overall complexity of the data set by increasing the number of drug use dependent measures. Since many of the substances are obtained illicitly, their concentration is unstandardized. Users are typically unaware of the precise dose involved in any drug administration and occasionally are misinformed about the identify of the particular drug. Furthermore, the actual dose delivered varies with the route of administration. For example, cocaine can be used intranasally, intravenously, or by smoking and the risks and effects associated with each mode of administration differ importantly. Compounding these uncertainties of quality is the temporal pattern of drug use which may involve consecutive or concurrent use of various substances. Drugs may be combined to potentiate a particular effect, to eliminate an undesired side effect or simply because they are available.

Drug

Dependence

137

To avoid the inaccuracies of estimating dose, illicit drug use is usually measured as frequency of use rather than in quantity-frequency terms, but quantification difficulties persist. Problems arise in the specification of the unit of measurement. For marijuana use, is the appropriate unit of administration the “joint” or the “puff”? If an individual snorts a line of cocaine in each nostril is this one or two administrations? Furthermore, clients define use of a drug as a problem at frequencies of use that vary enormously with the particular substances involved (Wilkinson et al., 1987). For example, the frequency of drug use that is associated with problems averages in the tens per annum for LSD, whereas it is in thousands of uses for cannabis or alcohol. This variability should be an important consideration in decisions about combining drugs within single categories.

Reliability

and Validity

Despite its complex nature multiple drug use can be validly and reliably measured. The range of methods that have been used are similar to those employed in the fields of alcohol and tobacco use and include direct observation (Mello, 1977), biochemical assays (Martin, Wilkinson, & Kapur, 1988), self-reports (Wilkinson et al., 1987), reports of collateral informants (Cdrdingley, Wilkinson, & Martin, 1989) and actuarial data (Sells & Simpson, 1980). In evaluating treatment outcome, self-report data are virtually indispensible because of the time intervals over which behavior is to be assessed and the relative unintrusiveness of the method. Furthermore, mounting empirical evidence indicates that verbal reports of drug users can have satisfactory reliability and validity (Maisto & Cooper, 1980; Rounsaville, Kleber, Wilber, Rosenberger, & Rosenberger, 1981; Aiken, 1986; Martin et al., 1988). Indeed, in studies where clients’ self-report information has been contrasted with more objective sources of the data, observed discrepancies were more likely to involve uncorroborated client reports of drug use than the reverse (Martin et al., 1988; Cordingley et al., 1989). However, the reliability of self-report data is subject to situational influences (Skinner, 1984) and cannot be presumed. Aiken (1986) found a tendency among clients to “fake positive” in order to make a favorable impression on treatment staff when applying for treatment. Mensch and Kandel (1988) observed that familiarity with the interviewer depressed drug use reporting in a national survey of drug use among youth. These data underscore the importance of carefully structuring data gathering procedures to minimize potential compromises to the validity of self-report.

138

G.W. Martin and D.A. Wilkinson

Measurement

Procedures

A wide variety of procedures have been adopted to address the complexities of multiple drug use behavior. These methods have been thoroughly reviewed by Wells, Hawkins, and Catalan0 (1988) who identified five distinct approaches: (1) measures of consumption, (2) categorical classifications, (3) weighted indices, (4) composite indices, and (5) patterns of use. A sixth approach which should also be noted derives from the concept of a drug dependence syndrome, based on core symptoms of drug dependence, as distinct from the adverse consequences associated with drug use. Evidence of an opiate dependence syndrome exists (Sutherland, Edwards, Taylor, Phillips, & Gossop, 1988; Skinner & Goldberg, 1986) but its validity remains in question (Hasin, Grant, Harford, & Endicott, 1988). The first three of the above methods represent different approaches to the quantification of drug use behavior. Measures of consumption typically involve estimates of the frequency with which use of particular drugs or drug classes occurs, although in rare instances they may also include estimates of the quantity of a drug consumed, derived from drug costs or other dose-related information (e.g., Sutherland et al., 1986; Wilkinson et al., 1987). These methods enable the description of a complete profile of drug use, but can sacrifice sensitivity with respect to the severity of drug use by ignoring such factors as dose, combined drug use and route of administration. They also leave dangling the issue of how to combine the multiple estimates of frequency into a manageable number of dependent measures. Categorical classifications and weighted indices are two procedures which attempt to reduce the frequency estimates to a single drug use measure. The former, exemplified in TOPS (Hubbard et al., 1984), involves the development of a hierarchical scale of drug involvement ranging from abstinence to frequent use of the most serious drug type (e.g., daily heroin use). This procedure creates a single ordinal scale of drug involvement, but does so at the expense of sensitivity in the measurement of frequency, and necessitates arbitrary judgements about the seriousness of types of drug use. Thus, a client who reduces marijuana consumption from 20 “joints” per day to 1 use per day may appear unchanged on the drug involvement scale, whereas a heroin addict who becomes an alcoholic may appear to be dramatically improved. Weighted indices combine all drug use onto a single scale by assigning weights to the types of drugs and then summing weighted frequencies across drug types. The weights may be conceptually derived to reflect the significance of use of the various drug types, or empirically determined based on prevalence of use (Clayton & Voss, 1981). There have been

Drug Dependence

139

several attempts at constructing such indices (for reviews see Clayton & Voss, 1981; Douglass & Khavari, 1982). However, in spite of the advantage of parsimony, reservations about weighted indices persist: logical inconsistencies have emerged in their application (Douglass & Khavari, 1982); the assumption of unidimensionality of drug use may be invalid (Wilkinson et al., 1987); and the collapsing of all drug use to a single index may mask important clinical differences associated with particular patterns of drug use (Hubbard et al., 1986). An alternative approach to the measurement of drug use involves the combination of a variety of features of the drug use behavior, and its related problems, to form a composite index. This approach provides the latitude to be more comprehensive in the range of factors considered and enables their expression in the form of a single dependent measure. However, it must be established empirically that the items included in the index are sufficiently intercorrelated to legitimize their collapse to a unidimensional scale. The best example of this method is the composite index of drug use of the ASI, which includes measures of frequency of use, drug-related problems, and client severity ratings and has the added virtue of established reliability and validity (McLellan, Luborsky, Woody, & O’Brien 1980). It has also been used in several studies of treatment outcome, enhancing one’s ability to make between-study comparisons. Its limitations include a narrow window of drug consumption data (30 days), and comparative insensitivity to changes in the pattern of drug consumption. Evidence for the prominence of changes in pattern of drug use during the recovery process (Simpson & Marsh, 1986) has given impetus to an emphasis on temporal patterns in the analysis of treatment outcome (Wells et al., 1988). There is growing awareness of the importance of attending to patterns of multiple drug use among clients presenting for treatment, but there is no consensus concerning the preferred method of measuring multiple drug use behavior. Each method has imperfections involving some sacrifice of sensitivity or comprehensiveness. There is some evidence for interrelatedness of the different measurement approaches (Craddock et al., 1985), but there are not studies of how, or whether, interpretation of treatment outcomes is differentially affected by the variety of approaches. Clearly, the comparability of studies is restricted by procedural variation. DESIGN Evaluating

ISSUES

Treatment Eficacy

Most early studies of treatment of drug abuse involved cases of severe opioid dependence (Rush, 1979) and the outcomes were “sin-

140

G.W. Martin and D.A. Wilkinson

gularly unimpressive” if not “disastrous” (Callahan, 1980). Thus, it is understandable that, faced with a rapidly proliferating drug problem, the initial impetus to outcome studies was to evaluate whether any positive effects were associated with treatment. Several large studies ensued (e.g., Sells & Simpson, 1980; DeLeon, Wexler, & Jainchill, 1982; Hubbard et al., 1984; McLellan et al., 1986) and the argument that “treatment works” was advanced on the strength of three features of the results: (1) at follow-up, consistent positive outcomes were observed on measures of drug use and social functioning; (2) the outcomes for clients who entered treatment following detoxification were superior to those of clients who rejected or failed to begin treatment; and (3) outcomes improved as a function of treatment retention. Encouraging though these results may seem, they fail to rule out the alternative hypothesis of a self-selection effect. None of the studies employed random assignment. Thus, “minimal treatment” comparison groups were self-selected by clients who may have been less ready for behavior change for various reasons. Clients who remained longer in treatment may have differed from those who stayed less time on expectancies, motivation or other unassessed characteristics. Lack of Controlled

Research

Too much of what is known about substance abuse treatment derives from uncontrolled research. Replication and statistical control add weight to the evidence but cannot substitute for the rigor of experimental design (Hall, 1984). Arguments against random assignment in evaluating treatment for drug abuse (McLellan, O’Brien, Kron, Alterman, & Druley, 1980; Sells & Simpson, 1980) have been too readily accepted. Opposition to controlled studies appears to reflect practical constraints of the service delivery system and the resistance of clinical staff and administrators (Bale et al., 1980). Ironically, the evidence from uncontrolled studies offers little suggestion that clients would receive short shrift by random assignment to the treatment approaches in common use (Simpson & Savage, 1981/82; Martin, 1987). In the past decade some excellent research has addressed the efficacy of highly specified treatments including; psychotherapy (Rounsaville, Glazer, Wilber, Weissman, & Kleber, 1983; Woody et al., 1983), contingency management (Stitzer, Bigelow, & Liebson, 1980; Dolan, Black, Penk, Robinowicz, & DeFord, 1986; Stitzer, Bickel, Bigelow, & Liebson, 1986), family therapy (Stanton, Todd, Steir, VanDeusen, & Cook, 1982), and relaxation therapy (Khatami, Mintz, & O’Brien, 1977). These controlled studies represent a substantial improvement in research design over earlier outcome evaluations. Replications, and resolution of

Drug Dependence

141

substantive issues raised by these studies, are necessary before firm conclusions about the efficacy of the treatments can be reached, but the “giant leap” of methodology that they represent should be emulated in future research. Although randomized controlled trials are more readily conducted in the context of adjunctive methadone therapy, the generalizability of such studies is very restricted. The majority of drug dependent clients are not opiate dependent, and even among those who are, adjunctive treatment with methadone may not be the optimum strategy. Furthermore, the restriction of the best controlled clinical studies to methadone programmes could serve to perpetuate a preoccupation with heroin use to the exclusion of other forms of drug dependence (Tims & Holland, 1984), and thus reinforce a tendency to conceptualize services in terms of their relevance to opiate users. This tendency has been realized in the creation of a treatment category (Drug Free Outpatient Programs[DF]) which is defined as a treatment modality by what it excludes (residential or pharmacological interventions) rather than by unifying theoretical concepts (Kleber & Slobetz, 1979). Treatment Specificity and Therapeutic Process

The practice of grouping conceptually distinct treatments on the basis of “surface” similarities runs directly counter to the necessity of achieving procedural specificity in soundly evaluating treatment differences. It can also serve to perpetuate the misconception that the context in which a treatment is offered (e.g., outpatient vs. inpatient) is of greater significance than the treatment model employed, and therapist effects related to its delivery. An example can illustrate the potential pitfalls of simplistic categorization of treatments. Two broad-spectrum residential behavioral treatment programmes were contrasted with a brief outpatient behavioral self-control training. The content of the two residential programmes was identical, but the milieu of the unit was controlled using a credit system based on either group- or individually-contingent reinforcement. There was no difference in overall effectiveness of inpatient versus outpatient treatment at 12 and 24 months (Wilkinson, Martin, Cordingley, & Leigh, 1989). However, there was a significantly higher rate of successful outcome in the residential treatment involving group contingent reinforcement than in either the identical residential programme with individual reinforcement or the outpatient treatment. Thus, a highly specific attribute of the residential treatments was associated with differential effectiveness, whereas the more prominent features of overall treatment intensity and whether inpatient or outpatient were not. Greater specificity of the treatments involved in comparison studies is

142

G.W. Martin and D.A. Wilkinson

clearly indicated, including the use of written manuals, and procedures for ensuring compliance to the treatment protocol (Hall, 1984). In addition, it is probable that therapist variables deserve increased attention. In treatment for alcohol problems therapist empathy appears important (Miller, Taylor, & West, 1980), but in the drug dependence literature studies have focussed upon whether credentials (professional vs. paraprofessional) and experience with addictions (ex-addict or not) differentially affect outcome (see reviews by Allison & Hubbard, 1985; Hall, 1983). In general, no differences have been observed, but the studies have lacked methodological rigor (Hall, 1983). Nurco et al. (1988) studied the relation to treatment outcome of client/counsellor congruence on perception of the presenting problems and the manner with which they should be dealt. They observed only a slight relationship, but the study exemplifies the possibility of methodologically sound investigations of variables for which there are conceptual reasons for expecting effects. In view of the frequency with which the technique is used and advocated in the treatment of drug dependence, the level of therapist confrontation of clients is clearly a variable that cries out for evaluation as a predictor of treatment outcome and dropout. Confrontation techniques have been the hallmark of therapeutic communities based upon the Synanon model (Yablonsky, 1965), and are frequently advocated as a method to break down the “denial” that is alleged to characterize drug abusers and alcoholics. Despite evidence suggesting that certain types of clients may benefit differentially from programmes stressing this technique (Waal, 1985), and findings that some clients are probably harmed by such programmes (McClellan, 1986), the impact of this important therapist behavior remains largely unstudied. Follow-up

Procedures

The explication of follow-up procedures for the evaluation of treatments of drug abuse should address four important issues: (1) who should be included in follow-up samples; (2) the duration of the interval over which follow-up is to occur; (3) the point from which the follow-up interval is measured; and (4) the extent to which clients are “at risk” to use drugs during the follow-up interval. The evaluation of any treatment of drug dependence should consider its attractiveness to, as well as its effectiveness for, the population for whom it is intended (Bale et al., 1980). While a simple numerical accounting of the rates at which assigned clients begin and complete treatment may suffice to make inferences about the attractiveness of treatment, the issue becomes more complex when controlled comparisons are involved, especially when the attractiveness of the treatments is expected to be unequal. In such instances, a strong argument can be made for the follow-up of all assigned

Drug Dependence

143

clients so that a comparison of the treatments can be made based on all of the randomly assigned subjects. Any other inclusion criterion suffers from the shortcoming of introducing bias associated with attrition of subjects. Furthermore, the post hoc evaluation of any other inclusion criterion (e.g., treatment completion) can be accommodated within this design. There is no clear consensus about the optimum duration of follow-up for the evaluation of differential treatment outcome, though there seems to be de facto agreement that at least six months is desirable. In part, the lack of consensus may reflect the fact that two issues are conflated. On the one hand investigators wish to maximize the sensitivity of designs, which argues for short follow-up intervals (Sanchez-Craig & Wilkinson, in press), and on the other hand there is a very proper concern to assess the stability of outcome in what is viewed as a chronically relapsing condition. In young drug dependent clients, there is some evidence that treatment outcome may improve over time (Simpson & Marsh, 1986; Wilkinson et al., 1989), but this improvement may be more related to subsequent treatment or maturation effects than to a delayed impact of treatment. The issue awaits clarification by more intensive study of the recovery process (Wells et al., 1988). In the meantime, the use of multiple outcome evaluations, some quite shortly after treatment (e.g., 3 months), and distributed over two or more years, appears to be a prudent research strategy. For most studies the follow-up interval is measured from the date of treatment assignment (e.g., Bale et al., 1980; McLellan et al., 1986; Rounsaville et al., 1983) but several studies have defined the interval from the point of treatment termination, including the TOPS and DARP studies which combined within treatment measures and posttreatment follow-up (Sells & Simpson, 1980; Hubbard et al., 1984; Harrison & Hoffman, 1987). The preferred approach is the one based on treatment entry. This ensures that the outcome of treatment is evaluated over an interval that is common to all subjects, but introduces a possible confound of “time at risk” (Craddock et al., 1985) when one or more of the treatments involves a structured residential component. The concept of “time at risk” involves the assumption that time in residential treatment, in hospital, or in prison is associated with vastly reduced opportunity for drug use, and as such should be factored into evaluations of outcome (Craddock et al., 1985). In evaluation of residential treatment, follow-up should proceed well beyond its completion to ensure adequate time at risk for valid evaluation of the effect of treatment. CONCEPTUALIZATION

OF TREATMENT

OUTCOME

In addition to problems in the measurement of the target behavior (drug use), there is a lack of consensus concerning other procedures

144

G.W. Martin and D.A. Wilkinson

for the evaluation of treatment outcome. At issue is the question of what dimensions of life functioning should be assessed, and how the dependent measures should be combined in the determination of overall outcome. Furthermore, there has been a lack of uniformity in the criteria used to rate successful outcome, making valid comparisons among the studies difficult. There is general consensus that outcome evaluation should address more than drug use, but there is little agreement about how this is best achieved. Three general approaches to the assessment of outcome are used. In the first, drug use is the principal criterion measure. Other dimensions of life functioning, such as employment and criminality, may also be evaluated but these are considered in the context of convergent validation of the change in drug use (Sobell & Sobell, 1980; Wilkinson & Martin, 1984, August). A second approach involves developing a composite index of outcome, comprised of drug use and other criterion measures. This approach is characterized by the summation of ratings of client progress in various domains of functioning to yield a total outcome score (Sells & Simpson, 1980). Opponents of this approach argue that outcome is multidimensional and the various outcome measures are poorly correlated; therefore they should be assessed independently (McLellan, Luborsky, Woody, & O’Brien, 1980; Rounsaville, Tierney, CritsChristoph, Weissman, & Kleber, 1982). The case for multidimensionality of treatment outcome derives from two principal sources. McLellan, Luborsky, Woody, and O’Brien (1980) have observed low intercorreiation among the seven severity indices of the AS1 (medical, employment, family, alcohol, drug, legal, psychological), and have concluded that they represent separate dimensions of functioning. Similarly, in principle components analysis of a wide variety of dependent measures, separate, and poorly interrelated, factors have been observed (Rounsaville , Tierne y , Crits-Christoph, Weissman, & Kleber, 1982). These findings raise serious questions concerning the legitimacy of unvalidated composite indices, particularly those which cross domains of functioning. Interestingly, there is evidence that the achievement of successfuZ drug use outcome is associated with significant improvement in various other areas of functioning (Wilkinson & Martin, 1984, August; McLellan et al., 1986) and that improvements reverse if relapse occurs (Rounsaville, Kosten, & Kleber, 1987). This tendency provides some support for a convergent validation approach, based on drug use as the principal criterion measure. Criteria for Successful Outcome

The criteria for ratings of successful outcome vary substantially

among

Drug

Dependence

145

studies. In some, the stated criterion is abstinence from all nonprescribed psychoactive drugs (Harrison & Hoffman, 1987); in others, the most favorable outcome category can involve drastic reduction in use of some illicit drugs, but consumption of up to 8 oz of 80 proof alcohol per day and unlimited marijuana use (Sells & Simpson, 1980). Abstinence is not a static state and its definition may permit “lapses” of periodic low frequency use (Rounsaville et al., 1987). Nonabstinent outcomes are common (Wilkinson & Martin, 1983) necessitating difficult decisions concerning “permissable” levels of drug use within the context of a successful outcome rating, despite the absence of criteria for “safe” or “socially acceptable” levels of use. The adoption of uniform standards that could be empirically validated would greatly enhance the overall quality and interpretability of research. CONCLUSIONS Heterogeneity of the drug user population is a major barrier to the accumulation of research evidence concerning the effectiveness of available treatment methods. It is an irrationality of the addictions field that the use of alcohol and tobacco are treated as sufficiently distinct to justify independent clinical research activity, but the use of all other substances tends to be lumped together in a single catch-all category. The identification of homogeneous subsets of the drug user population, including users of alcohol and tobacco, should be a research priority. The quality of behavioral research has improved dramatically in the past decade, with the emergence of several controlled treatment comparisons, but continued resistance to randomization is evident from the predominance of studies with treatments offered as adjuncts to methadone maintenance. Studies of treatment efficacy point to few circumstances in which randomization and the best interests of the client would be in conflict (Johns, 1988; McLellan, O’Brien, Kron, Alterman, & Druley, 1980; Rounsaville & Kleber, 1986; Simpson & Savage, 1981/82), but the research community has failed to endorse this view. There must be a greater effort to win the confidence and cooperation of clinical staff and administrators in the conduct of controlled comparisons. There are two important ways in which treatment outcome studies can be improved. Firstly, the concept of a drug-free outpatient program “modality” should be abandoned in favor of greater differentiation and specificity of the myriad of treatments subsumed by this label. Similarly, contrasts such as outpatient vs. residential treatment should yield to descriptions of treatment that reflect clearly specified concepts, principles and practice. The efficacy of these treatments should be evaluated in controlled comparison studies. Secondly, greater attention should be paid to the therapeutic process: to the alliance that is formed between client and therapist (Rounsaville et al.,

G.W. Martin and D.A. Wilkinson

146

1987), and to the impact of particular therapist behaviors. An important example of the latter is therapist confrontation, which is widely used in treatment of drug abuse but unstudied. It is clear that there is no consensus concerning the preferred method of measuring multiple drug use behavior. Each method of measurement involves compromises of accuracy, specificity, inclusiveness or parsimony. Little is known about how the characterization of outcome may be affected by the various approaches to measuring drug use, but idiosyncratic procedures continue to characterize most studies. It is time that funding agencies establish guidelines and standards for measurement, so as to achieve greater comparability among the projects they support. In evaluating treatment outcome, problems of measurement are compounded by conceptual problems. The use of composite outcome measures may not be justified because of a lack of interrelationship among the combined ratings. Drug use should be the principle criterion measure by which treatment effectiveness is judged, but difficulties in the measurement of drug use are compounded by the lack of criteria for interpreting reports of low frequency drug use, and the political and ideological contentiousness of even countenancing such an outcome as successful. REFERENCES Aiken,

L.S. (1986). Retrospective self-reports by clients differ from original reports: Implications for the evaluation of drug treatment programs. The hternational Journal

Allison,

of the Addictions,

21, 767-788.

M., & Hubbard, R.L. (1985). Drug abuse treatment process: A review of the literature. The International Journal of The Addictions, 14, 503-511. Bale, R.N., VanStone, W.W., Kaldau, J.M., Engelsing, T.M., Elasoff, R.M., & Zarcone, V.P. (1980). Therapeutic communities vs. methadone maintenance. Archives of General Psychiatry, 37, 179-190. Beschner, G.M., & Friedman, AS. (1979). The drug problem: Background in the sixties and seventies. In G.M. Beschner & AS. Friedman (eds.), Youth drug abuse. Lexington, MA: Lexington Books. Braught, G.N., Kirby, M.W., & Berry, G.J. (1978). Psychosocial correlates of empirical types of multiple drug abusers. Journal of Consulting and Clinical Psychology, 46, 1463-147.5. Callahan, E.J. (1980). Alternative strategies in the treatment of narcotic addiction: A review. In W.R. Miller (Ed.), The addictive behaviors. New York: Pergamon. Clayton, R.R., & Voss, H.L. (1981). Young men and drugs in Munhatten: A cau.wZ analysis, NIDA Research Monograph No. 39. Rockville, MD: National Institute on Drug Abuse. Cordingley, J., Wilkinson, D.A., & Martin, G.W. (1989). Corroboration by collaterals of posttreatment self-reports from multiple drug users. Manuscript submitted for publication. Craddock, S.G., Bray, R.M., & Hubbard, R.L. (1985). Drug use before and during drug abuse treatment: 1979-1981 TOPS admission cohorts, Treatment Research Monograph Series. Rockville MD: National Institute on Drug Abuse. DeLeon. G. (1985). TC research: State of the art. In A. Acamoora. and E. Nebelkoof (Eds.),‘Bridging services: Drug abuse, human services and the therapeutic communi&,

Drug Dependence

147

Proceedings of the World Conference of Therapeutic Communities, Sept. 16, San Francisco, CA, Waldon House Inc. DeLeon, G., Wexler, H.K., & Jainchill, N. (1982). The therapeutic community: Success and improvement rates 5 years after treatment. The Internntional Journal of the Addictions,

Dolan,

17, 703-747.

M.P., Black, J.L.. Penk, W.E., Robinowicz, R., & DeFord, H.A. Predicting the outcome of contingency contracting for drug abuse. Therapy,

(1986). Behavior

17, 47&474.

Douglass, F.M., & Khavari, K.A. (1982). A major limitation of the percentile index of overall drug use indulgence. The International Journal of the Addictions, 17, 283-294. Farley, E.C., Santo, Y., & Speck, D.W. (1979). Multiple drug-abuse patterns of youth in treatment. In G.M. Beschner & A.S. Friedman (Eds.), Youth drug abuse. Lexington, MA: Lexington Books. Goldstein, M.S., Surber, M., & Wilner, D.M. (1984). Outcome evaluations in substance abuse: A comparison of alcoholism, drug abuse, and other mental health interventions. The International

Journal

of the Addictions,

19, 479-502.

Hall, S.M. (1983). Methadone treatment: A review of the research findings. In J.R. Cooper. F. Altman, B.S. Frown, & D. Szechowicz (Eds.), Research on the treatment of narcotic addiction: State of the art, NIDA Treatment Research Monograph - . Series. Rockville, MD: National Institute of Drug Abuse. Hall. S.M. 119841. Clinical trials in drug treatment: Methodology. In F.M. Tims & J.P. Ludford (Eds.), Drug abuse &eatrnent evaluation: Strategies, progress, and prospects, NIDA Research Monograph No. 51. Rockville, MD: National Institute of Drug Abuse. Harrison. P.A., & Hoffman, N.G. (1987). Adolescent residential treatment intake and follow-up findings. St. Paul, MN: CATOR, the Chemical Abuse/Addiction Treatment Outcome Registry. Hasin, D.S., Grant, B.F., Harford, T.C., & Endicott, J. (1988). The drug dependence syndrome and related disabilities. British Journal of Addictions, 83, 4.5-55. Holland, S., & Griffin, A. (1984). Adolescent and adult drug treatment clients: Patterns and consequences of use. Journal of Psychoactive Drugs, 16,79-89. Hubbard, R.L., Brat, R.M., Craddock, S.G., Cavanaugh, E.R., Schlenger, W.E., & Rachel, J.V. (1986). Issue in the assessment of multiple drug use among drug treatment clients. In M.C. Braude & H.M. Ginzberg (Eds.), Strategies for research on the interactions of drugs of abuse, NIDA Research Monograph No. 68. Rockville, MD: National Institute on Drug Abuse. Hubbard, R.L., Rachel, J.V., Craddock, S.G., & Cavanaugh, E.R. (1984). Treatment outcome prospective study (TOPS): Client characteristics and behaviors before, during. and after treatment. In F.M. Tims and J.P. Ludford (Eds.), Drug abuse treatment evaluation: Strategies, progress and prospects, NIDA Research Monograph No. 51. Rockville, MD: National Institute of Drug Abuse. Jaffe, J.H. (1984). Evaluating drug abuse treatment: A comment on the state of the art. In F.M. Tims & J.P. Ludford (Eds.), Drug abuse treatment evaluation: Strategies, progress, and prospects, NIDA Research Monograph No. 51. Rockville, MD: National Institute of Drug Abuse. Johns, A. (1988). Editorial: The alliance between research and clinical practice in drug dependence - wedlock or deadlock? British Journal of Addiction, 83, 725-727. Khatami. M., Mintz, J., & O’Brien, C.P. (1977). Biofeedback-mediated relaxation in narcotic addicts. Behavior Therapy. 8, 968-969. Kleber, H.D., & Slobetz, F. (1979). Out-patient drug free treatment. In R.I. DuPont, A. Goldstein, & J. O’Donnell (Eds.), Handbook on drug abuse, Washington, D.C.: National Institute on Drug Abuse, U.S. Government Printing Office. Kosten, T.R., Rounsaville, B.J., & Kleber, H.D. (1987). Multidimensionality and the prediction of treatment outcome in opioid addicts: 2.5 -year follow-up. Comprehensive Psychiatry, 28,s13. Kozlowski, L.T., Wilkinson, D.A., Skinner, W., Kent, C., Franklin, T., & Pope, M.

G.W. Martin and D.A. Wilkinson

148

(1989). Comparing tobacco cigarette dependence with other drug dependencies: Greater or equal ‘difficulty quitting’ and ‘urges to use’ but less ‘pleasure’ from cigarettes. Journal of the American Medical Association, 261, 898901. Lavenhar, M.A. (1979). Methodology in youth drug-abuse research. In G.M. Beschner & A.S. Friedman (Eds.). Youth drug abuse. Lexinaton. MA: Lexineton Books. Maisto, S.A., & Cooper, A.M. (1980). A historical perspective on aiohol and drug treatment outcome research. In L.C. Sobell, M.B. Sobell, & E. Ward (Eds.), Evaluating alcohol and drug abuse treatment effectiveness. New York: Pergamon. Martin, G.W. (1987). Effectiveness of treatmentforyoung drug users: A review. Unpublished manuscript. Addiction Research Foundation, Toronto. Martin, G. W., Wilkinson, D.A., Cordingley, J., & Leigh. G.M. (1987, August). Variations in patterns of drug use among adolescent and young adult multiple drug for early identification. Paper presented at the 4th International

users:

Implications

Conference on the

Treatment of Addictive Behavior, OS (Bergen), Norway. G.W., Wilkinson, D.A., & Kapur, B.M. (1988). Validation of self-reported cannabis use by urine analysis. Addictive Behaviors, 13, 147-150. McLellan, A.T. (1986). “Psychiatric severity” as a predictor of outcome from substance abuse treatment. In R.E. Meyer (Ed.), Psychopathology and the addictive disorders. New York: The Guilford Press. McLellan, A.T., Luborsky, L., & O’Brien, C.P. (1986). Alcohol and drug abuse treatment in three different populations: Is there improvement and is it predictable? American Journal of Drug and Alcohol Abuse, 12, 101-120. McLellan, A.T., Luborsky, L., Woody, G.E., & O’Brien, C.P. (1980). An improved evaluation instrument for substance abuse patients: The Addiction Severity -Index. The Journal of Nervous and Mental Disease. 168. 26-33. McLellan, A.T., Luborsky. L., Woody, G.E., O’Brien, C.P., & Druley, K.A. (1983). Predicting response to alcohol and drug abuse treatment: The role of psychiatric severity. Archives of General Psychiatry, 40, 62C-625. McLellan, A.T., O’Brien, C.P., Kron, R., Alterman, A.I., & Druley, K.A. (1980). Matching substance abuse patients to appropriate treatments: A conceptual and methodological approach. Drug and Alcohol Dependence, 5, 189-195. Mello. N.K. (1977). Stimulus self-administration: Some implications for the prediction of drug abuse liability. In T. Thompson & K.R. Unna (Eds.), Predicting dependence liability of stimulant and depressant drugs. Baltimore, MD: University Park Press. Mensch, B.A., & Kandel. D.B. (1988). Underreporting of substance use in a national longitudinal youth cohort: Individual and interviewer effects. Public Opinion Quarterly, 52, 100-124. Miller, W.R.. Taylor, C.A., & West, J. (1980). Focused versus broad-spectrum behavior therapy for problem drinkers. Journal of Consulting and Clinical Psychology, 48, 590-601. Nurco, D.N., Shaffer, J.W., Hanlon, T.E., Kinlock, T.W., Duszynski, K.R., & Stephenson, P. (1988). Relationship between client/counselor congruence and treatment outcome among narcotic addicts. Comprehensive Psychiatry. 29,48-54. Rounsaville, B.J., Glazer, W., Wilber, C., Weissman, M.M., & Kleber, H.D. (1983). Short-term interpersonal psychotherapy in methadone-maintained opiate addicts. Martin,

Archives

of General

Psychiatry,

40, 629636.

Rounsaville, B.J., & Kleber. H.D. (1986). Psychiatric disorders in opiate addicts: Preliminary findings on the course and interaction with program type. In R.E. Meyer (Ed.), Psychopathology and the addictive disorders. New York: The Guilford Press. Rounsaville, B.J., Kleber, H.D., Wilber, C., Rosenberger, D., & Rosenberger, P. (1981). Comparison of opiate addicts’ report of psychiatric history with the reports of significant-other informants. American Journal of Drug and Alcohol Abuse, 8, 51-69. Rounsaville. B.J., Kosten, T.R., & Kleber, H.D. (1987). The antecedents and benefits of achieving abstinence in opioid addicts: A 2.5year follow-up study. American Journal of Drug and Alcohol Abuse, 13, 213-229.

Drug Dependence

149

Rounsaville, B.J., Tierney, T., Crits-Christoph, K., Weissman, M.M., & Kleber, H.D. (1982). Predictors of outcome in treatment of opiate addicts: Evidence for the multidimensional nature of addicts’ problems. Comprehensive Psychiatry, 23, 462-478. Rounsaville, B.J., Weissman, M.M., Kleber, H.D., & Wilber C. (1982). Heterogeneity of psychiatric diagnosis in treated opiate addicts. Archives of General Psychiatry, 39, 161-166. Rush, T.V. (1979). Predicting treatment outcomes for juvenile and youth-adult clients in Pennsylvania substance abuse system. In G.M. Beschner & A.S. Friedman (Eds.), Youth drug abuse. Lexington, MA: Lexington Books. Sanchez-Craig, M., & Wilkinson, D.A. (in press). Brief treatments for alcohol and drug problems: practical and methodological issues. In T. Loberg, W.R. Miller, P.E. Nathan, & G.A. Marlatt (Eds.), Addictive behaviors: Prevention and early intervention. Amsterdam: Swets & Zeitlinger, BV. Sells, S.B., & Simpson, D.D. (1980). The case for drug abuse treatment effectiveness, based on the DARP research program. British Journal of the Addictions, 11, 14%159. Shaffer. J.W., Nurco, D.N., Ball, J.C., & Kinlock, T.W. (1986). Patterns of non-narcotic drug use among male narcotic addicts. The Journal of Drug Issues, 16,435442. Simpson, D.D. (1981). Treatment for drug abuse: Follow-up outcomes and length of time spent. Archives of General Psychiatry, 38, 875-880. Simpson, D.D., & Marsh, K.L. (1986). Relapse and recovery among opioid addicts 12 years after treatment. In F.M. Tims & C.G. Leukefeld (Eds.), Relapse and recovery in drug abuse. NIDA Research Monograph 72. Rockville, MD: National Institute on Drug Abuse. Simpson, D.D., & Savage, L.J. (1981/82). Client types in different drug abuse treatments: Comparisons of follow-up outcomes. American Journal of Drug and Alcohol Abuse, 8, 401418. Simpson, D.D., & Sells, S.B. (1974). Patterns of multiple drug abuse: 1969-1971. The international Journal of ‘the Addictions, 9, 301-314: Simpson, D.D., & Sells, S.B. (1982). Evaluation of drug abuse treatment effectiveness: Summary of the DARP follow-up research. In NZDA treatment research report. Rockvillk, MD: National Institute of Drug Abuse. Skinner, H.A. (1980). Statistical approaches to the classification of alcohol and drug addiction. Addiction Research Foundation, Toronto, Substudy No. 1157. Skinner, H.A. (1984). Assessing alcohol use by patients in treatment. In R.G. Smart, H.D. Cappell, F.B. Glazer, Y. Israel, H. Kalant, R.E. Popham, W. Schmidt, & E.M. Sellers (Eds.), Research advances in alcohol and drug problems (Vol. 8). New York: Plenum Press. Skinner, H.A., & Goldberg, A.E. (1986). Evidence for a drug dependence syndrome among narcotic users. British Journal of Addiction. 81, 479484. Sobell, L.C., & Sobell, M.B. (1980). Convergent validity: An approach to increasing confidence in treatment outcome conclusions with alcohol and drug abusers. In L.C. Sobell, M.B. Sobell, & E. Ward (Eds.), Evaluating alcohol and drug abuse treatment effectiveness. New York; Pergamon. Stanton, M.D., Todd, T.C., Steir, F., VanDeusen, J.M., & Cook, L. (1982). Treatment outcome. In M.D. Stanton & T.C. Todd (Eds.), The family therapy of drug abuse and addiction. New York: The Guilford Press. Stitzer, M.L., Bickel, W.K., Bigelow, G.E., & Liebson, I.A. (1986). Effect of methadone dose contingencies on urinalysis test results of polydrug-abusing methadone-maintenance patients. Drug and Alcohol Dependence, 18, 341-348. Stitzer, M.L., Bigelow, G.E., & Liebson, I.A. (1980). Reducing drug use among methadone maintenance clients: Contingent reinforcement for morphine-free urines. Addictive

Behaviors,

5, 333-340.

Sutherland, G., Edwards, G., Taylor, C., Philips, G.T., & Gossop, M.R. (1988). The opiate dependence syndrome: Replication study using the SODQ in a New York clinic. British Journal of Addiction. 83, 755-760. Sutherland, G., Edwards, G., Taylor, C., Phillips, G., Gossop, M., & Brady, R. (1986). The measurement of opiate dependence. British Journal of Addiction. 81, 48.5494. .l40RT 11:3-c

G. W. Martin and D.A. Wilkinson

150

Tims, F.M., & Holland, H. (1984). A treatment evaluation agenda: Discussion and recommendations. In F.M. Tims & J.P. Ludford (Eds.), Drug abuse treatment evaluation: Strategies, progress, and prospects, NIDA Research Monograph No. 51. Rockville, MD: National Institute of Drug Abuse. Waal, H. (1985). Evaluating the alternative: A follow-up study of two different farming communes. The Journal of Drug Issues, 15,413421. Wells, E.A., Hawkins, J.D., & Catalano, R.F. Jr. (1988). Choosing drug use measures for treatment outcome studies. I. The influence of measurement approach on treatment results. The International Journal of Addictions, 23, 851-873. Wilkinson, D.A., & LeBreton, S. (1986). Early. indications of treatment outcome in multiple drug users. In W.R. Miller & N. Heather (Eds.), Treating addictive behaviors. New York: Plenum Publishing Corp. Wilkinson; D.A., Leigh, G.M., Cordingley, J., Martin, G. W., & Lei, H. (1987)Dimensions of multiple drug use and a typology of drug users. British Journal of Addictions, 82, 259-273. Wilkinson, D.A., & Martin, G.W. (1983, December). Experimental assessment of two behavioural treatments for multiple drug abuse. Paper presented at the A.A.B.T. 17th Annual Conference, Washington,.D.C. Wilkinson, D.A., & Martin, G.W. (August 1984). Concordantfindings with multiple outcome measures in evaluation of behavioral treatment of multiple drug abuse. Paper presented in the symposium on Evaluation of Drug Abuse Treatments: Standards and Criterion Variables. International Conference on the Treatment of Addictive Behavior, New Benvick, Scotland. Wilkinson, D.A., Martin, G.W., Cordingley, J., & Leigh, G.M. (1989). Experimental comparison of broad-spectrum training for multiple drug

residential

treatment

and

brief,

of General

Psychiatry, 40, 639-645. back: Synanon. New

Yablonsky, L. (1965). The tunnel

manuscript,

outpatient

Addiction

self-control

Research Foundation, Toronto. Woody, G.E., Luborsky, L., McLellan, A.T., O’Brien, C.P., Beck, A.T., Blaine, J., Herman, I., & Hole, A., (1983). Psychotherapy for opiate addicts: Does it help? Archives

users. Unpublished

York: MacMillan.