A model for drug abuse treatment program evaluation

PREVENTIVE

MEDICINE

2, 510-523 (1973)

A Model

for Drug Abuse Treatment Evaluation’

Program

AMIRAM SHEFFET, ROBERT F. HICKEY, MARVIN A. LAVENHAR, EDWARD A. WOLFSON, HELEN

DUVAL,

DAVID

MILLMAN

AND DONALD

B. LOURLA

The Department of Preventive Medicine and Community Health, College of Medicine and Dentistry of New Jersey, New Jersey Medical School, Newark, New Jersey 07103

In the delivery of mental health services few provisions have been made for accountability. Until recently there has been an absence of public and governmental demands for such assessment and a lack of evaluative models or indices. Clearly the field of drug dependency treatment, as only one area of the broad mental health spectrum, is in dire need of objective, extramural evaluation. There are two basic questions that must be answered: Are individual agency goals being met? If so, to what extent? Are there maximally efficient procedures for service delivery that can attain the same goals to the same extent, at lesser cost? In other words, how good is a particular agency? When relating these questions to drug abuse treatment programs, the deductive query is, “are present programs good enough to justfy society’s retention and further government or private support?’ (10). To date, the available data have been, for the most part, uncritically gathered with judgments as to efficacy based upon minimal documented evidence. As Twain and Associates stated in 1970, “all too often important decisions of social policy and programmatic change are based on speculation, hearsay or personal experience. It is frequently assumed that reasoned judgement and experience are all that is needed for decision making about programs” (13). Additionally, the use of reasoned judgment in drug treatment programs is, at best, questionable as the established experience is still in a stage of infancy. Treatment programs have not been pressured, until recently, to take a close introspective look at the effectiveness and efficiency of their particular operations. Furthermore, as a rule, systematic data collection and feed-back systems are nonexistent or extremely primitive in existing programs. In identifying high risk areas for heroin dependence, it is clear that Newark, New Jersey has one of the most severe narcotic problems in the country. Realizing the growing needs of the community, the New Jersey Medical

1 Supported by the National Institute of Mental Health Grant No. 5-H19DA 17843-05. 510 Copyright @ 1973 by Academic Press, Inc. All rights of reproduction in any form reserved.

DRUG ABUSE

PREVENTION

FORUM

511

School established a Division of Drug Abuse within the Department of Preventive Medicine and Community Health in 1969. In the same year the Division was established, the National Institute of Mental Health provided funds to permit implementation of the Division’s programs. The New Jersey Medical School was then in a position to embark upon a unique, comprehensive narcotic addiction treatment and rehabilitation program for the metropolitan Newark community in cooperation with six established and independent agencies already involved in providing such services. It was through contractual agreement with these agencies that the multimodality treatment structure of the Medical College program was designed. Upon signing contracts with the College, the community agencies became fully recognized “affiliates” of the College and the Division of Drug Abuse. Realizing that no treatment modality had been proved predictably successful, the College was in a position to accept the philosophy of each affiliate. To operate the total program, the Medical College was designated by the National Institute of Mental Health as the grantee and was held responsible for the following activities: Serving as the fiscal coordinator for the entire National Institute of Mental Health Program; Serving as coordinator for all program aspects and fostering all interagency referrals; Maintaining and housing the central confidential narcotics registry; Providing detoxification as well as medical services for program patients; Providing consultation services for the entire system; Tracking all patients entering the system, whether they remain or not, and reporting through a structure of confidentiality on a monthly basis to the National Institute of Mental Health. DESCRIPTION

OF EACH SERVICE ELEMENT

The system includes intake, detoxification, in-patient and out-patient treatment, consultation, medical care, after-care, and follow-up. The Division of Drug Abuse, through its Drug Abuse Clinic (D.A.C.) located in the Harrison S. Martland Medical Center, is responsible for the registration, counseling, referral, and follow-up of all in-patients and out-patients in the system. The Intake Team is responsible for the registry of patients, the outlining of a treatment plan for each patient, and the referral of the patient to the most appropriate affiliate resource. Intake for all patients begins with an initial interview and completion of our own registry form and the face page of the Texas Christian University admission form. Additionally, a detailed psychosocial questionnaire is completed which enables the caseworker, in consultation with the patient, to determine what services the patient needs. These services may include: (1) Direct referral to an affiliate for treatment; (2) Placement on the Drug Abuse Clinic waiting Martland Medical Center detoxification ward;

list for admission

to the

512

A. SHEFFET

ET

AL.

(3) Admission to psychiatric, medical, or surgical wards for detoxification and treatment of additional medical problems; (4) Referral to job training agencies or to evaluative training agencies such as the New Jersey Rehabilitation Commission; (5) On-going counseling; (6) Psychiatric and/or medical or surgical consultation; (7) Referral to other appropriate out-patient clinics; or (8) Any combination of the above services. These procedures are also applicable to the Hospital out-patient who does not initiate a visit to the Drug Abuse Clinic. These out-patients are reached through: (1) Emergency Room. Many patients who come to the Emergency Room for medical, surgical, or psychiatric problems are dependent upon opiates but have never sought treatment for their drug problem. The Intake staff attempts to see all such patients and interest them in a drug treatment program. (2) Pre-Natal Clinic. A two-way referral system operates between the PreNatal and Drug Abuse Clinics. The Pre-Natal staff directs many patients to the Drug Clinic for treatment and conversely. Those patients in the third trimester of a pregnancy are admitted to an obstetrical ward for detoxification prior to delivery. (3) Other out-patient clinics which make direct referrals upon occasion. A detoxification ward for in-patients was established providing the necessary services for six male and two female patients. The detoxification period is usually 6-7 days, during which decreasing amounts of methadone are given, usually starting with a 40 mg daily dose administered by mouth. During this period the patients receive intensive counseling in group and individual settings from the caseworker assigned to this ward. They also receive orientation from staff members of affiliate agencies on the kinds of treatment available to the patient. The emphasis during the patient’s stay is placed upon the acceptance of some type of on-going treatment. However, if this is resisted by the patient, assistance with job finding and other social services is given. The nurses’ role is crucial and wide-ranging. They and the nursing assistants are responsible for the routine nursing care as well as emergency care when necessary. In addition, they must partially fulfill the role of a counselor in explaining the function of the Division of Drug Abuse and the Drug Abuse Clinic as well as some of the implications of drug use in general. The nursing staff assists in the medical and psychological evaluation of patients, collects data for the program, and notifies the Intake Team of transfers of patients to other wards. On the psychiatric ward an average of six beds are occupied at any given time by drug-dependent patients. These patients range in type from the psychotic to the slightly disturbed heroin user who may have manipulated the police into bringing him to the hospital. The Drug Abuse Clinic caseworker interviews each patient to determine if routine treatment for drug abuse is in-

DRUG

ABUSE

PREVENTION

FORUM

513

dicated and so advises the psychiatric staff. The Drug Abuse Clinic staff member, working closely with the psychiatrist and other psychiatric personnel, then counsels the patient (and often his family) to accept the rehabilitation program best suited to his needs. Six beds are set aside on the Obstetrics-Gynecology ward for drug dependent patients. The Drug Abuse Clinic caseworker assigned to that ward works with three kinds of patients: (a) Patients who are in the last trimester of pregnancy and are admitted for detoxification; (b) Patients on methadone maintenance who are admitted 2 wk prior to delivery so their maintenance dosage may be lowered; and (c) Patients admitted for delivery who are drug addicts but have never sought treatment. Besides the usual treatment-oriented counseling, the Drug Abuse Clinic worker helps these patients to utilize a wide range of social services (welfare, housing, adoption counseling, etc.). Liaison exists between the Drug Abuse Clinic and the medical and surgical wards. Many drug addicts are brought to these wards primarily for medical or surgical problems and during their stay must be seen and counseled by a Drug Abuse Clinic worker. Opiate dependent persons who are also barbiturate addicts, along with those who are dependent upon barbiturates only, are routinely admitted for detoxification after screening by the Clinic staff. The primary sources for establishment of a data base are our six affiliated agencies and the Drug Abuse Clinic in the Martland Hospital (including patients admitted to the Hospital for medical reasons other than detoxification). All registrations are coded and key punched for data processing in order to obtain epidemiological, sociological, and medical data. A system of patient tracking was designed and entirely computerized and is capable of providing comprehensive statistical data on the movements of patients into, or out of, and within the confines of the treatment program. It has the capacity of alerting individual workers through a periodic printout of those patients who are due for a 2-mo status evaluation. Additionally, it serves as an aid to patient treatment since any action involving a patient is pin-pointed on the print-out, thus allowing for more thorough patient care. The tracking system is so programmed that deficiencies in reporting are fed back to the responsible caseworker on a weekly basis for appropriate follow-up. One of the major achievements has been the introduction of the “patient disposition report.” This form is the instrument by which all patient activities are reported to the record system. It is preceded for data processing and is very simply designed to facilitate its use by workers of various levels of competence and training. Since each worker is assigned a code-case load, responsibility can be determined by the computer and can be more efficiently monitored. Outside of the Medical Center and the services provided by the College, Drug Abuse Clinic staff members maintain constant contact with the affiliates. Clinic employees are responsible for referral of patients to the affiliates as

514

A. SHEFFET

ET AL.

well as collecting information on patients who were admitted directly to an affiliate center. The division of workload responsibility in the Drug Abuse Clinic is set out according to team assignments. The Intake Team, as previously mentioned, coordinates the processing of patients to the various modalities. The Hospital Team locates drug-dependent patients admitted to Martland Medical Center with a primary diagnosis of something other than drug addiction. These patients are dispersed throughout all hospital services. The team workers are responsible for encouraging these individuals to enter one of the treatment modalities. There are three other teams that are concerned with maintaining liaison with all the contract affiliates. Each team is composed of a supervisor known as a mental health counselor and two to four case expediters. The mental health counselors have at least a bachelors degree and several years of experience in drug programs. However, most persons in these positions hold the masters degree in the humanities along with experience in similar work. The case expediters have a minimum of a high school diploma or its equivalent plus at least 2 yr experience in the drug rehabilitation field. Many of the persons filling this position are former drug-dependent individuals. The breakdown of the philosophies represented by the affiliate agencies are as follows: (a) Therapeutic community #l (T.C. 1). A strong emphasis is placed upon peer pressure, discipline, and evaluation. The treatment role of reentry and job development is stressed. A minimum stay of 6 mo is expected. (b) Therapeutic Community #2 (T.C. 2). A strong work-oriented philosophy within a very tight structure with a minimum stay of 9 mo expected. (c) Therapeutic Community #3 (T.C. 3). Dominated by a psychiatric orientation with a minimum treatment stay of I2-mo. (d) Church Affiliated After Care (O.P. I). An out-patient facility providing group and individual counseling and psychological testing. (e) Out-Patient Methadone Detoxification (O.P. 2). An out-patient center offering 4 day detoxification and follow-up group counseling. (f) Methadone Maintenance Treatment Program (M.M.). Patients are built up and stabilized on an in-patient basis; after 6 wk they are sent to the out-patient clinic. The total maximum patient population at any one time could be as many as 1600. The services offered by each affiiiate in&de medica care, counseling, genera1 education advancement and detoxification. In the following discussion survival means, “treatment survival,” that is, retention of patients in a given program. A major problem was encountered in comparing retention rates of residential therapeutic communities each of which utilizes confrontation techniques to some extent. This commonly used treatment method has been and still is unacceptable to many patients entering this treatment milieu. Therefore, drop-out rates during the initial 4 wk of treatment were significantly higher for these programs and lower for out-patient modalities. In order to provide a common base for survival comparisons,

DRUG

ABUSE

PREVENTION

FORUM

515

patients were not considered to have entered a significant treatment effort until they had been in a program for a minimum of 4 wk. This time reference was established after noting that the drop-out rate was 54% during the initial 4-wk period in the therapeutic communities. The study period in question extends over 23 mo during which time a total of 3379 patients entered the treatment program. Of a total of 2156 patients who entered treatment 1508 patients (45%) entered a “significant treatment.” The remaining 1871 were divided into three groups, 648 patients entered rehabilitation treatment but did not remain in treatment for the 4-wk minimum, 704 patients received detoxification treatment and left the program, and 519 patients left during the initial intake procedure. Three hundred (58%) of the latter group of patients were on the waiting list when they left the program. To develop the data base, every patient was seen and interviewed by a case expediter on at least a monthly basis while in treatment. The nature of these interviews was informal but they were very helpful in measuring patient responsiveness to the modality. To facilitate comparisons, two sets of survival curves were generated during the study period. Model I statistics outlined the patient retention in a modality from the first moment the patient entered treatment. Model II statistics described the survival of patients who had been in treatment a minimum of four weeks and were therefore considered as having undergone “significant treatment.” RESULTS

A. Characteristics

of Patients by Treatment

Modality

Table I provides a summary of the distribution of selected patient characteristics by treatment modality. The sex distribution was fairly constant across all modalities; approximately three of four patients in each program were male. On the other hand, a comparison among modalities of the distribution of patients by ethnicity revealed highly significant differences. Although black patients comprised approximately two-thirds of the study population, the proportion of black patients in each treatment program ranged from a low 42% (T.C. 1) to a high 77% (O.P. 2). Patients of Puerto Rican origin accounted for approximately 4% of the study population, less than 1% of the T.C. 3 patient population and as much as 7% of the O.P. 1 patient group. It was noted that the proportion of white patients as well as the proportion of female patients entering treatment were increasing during the later stages of the study period. It remains to be seen whether this observation represents a significant trend. Comparisons of other demographic characteristics among modalities yielded some interesting differences. In the entire study population, about one in five patients was not gainfully employed (excluding students and housewives), approximately one in three were high school graduates, and almost three of five were Newark residents. The proportion of unemployed individuals ranged from 10% (T.C. 3) to 38% (0.P 1). The percentage of high school graduates was lowest among Methadone maintenance registrants (30%) and highest among patients treated at T.C. 1 (45%). Newark residents

A. SHEFFET

516

TABLE

ET

AL.

I

PERCENT DISTRIBUTION OF SELECTED PATIENT CHARACTERISTICS BY TREATMENT MODALITY

Characteristic

T.C. 1

Sex: % Male 73.0 Ethnicity: % black 42.3 % Puerto Rican 1.0 Age: % under 21 48.0 70 21-25 38.8 70 over 25 13.3 Usual occupation: % white collar 12.4 % blue collar 70.6 11.9 % unemployed Education: % less than 9 yr 10.3 % 9-11 yr 44.6 % more than 11 yr 45.1 Residence: % Newark residents 16.8 Age at onset of illegal drug use: % under 18 80.5 Duration of Drug Dependence: % less than 2 yr 8.7 % more than 5 yr 50.8 Previous treatment: % previously treated 58.7 Total patients 196

No significant treatment

Total patients

T.C. 2

T.C. 3

O.P. 1

O.P. 2

M.M.

77.2

79.3

76.8

70.5

77.3

74.0

74.8

56.7 3.0

56.6 0.9

49.6 7.0

77.5 5.1

66.3 5.1

73.5 4.3

66.2 4.3

47.3 37.9 14.7

38.7 50.0 11.2

31.0 46.7 22.3

32.8 35.2 31.9

1.7 30.4 67.9

23.8 43.2 33.1

28.5 41.8 29.6

17.0 56.9 15.7

16.5 67.8 10.4

11.7 41.3 37.5

12.3 59.5 14.3

15.9 64.2 17.6

14.3 64.6 16.4

14.2 60.9 18.5

13.7 54.3 32.0

8.6 47.4 44.0

9.4 54.7 35.9

11.3 51.6 37.2

14.4 55.2; 30.4

12.6 52.8 34.7

12.0 52.6 35.4

30.6

25.0

48.1

71.9

79.0

69.5

58.6

77.9

82.8

62.9

53.8

61.5

61.7

65.0

16.7 42.4

10.0 49.1

13.9 46.3

22.8 32.6

1.7 85.6

15.5 50.7

14.5 49.9

43.5 372

58.6 116

35.0 426

29.5 217

55.2 181

36.5 1871

39.7 3379

constituted only 17% of the T.C. 1 registration, but as much as 79% of the Methadone maintenance registration. In general, patients registered in outpatient programs were more likely to be unemployed and residents of Newark, and less likely to have attained a high school degree than patients treated in in-patient facilities. A comparison of variables related to patient history of drug abuse and previous treatment also revealed considerable variability among treatment modalities. Almost two-thirds of the study patients began using drugs before reaching their 18th birthday, one-half of them were abusing drugs for more than 5 yr, and approximately 40% had undergone treatment prior to entering the New Jersey Medical School Program. In general, in-patients were younger than out-patients and were most likely to start abusing drugs at an earlier age. The highest proportion of long-term drug abusers (more than 5 yr) was recorded for Methadone Maintenance patients (86%). The percentage of

DRUG

ABUSE

PREVENTION

FORUM

517

patients who were abusing drugs for less than 2 yr was highest at O.P.2 (23%). Individuals registered at each of the two therapeutic communities and at the methadone maintenance facility were more likely to have entered the program with a history of previous treatment than those registered at the two outpatient modalities. Early in the program period, an attempt was made to randomly assign incoming patients to treatment facilities. This procedure proved to be impractical and was discontinued. It is clear that a nonrandom self-selection process exists by which certain segments of the patient population are disproportionately represented in some treatment programs. This finding emphasizes the importance of subdividing the patient population into homogeneous subgroups before meaningful comparisons of treatment efficacy can be made among various programs. Certainly comparative evaluations of treatment success have little meaning if they do not take into consideration differences in patient populations that penalize those programs which attract a relatively large number of “high risk” patients (however one defines “high risk”). It is also interesting to note (from Table I) that patients who never reached a significant treatment stage differed with respect to some characteristics from those who were given significant treatment. The former group of patients were more likely to be older, black, and a resident of Newark than the latter group of patients.

B. Modality

Retention

Although the state of the art of treating drug dependent individuals is still in an embrionic stage, much experience and knowledge has been acquired over the past decade. Even with the advances which have been made, we still lack a definition or description of what success is or when it has been achieved in treating drug users. According to Glasscote and associates (9), we may never reach a consensus of opinion on this point. Rather than attempt to arrive at a definition of success, we adopted the hypothesis alluded to by Gearing of Columbia University (3). Gearing’s findings suggest that the longer a patient remains in treatment, the greater the chances of success based upon four criteria: (A) Freedom from heroin “hunger” as measured by repeated, periodic “clean” urine specimens; (B) Decrease in anti-social behavior as measured by arrest and/or incarceration; (C) Increase in social productivity as measured by employment and/or schooling or vocational training; and (D) Recognition of and willingness to accept help for excessive use of alcohol and other drugs, or for psychiatric problems. It also seemed appropriate to evaluate the patient retention rates in the various treatment modalities because at this early stage of evaluation there were few “graduates” in each of our affiliate programs. The corollary assumption is that those patients leaving treatment within the first year are, in the overwhelming majority of cases, going to return to drugs. This appears to be

518

A.

SHEFFET

ET

TABLE MAIN

SURVJYAL

AL.

II

TRENDS MODEL II -ONLY PATIENTS REMAINING AT LEAST Fouw WEEKS ARE INCLUDED Percent

remaining

IN TREAT~L~ENT

in treatment

Classification of modality

No. of pts.

4 wk

8 wk

12 wk

26 wk

52 wk

78 wk

TX. T.C.

1 2

162 372

100.0 100.0

92.4 89.1

82.3 74.2

65.7 38.6

54.8 23.5

38.8 18.7

T.C.

3

116

100.0

80.2

42.8

23.1

3.9

2.6

O.P. 1 O.P. 2 M.M.

426 217 181

100.0 100.0 100.0

63.6 83.3 96.1

36.0 33.8 83.4

11.5 9.0 72.1

3.7 4.3 61.6

52.4

reasonably well documented (3,7,8,12). Survival or retention curves were, therefore, generated to compare the various programs. Table II and Figs. 1 and 2 demonstrate the patient survival by modality. Since many residential therapeutic communities consider 12 mo to be a reasonable treatment period time frame, particular note should be given the 52-wk survival rate. Figure 1 summarizes the data relating to any treatment at all (Model I); whereas, Fig. 2 and Table II refer to Model II and include only patients remaining in treatment for at least 4 wk. It is clear that there were striking differences among modalities in retention rates. Methadone maintenance and T.C. 1 were approximately equal; T.C. 2 was intermediate in retention capacity; and T.C. 3 as well as both out-patient programs had poor 1-yr retention rates. The relative positions of the curves (Figs. 1 and 2) were for the most part unchanged whether or not the 4-wk drop-outs were included. There was no evidence that programs with better retention rates in Fig. 2 appeared to do better because high risk patients left the program during the first four weeks.

Weeks

FIG. 1. Survival curves those entering treatment).

in trsatment-

for six narcotics

addiction

treatment

programs-Model

I (includes

all

DRUG

ABUSE

PREVENTION

FORUM

519

I

z

iT.C.3

---o.p;1

-2 2

1 0

h ’ 6

h

’

’

‘:I

16 24 Weeks in treatment

’

32 -

’

40

0 0

46

’ 56

FIG.% Survival curves for six narcotics addiction those remaining in treatment at least 4 wk).

1

64

1 72

treatment

I!)

1 66

SO

b

96

programs-

1

104

Model

II (includes

only

In order to understand the variables which might affect patient survival, the retention capabilities of various population subgroups were compared. Comparisons which were statistically significant demonstrated that males in T.C. 1 survived longer than males in T.C. 2; males in T.C. 2 did better than males in T.C. 3. All patients in O.P. 1 did better than the patients in O.P. 2. In the following areas of comparison, the differences approached statistical significance. Males survived longer in therapeutic communities than females; whereas, females survived longer in out-patient programs than males. White males survived longer than black males in out-patient centers. All of these comparisons may very well prove to be statistically significant as more data are collected. These comparisons did not particularly aid our initial objective, namely to identify several demographic characteristics which would permit the matching of an addict to the program that could best meet his particular needs. However, one analysis was helpful in this regard; in T.C. 2 high school graduates had significantly greater retention rates than those who had not completed high school (see Fig. 3). In striking contrast, high school graduates in T.C. 1 did no better than the dropouts.

Grad.

orop. Gmd.

0

FIG.3.

Survival

6 16 24 Weeks in treatment

32 -

40

46

56

64

72

60

66

curves at T.C. 1 and 2 for male high school dropouts

96

t

104

and graduates

-Model

II.

520

A. SHEFFET

ET AL.

To investigate further program retention and demographic characteristics which might affect patient survival, comparisons were generated on the basis of the location of the patients’ residences. As the majority of patients in the study period were black, patient survival for black Newark (inner-city) residents and black non-Newark residents was analyzed for T.C. 1, T.C. 2, and T.C. 3. There was an increased survival rate for non-Newark blacks over Newark blacks. As the number of patients available for analysis increases, perhaps other statistically significant differences within treatment modalities will emerge.

C. Graduates Thus far, there have been relatively few “graduates.” In the case of therapeutic communities, names are submitted by the community and verified independently by our unit. Only graduates making a satisfactory adjustment were included in this analysis. In the case of methadone maintenance and out-patient programs, any addicts still in treatment after 1 yr and considered to be adjusting well were, for the purposes of this analysis, called graduates. In analyzing the demographic characteristics of 146 program graduates and comparing these to the characteristics of the total patient population (including those who accepted and those who refused treatment) no striking differences were found. For example, the percentage of males and females entering treatment was the same as the sex characteristics for those graduating (males 75%, females, 25%). Some comparisons were surprising in that they did not yield significant difference in survival. The most interesting was the fact that patients entering treatment voluntarily survived just as long as those entering under pressure from law enforcement agencies. In other words, it would appear that judicial pressure exerted upon addicts to enter treatment programs had no effect upon how long an individual would stay in treatment. The largest number of graduates have been from two programs, 92 from M.M., and 28 from T.C. 1 (63 and 19%, respectively, of the total 146 graduates). No statistically significant differences were observed with regard to the distribution of various demographic characteristics among the total study population, those who entered treatment in all programs, and the graduate populations. There were also no significant differences detected between graduates and those entering treatment in those two programs. Thus, the graduates did not constitute an identifiably different population in regard to the demographic characteristics studied. DISCUSSION

As Glasscote et al. (9) stated, therapeutic communities have been faced with severe restrictions with regard to their capabilities for evaluation. The most pressing problems have been a lack of funds for in-depth evaluation, a lack of methodological expertise and a lack of objectivity. Many of the data presented by various therapeutic groups are clearly unacceptable. These are apparently based on biased data and on incomplete or inadequate follow-up. Some groups exclude from efficacy evaluation those dropping out in the first 90

DRUG

ABUSE

PREVENTION

FORUM

521

days, even though these dropouts may constitute from 50 to over 80% of the initial treatment population. One of our therapeutic community affiliates boasts publicly of a cure rate of at least 8O%, but in our objective analysis, the recovery rate has been found to be less than 5%. This kind of discrepancy can only be obviated by the kind of standardized extramural evaluation herein described. Far and away, the best evaluation of treatment programs for addicts are those of Dr. Frances Gearing who ‘has, for the last seven years, carefully looked at methadone maintenance programs in New York City (2-8). Even these valuable studies are not entirely extramural; the funding and staff have been limited so that, except for small followup studies on specific aspects, the data have been gathered intramurally, fed into the computer, and then evaluated extramurally. Our own program differs in that all data are gathered extramurally, and the evaluation, using standardized criteria, covers a variety of modalities. The data we have collected are intriguing in several respects: First, the 52-wk retention rate of patients in the methadone maintenance program under our surveillance was less (62%) than the approximately 80% l-yr retention found by Dr. Gearing in her studies and also reported by other observers (3,4,9,II). Second, there were striking differences in retention rates among our three therapeutic communities. Each felt it had a very good recovery rate. Indeed, two of the three are nationally renowned and their literature and pronouncements suggested that the overwhelming majority of treated addicts recovered. Yet at the end of one year, one had lost 45% of the addicts who had entered treatment and remained for at least 4 wk. The other had lost 96 % of such patients in a similar time period. Clearly, it is imperative that disinterested assessment be made of various treatment modalities if we hope to obtain valid and generally applicable data. Third, the results of treatment in out-patient, nonmaintenance units was discouraging. This is hardly surprising. Drug dependency treatment in the 1950’s and early 1960’s was carried on in either in-patient units or out-patient units. Out-patient treatment alone was found to be so unsuccessful that by the mid-1960’s, it was felt that at least initial therapy should be carried out on an in-patient basis, Now the pendulum is swinging back to treatment solely on an out-patient basis. Our results with two out-patient units suggest that great caution must be exercised in regard to out-patient, nonmaintenance programs. Most are likely to fail and all receiving public funds should be scrutinized carefully on an extramural basis. Fourth, we thought that it should be relatively easy to match a given addict to the best program for that individual by relating treatment success studied prospectively to simple demographic characteristics. Clearly, this is not so. Success or failure in a given modality could not be related to age, sex, duration of drug use, age at onset of use, prior treatment attempts, ethnic@, occupation, or employment. Indeed, the only variable of major significance was educational status, and this applied to only one of the six programs studied. However, this is the kind of relationship we are seeking. To improve its success

522

A. SHEFFET

ET

AL.

rate and expend scarce monies most effectively, the above-mentioned program would do well to consider restricting its admissions to those most likely to benefit from its particular therapeutic regimen, that is, in this case, those who have graduated from high school. This is feasible because there are many more addicts than effective rehabilitation units; consequently, each unit can define specific requirements for admission and still easily maintain a full roster, Addicts who had not attained that educational level would be referred to programs such as T.C. 1 in which educational attainment did not affect treatment outcome. The meaning of the finding that non-Newark residents in some programs did better than Newark residents is currently unclear and is now under more detailed investigation. What we are proposing is that we attempt to augment our sophistication so that programs stop admitting all addicts and confine their activities to those more likely to respond to that particular modality. If this could be done, then initial success or failure rates for any valid rehabilitation program or modality would not be considered as a reason for approbation or condemnation, but rather would be viewed from a more constructive epidemiological point of view. If a program has only a 4% success rate, as our T.C. 3 affiliate, the objective would be to identify that 4% and determine whether they could be categorized and separated demographically or psychosocially from nonsuccesses; that program could then begin to concentrate on a clientele with specific characteristics, and thereby increase its success rate. Clearly, in our studies of retention rates and our analysis of graduates, simple demographic analyses are thus far inadequate to match addict to program. Currently, we are doing the following to improve our analysis: (a) Investigating not only retention but also the fate of those leaving the program 1-12 mo after starting therapy. Studies by Gearing (3) and by Perkins and Block (12) suggest that the overwhelming majority of such drop-outs rapidly return to heroin abuse and its adverse social consequences. (b) Administering a far more detailed questionnaire that will assess psychosocial characteristics, such as alienation and frustration, and will permit a more detailed analysis of functioning in addition to retention or “successful” graduation. This is Phase II of our three pronged investigation, (c) Starting an on-site assessment of four treatment programs (Phase III) to investigate whether success or failure relates not to demographic characteristics (Phase I evaluation herein reported) or to demographic plus psychosocial characteristics (Phase II evaluation), but rather to charisma of the program, its leadership and specific interrelationships between program and addict. If this turns out to be true, then matching addict to program will be less important than focusing on the nature of the program itself. It will for example be desirable to assess survival in T.C. I which has a lower percentage of blacks, a higher percentage of non-Newark residents and a higher percentage of high school graduates by analyzing these three parameters together rather than as discrete variables. (d) As numbers in each program increase, we are turning increasingly to multivariate statistical analysis. Once our data are completed a careful corn-

DRUG

ABUSE

PREVENTION

FORUM

523

parison will have to be made of our data and those from other evaluating units including the groups at Texas Christian University, The Johns Hopkins University, and Columbia University School of Public Health. It seems to be most likely that our three-phased evaluation will show that both demographic-psychosocial characteristics and programatic content will be important. Although the results presented in this report are preliminary, and although our analysis of Phase I evaluation is still incomplete, the data show clearly the enormous value of extramural, standardized, prospective evaluation of rehabilitation programs for drug-dependent persons. This kind of evaluation must be done for every major program in order to attempt to assign an addict to the program optimal for him, to choose the best programs for a given geographic area and to expend public and private monies most prudently. REFERENCES 1. BENT, R. J., KIESLER, D. J., AND PUTMAN, D. G. “A Model for Program Evaluation and Development-Descriptive and Experimental Methodology for Drug Abuse Services.” Georgia Mental Health Institute, Atlanta, Georgia, 1973. 2. GEARING, F. R.“A Road Back from Heroin Addiction,” p. 157. Proceedings of the Fourth National Conference on Metbadone Treatment. National Association for the Prevention of Addiction to Narcotics, New York, 1972. 3. GEARING, F. R. Successes and Failures in Methadone Maintenance Treatment Heroin Addiction in New York City, USPHS publication No. 27, p. 2172. Proceedings Third National Conference on Methadone, National Institute of Mental Health, Rockville, Maryland, 1970. 4. GEARING, F. R. “Methadone Maintenance Treatment Five Years Later-Where are They Now?” Columbia University; Paper presented at the American Public Health Association Annual Meeting, Adantic City, N. J., November 1972. 5. GEARING, F. R. “People Versus Urines,” p. 325. Proceedings Fourth National Conference on Methadone Treatment, National Association for the Prevention of Addiction to Narcotics, New York, 1972. 6. GEARING, F. R. “Death Before, During and After Methadone Maintenance Treatment in New York City,” p. 493. Proceedings Fourth National Conference on Methadone Treatment, National Association for the Prevention of Addiction to Narcotics, New York, 1972. 7. GEARIX, F. R., AND BRILL, H. “Methadone Maintenance Treatment Programs in New York City and Westchester County.” Monograph, Columbia University, New York, April 1972. 8. GEARING, F. R. “Myth Versus Fact in Long Term Methadone Maintenance Treatment; The Community’s Viewpoint,” p. 945. Proceedings Fifth National Conference on Methadone Treatment, National Association for the Prevention of Addiction to Narcotics, New York, 1973. 9. GLASSCOTE, R., SUSSEX, J., JAFFE, J., BALL, J., ANU BRILL, L. “The Treatment of Drug Abuse, Programs, Problems and Prospects.” Joint Information Service, Washington, D. C., 1972. 10. MILKMAN, R. “Federal Drug Abuse Program Evaluation: Special Action Office for Drug Paper presented at the Fifth National Methadone Conference, WashAbuse Prevention.” ington, D. C., March 1972. 11. PERKINS, M. E., AND BLOCH, H. I. Survey of a methadone maintenance treatment program. Amer. J. Psychiat. 126,1389-1396 (1970). 12. PERKINS, M. E., AND BLOCH, H. I. A study of some failures in methadone maintenance treatment. Amer. J, Psychiat. 128,47-51 (1971). 13. TWAIN, D., HARLOW, E., AND MERWIN, D. “Research and Human Services; A Guide to Collaboration For Program Development.” Research and Development Center, Jewish Board of Guardians, New York, 1970.

of