Evaluation of an inpatient alcohol rehabilitation programme

333

Drug and Alcohol Dependewe, ll(1963) 333452 Elsevier Scientific Publishers Ireland Ltd.

EVALUATION OF AN INPATIENT REHABILITATION PROGRAMME

ALCOHOL

D. IAN SMITH

Western Australian Akohol and Drug Authority, Salvatori House, 35 Outmm Stwet, West Perth, 6005 (Australia) (Received December 29th, 1982)

SUMMARY

A retrospective matching procedure was used to form a control group of 145 men who underwent detoxification, but unlike the treatment group did not enter the Quo Vadis Hospital rehabilitation programme. On a wide range of variables for which data was obtained at follow-up interview, respondents in the two groups were very similar. By contrast, official records data showed that the treatment group had significantly more readmissions to the detoxification hospital during the follow-up period than the control group. A number of methodological issues are briefly discussed.

Key

words:

Treatment-Alcohol

-Detoxification

-Outcome

INTRODUCTION In the field of alcohol treatment there is an urgent need for evaluative studies. In a number of literature reviews [1,23 it has been shown that many rehabilitation programmes do not achieve their objectives and may even be worse than ineffective [3]. As a result, health and welfare workers have become increasingly aware of the need to scientifically evaluate the effectiveness of such programmes. The purpose of this paper is to summarise the principal findings of a study* which evaluated the effectiveness of the Western Australian Alcohol and Drug Authority’s Quo Vadis Hospital rehabilitation programme by

*D.I. Smith, unpubliahed report, Western Australian Alcohol and Drug Authority, 1981 and unpublished supplementary report, 1982.

@ 1983 Elsevier Scientific Publishers Ireland Ltd. Printed and Published in Ireland

334

comparing a group of persons who entered the programme during 1978 and 1979 with a similar group of persons who did not enter the programme. The Quo Vadis Hospital programme The Hospital could accommodate 35 patients and had 30 full-time and eleven part-time ataB comprising Nurses, Welfare officers, Rehabilitation Assistants and support ataB. Medical OfEcers and an AA Counsellor regularly visited the Hospital. In the work rehabilitation programme patients were encouraged to undergo re-training in work skills and develop regular working habits so that when a person left the Hospital he would be ready to obtain employment. In a weekly talk by a Medical Oflicer the adverse physiological effects of alcohol were stressed. The nursing staff reinforced the material presented by the Medical Oflicer and encouraged patients to have a high standard of nutrition, hygiene and physical health. Some emphasis was placed on the development of leisure time activities. Patients regularly participated in crafts and hobbies sessions. A range of sporting facilities were available for use by the patients at the Hospital and regular weekend outings were arranged to places with non-drinking facilities. Participation in AA was a vital programme constituent. The patients attended one outside evening meeting and conducted their own Quo Vadis Hospital meeting one night a week. In view of the AA component, not surprisingly the programme had an abstinence goal. Patients were not permitted to consume alcohol and patients returning from leave were required to submit to a breathalyzer test. On admission patients were advised that a positive test result would lead to their discharge. While patients of both sexes, who may be either alcohol- or drug-dependent were admitted, the majority were male non-aboriginal alcohol-dependent persons. Approximately three-quarters of the patients were admitted following their discharge from Aston Hospital, the Authority’s medical detoxification unit in West Perth. Most of the remaining patients came from Carrellis Centre, the Authority’s assessment centre, with the balance being referred from hospitals and other health centres in the metropolitan area. Patients were encouraged to stay for approximately 12 weeks, but some left much sooner while others stayed for as long as 6 months. Type of experimental design The need to include an appropriate control group in studies evaluating alcohol treatment programmes has been stressed 131.Otherwise it would not be possible to determine the extent to which clients changed as a result of the treatment as distinct from the natural tendency of alcohol dependent persons to improve with time [2,41. A retrospective matching procedure similar to that of Annis and Liban [5] was used to form the control group. In the terminology of Campbell and Stanley 161,the experimental design was quasi-experimental and in parti-

335

cular it was a non-equivalent control group design in which the patients were self-selected, that is, the researcher did not control whether a person entered the treatment or control group. Especially if the two groups are well matched, such a study design can provide the necessary information to rule out the hypothesis that an intervention is having an effect. The use of a retrospective matching procedure had a number of advantages over random assignment to treatment and control groups in a prospective study. Firstly, the ethical question of denying patients the opportunity to enter treatment when facilities were available was avoided. Secondly, patient selection was not restricted to persons willing to participate in an experiment. This can be an important source of bias 141. Thirdly, as staff and patients did not know that the evaluative study was to be conducted, threats to external validity arising from special efforts by the staff or patients or from a Hawthorne effect did not occur. Fourthly, it ensured that a significant result in favour of the treatment group could not be attributed to an aversive state in the control group patients arising from their disappointment at their assignment to a control rather than experimental condition. Emrick 121summarised five studies and showed that the results suggested that elements in the treatment environment harm alcohol dependent persons by eliciting thoughts and feelings of disappointment, abuse, neglect or rejection when the patients are not selected for treatment after volunteering. This aversive state can function as an antecedent to further drinking, resulting in fewer patients improving. In the often cited study of Sobell and Sobell 171,no less than 40% of the clients in the non-behaviour therapy group felt permanently rejected at a 2-year follow-up interview. A further 20% initially experienced rejection, but then felt better. Only 13% felt ‘good’ or ‘okay’ about not being selected for the experimental treatment. The questioning of the value of random assignment has not been confined to the alcohol treatment field. Black [8], for instance, argued that where there are many factors which can influence the outcome and the factors are both multiple and unpredictable, effective randomisation becomes impossibly complex. In a leading article the British Medical Journal 191stated that ‘the controlled trial has been placed on too high a pedestal and needs to be brought back to earth’. After showing that random assignment can be inferior to the use of retrospective controls, Cranberg 1101concluded that ho one method is best in every case, and the choice of the method rests on consideration of the options and their consequences. Those designing a trial should not succumb to fashion in experimental design and assume that a ‘strictly controlled’ clinical trial is necessarily a trial that uses concurrent controls’. Due to the artificiality of random assignment, it is clear that evaluative studies of alcohol treatment programmes which include random assignment to treatment and control conditions will not necessarily give more accurate results than studies including retrospective matching. In a thought-provoking paper Athey and Coyne 1111argued that evaluative studies which involve the imposition of special treatment (or no

treatment) conditions by randomisation are unlikely to be of much, or perhaps any benefit. To overcome these important methodological problems, Cottheil [12] advocated that evaluators should follow matched groups of alcohol dependent persons exposed to different treatments or no treatment. This was the approach adopted for the Quo Vadis Hospital evaluation study. METHOD

Treatment

and control groups

The treatment group comprised 145 male non-aboriginal alcohol dependent persons who were first admitted to Quo Vadis Hospital in the period from January 1, 1978 to December 31, 1979, within 7 days of being discharged from Aston Hospital following detoxification. The one-week “cutoff” point was selected as patients sometimes like to spend a weekend at home between being discharged from Aston Hospital and being admitted to Quo Vadis Hospital. Aboriginal, female and drug dependent persons were excluded from the study due to the small number of such persons admitted to Quo Vadis Hospital. Patients admitted to Quo Vadis Hospital for rest and recuperation reasons before entering the Authority’s Ord Stree t Hospital probramme were also excluded. (The Ord Street Hospital has an eight week inpatient programme for persons with less physical damage and more verbal ability than those admitted to Quo Vadis Hospital). From the viewpoint of an evaluative study, 145 matched persons per group was a very satisfactory number, especially when it is realised that some major studies in the alcoholism treatment field have only had 50 or less persons per group

171. For each person included in the treatment group a similar person was selected from the Aston Hospital records to form the control group. The criteria for matching were: (i) male; (ii) alcohol dependent; (iii) nonaboriginal; (iv) never admitted to Quo Vadis Hospital; (v) approximately the same age; (vi) similar marital statu$; (vii) similar usual occupation as denoted by the categories in Congalton’s [13] scale; (viii) similar employment status of unemployed or employed when entered the study; (ix) similar place of birth; (x) admitted to Aston Hospital at approximately the same time as the treatment group person in order to elinate any confounding due to seasonal changes; (xi) approximately the same duration of hospitalisation in Aston Hospital prior to entering the study; (xii) approximately the same number of prior admissions to Aston Hospital in order that the two groups would be approximately equivalent in past treatment experiences, the importance of which was stressed by Schuckit and Cahalan [3].

*Data was analysed with the Statistical Package for the Social Sciences 1141. Where an F test indicated at a 0.05 level of probability that homogeneity of variance could not be assumed for a t-test comparison, a separate variance formula was used. All t-tests were two-tailed. For x2 tests with only one degree of freedom, Yates correction for continuity was used.

337

In addition to comparing the treatment and control groups on the variables used for matching, the two groups were compared *.** on a further 37 variables taken from the Authority’s records. Some significant differences between the two groups were found (Table I), but with the exception of the control group having had significantly more prior admissions to Ord Street Hospital the differences were apparently of no prognostic value in the sense improved patients in both groups. Although the difference was not significant, the control group had 9% more prior admissions to Aston Hospital. During the period from Hospital often used a short form to assist in determining whether a patient should be encouraged to go to Quo Vadis Hospital following detoxification. Some 44 of the treatment group patients, by only 14 control group patients were selected for assessment. Treatment group patients were rated as having significantly better physical helath and in total as being significantly more suible for admission to Quo Vadis Hospital. Even if the physical health sub-scale was omitted from the total the difference was still significant. It appears that the selection procedure produced a control group of patients who were very similar to the treatment group on a number of variables which have been demonstrated in previous research to be predictive of treatment outcome [1,3,41. However, it should be stressed that the two groups could have been different on variables for which no information was recorded. In particular, the possibility of the treatment group being more motivated to overcome their alcohol dependency should not be overlooked 1151.Thus the success rate of the control group was to be viewed as a minimum statement of the extent to which the treatment group could have been expected to improve without the benefit of the Quo Vadis Hospital programme. Further evidence of the equivalence of the two groups is to be found in the following four factors which apparently acted to determine whether an Aston Hospital patient went to Quo Vadis Hospital Firstly, some suitable patients could not have gone due to a shortage of accommodation at Quo Vadis Hospital on some occasions. Secondly, at follow-up interview nine control group respondents reported that they had wanted to go to Quo Vadis Hospital, but Aston Hospital staff would not let them go. Thirdly, as a

**In the interests of brevity many tables in the original and supplementary reports (footnote p. 000) have been omitted from the paper and deposited with the National Auxiliary Publications Service. They include further comparisons of treatment and control groups, comparison of patients interviewed and not not interviewed in both groups, additional comparisons of the two groups on official records follow-up and interview data, comparisons of the short and long stay sub-groups of the treatment group, comparisons of the men who did or did not ‘improve’ in each group, comparison of 55 patients excluded from the treatment group with the treatment and control groups, assessment of the methodology according to the Sobell’s criteria, product moment correlation matrices, factor analyses and discriminant function analyses. To obtain these, order NAPS Document No. 04094 from ASIWNAPS, Microfiche Publications, P.O. Box 3513, Grand Central Station, New York, N.Y. 10163. Remit with order $4 for microfiche or $26.05 for full-size photocopy.

338

TABLE I OUTCOME OF MATCHING PROCEDURE Figures in brackets next to the means are SD. *P < 0.05; **P -c 0.01;***P -=z0.001. Treatment

Control

Age

Under 35 years 36-44 years 45 - 49 years 56+ years

41 43 31 30

43 46 26 26

Marital status when entered the study Single Married or de-facto Separated, divorced or widowed

64 16 65

65 16 64

Usual occupation Professional, managerial, oflice and sales workers Skilled workers Semi-skilled workers Unskilled workers Invalid pensioner Not known

8 35 44 48 10 0

13 44 25 52 9 2

9 136

11 134

37 48 60

36 38 71

25 32 44 44

22 37 42 44

Employed when entered the study YeS No Place of birth Western Australia Elsewhere in Australia Overseas Date admitted to Aston Hospital for admission which led to inclusion in the study January-June, 1978 July -December, 1978 January -June, 1979 July, 1979 or later

,

Number of prior admissions to Aston Hospital 1 2 3 4 5 6-15 M%ilI

25 11 3 3 20 2.5t2.5)

62 37 15 13 7 11 2.7(2.0)

Duration in days of admission to Aston Hospital before entering the study

5.7(2.2)

4.9(2.4)**

83

TABLE

I (Cord.)

Blood alcohol level when admitted to A&m Hospital before entering study Mean duration in days of aI1 prior admissions to Aston Hospital Number of prior admissions Street HcepitaI

the

Treatment

Control

0.070% (0.094) (N = 124)

0.108% (o.llo)** (N = 119)

11.4 (10.5)

10.4 (8.2)

0.24 (0.64)

0.41 (O.&W+

4.5 (0.6) 2.7 (1.2) 3.0 (0.7) 2.9 (0.8) 13.1(1.9) (N=44)

3.9 (1.0)‘; 2.1(1.2) 2.6 (0.8) 2.3 (1.3) 11.0 (2.4)“’ (N = 14)

to Ord

Assessment at A&m Hospital of patients’ suitability for admission to Quo Vadis Hospital Physical health Support from family Social relationships Employment history Total

driver was not available to transport patients from Aston Hospital to Quo Vadis Hospital on weekends, patients discharged on weekends were significantly (P < 0.01) under-represented in the treatment group. Furthermore, the weekend figures could not be accounted for by an excess of Monday discharges to Quo Vadis Hospital as 36 treatment group men and 34 control group men had Monday discharges. Fourthly, and all else being equal, the longer a patient was at A&on Hospital, the greater was the likelihood that the staff would select him for assessment and admission to Quo Vadis Hospital. Table I shows that the treatment group had a significantly longer duration of hospitalisation at Aston Hospital before entering the study. The above four points appear to explain how it was possible to form approximately equivalent treatment and control groups of patients drawn from the one hospital over the same time period. The questionnaire A number of reviewers, e.g. Ref. 1 have quite appropriately criticized many evaluations of alcohol treatment programmes for placing an undue emphasis on changes only in alcohol consumption and especially the attainment of abstinence. This is not to say that drinking behaviour is not a useful and important outcome measure, but it should not be used as the only criterion since it does not correlate perfectly with all other areas of functioning. Adequate evaluation requires follow-up data on a variety of

indices, one of which is drinking behaviour. The questionnaire used by Armor et al. [l]was selected as the starting point for developing a suitable questionnaire for the study. Inappropriate questions (e.g., those pertinent only to female patients) were deleted, wording altered if necessary to make the questions suitable for an Australian survey, and some additional questions inserted. Rosenberg’s 1161ten item self-esteem scale, as modified by Charalampous et al. 1171was included in the questionnaire. Respondents were asked for details of their alcohol intake in the week prior to interview, after which they were asked if it was a typical week. The 27 respondents in the treatment group and the 30 in the control group who replied in the negative were then asked for details of their alcohol consumption in a typical week. For such persons their alcohol intake in a typical week was inevitably higher than for the week prior to interview as they were difficult to locate for interview when drinking heavily. A table produced by the Australian Associated Brewers I181 was used to convert all intake into grams of alcohol. Fhcedure

A Psychologist employed by the Authority was responsible for matching the treatment and control groups, and locating and interviewing the patients in both groups. All available information in the Authority’s records was first used to locate as many persons as possible. Next, visits to Voluntary Agencies and boarding houses yielded many persons. Approaches were also made to a wide variety of health and welfare agencies which readily agreed to assist in locating persons for interview. In.the advent of a person included in the study being admitted to Aston Hospital, the Psychologist was immediately notified. A search was made of death certificates for persons deceased in Western Australia. In summary, a wide variety of procedures were used to trace the persons included in the study. Any lead was followed up until the person was located or all avenues of enquiry were exhausted. Where a person could not be located, wherever possible, a ‘significant other’ interview was conducted. Three such interviews were conducted for the treatment group, and five for the control group. Respondents who were personally interviewed were requested to submit to a breathalyzer test at the conclusion of their interview. In addition to the ‘significant other’ interviews, some missing data resulted from respondents with high blood alcohol levels being unable to answer all questions. Reliability and validity of self-report information

With the exception of the questions relating to the Authority’s three hospitals, it was not feasible, or in many cases practicable, to obtain verification of the information volunteered during interview. In view of the heavy reliance placed on the self-report data below, 14 studies which examined the reliability of such data were reviewed in the original report

341

(footnote p. 333) since lack of evidence on response reliability clearly weakens any conclusions that can be drawn about programme outcome. With the exception of one paper, the studies reviewed generally supported the assertion that self-reports of life history data and of drinking and related behaviour are both reliable and valid. The use of self-reported information for programme evaluation purposes therefore appeared to be justified f191. Outcome of follow-up survey

Some 112 (77%) of the men in the treatment group were interviewed, while two declined to participate and four were found to be deceased. The corresponding figures for the control group were 94 (65%), seven and two respectively. A possible explanation for the difference between the groups is that as a result of the apparent negative outcome of their treatment experience, persons in the treatment group were readily accessible due to their significantly higher rate of admission to Aston Hospital and greater usage of voluntary agencies (see below). Although the difference was not significant, only 25.5% of the control group interviews were conducted at a health agency as compared to 35.7% for the treatment group. All the deceased persons had a cause of death which was either directly related to alcoholism or had been demonstrated in previous research to be frequently associated with alcoholism. Persons difficult to follow-up can sometimes have a worse outcome than patients located early [201. The question therefore arose as to whether as a result of the different follow-up rates for the two groups, the treatment group did not include a greater proportion of persons functioning poorly. In the last four months of the survey, 23% of the persons interviewed in the treatment group claimed to have been abstinent for the week prior to interview. For the control group the corresponding figure was 40%. Had the survey been continued, this suggests that persons located in the control group were more likely to have been abstinent in the week prior to interview than persons in the treatment group. In terms of both the method and interviewer effort devoted to locating persons for interview, the two groups were not significantly different. There was no difference between the two groups according to month of interview. As the treatment group entered the follow-up period significantly later than the control group, the mean length of the follow-up period to date of interview for the control group was 15.334 months (S.D. = 7.195) as compared to 12.085 months (SD. = 6.756) for the treatment group (P < 0.01). To correct for this difference, the answers which individual respondents in the control group gave to questions dealing with the total number of events (e.g., drink-driving convictions) in the follow-up period were multiplied by 0.835. No data modification was performed on answers to questions dealing with events in the preceding week or 30 days, or on official records data.

Comparisons of patients interviewed and not interviewed Patients not interviewed in the treatment group were significantly younger, had less prior admissions to both Aston and Ord Street Hospitals, less days between being discharged from Aston Hospital and admitted to Quo Vadis Hospital, and were rated higher on the physical health sub-scale of the Aston Hospital assessment of suitability for admission to Quo Vadis Hospital. Patients with an unskilled occupation were over-represented in the not interviewed group. Patients not interviewed in the control group were also significantly younger. With the exception of the physical health assessment for the control group all the other comparisons listed above were in the same direction as for the treatment group, but were not significant. Especially in the case of the control group, patients not interviewed were likely to have been born in the Eastern States of Australia and probably were not interviewed due to their return during the follow-up period. This may also explain why patients not interviewed in the treatment and control groups were significantly less likely to have been admitted to Aston Hospital in the follow-up period than the patients interviewed. The mean number of admissions for patients not interviewed in the two groups was very similar. The above differences clearly impose some limitations on the findings of the evaluative study. However, were a range of treatment options to have been available for alcohol dependent persons in Perth it can be argued that many of the patients not interviewed in the treatment group would not have been admitted to Quo Vadis Hospital for inpatient treatment. In terms of their age, small number of prior admissions to both Aston and Ord Street Hospitals, and higher physical health ratings these patients would seem to have been well suited for outpatient treatment. RESULTS

O&ial records data Details of all admissions which patients in the treatment and control groups had to Aston Hospital in the two years after entering the follow up period are shown in Table II according to how long after leaving Quo Vadis or Aston Hospitals respectively the admission occurred. The finding that for other than a 0 - al-month period the treatment group had significantly more re-admissions was quite unexpected. In total the treatment group had 267 re-admissions, while the control group had only 181, a difference of some 48%. It will be noticed from Table II that with time the magnitude of the difference between the two groups decreased from 152% in the 0 - 3-month period to 70% for the 22 - 24-month period*, while in the 19 - al-month period the control group had 24% more admissions. *By sub&acting the means for the 0 -2l-month period from the means for the 0 -24-month period one can obtain the means for the 22 - 24-month period.

343 TABLE

II

CUMULATIVE ADMISSIONS TO ASTON HOSPITAL FOR THE TREATMENT AND CONTROL GROUPS

DURING

THE FOLLOW-UP

PERIOD

‘P < 0.05: **P < 0.01. Period of admissions since entering

Treatment

follow-up period

Mean

S.D.

Mean

SD.

00000000-

0.469 0.786 1.035 1.228 1,421 1.579 1.690 1.841

1.155 1.542 1.801 2.006 2.403 2.650 2.910 3.086

0.186 0.414 0.579 0.766 0.910 1.014 1.159 1.246

0.527 0.925 1.211 1.395 1.590 1.732 1.895 1.991

3 months 6 months 9 months 12 montks 15 months 18 months 21 months 24 months

(R = 145)

Control (n = 145)

t

2.68** 2.49** 2.53” 2.28** 2.13-* 2.15’ 1.84 1.94*

A shortcoming of Table II is that the result may be a reflection of the low re-admission rate of the patients not interviewed and the different interviewing rates between the two groups. To examine this possibility, only patients in Western Australia during the follow-up survey (interviewed or declined to participate) have been included in Table III. The difference in the re-admission rate for the 0 -3-month period then became 191%, but as TABLE

III

CUMULATIVE ADMISSIONS TO ASTON HOSPITAL DURING THE FOLLOW-UP PERIOD FOR PATIENTS IN THE TREATMENT AND CONTROL GROUPS WHO WERE IN WESTERN AUSTRALIA DURING THE FOLLOW-UP SURVEY “P < 0 .05.* **p -c 0.01 Period of admissions since entering follow-up period

Treatment

Mean

SD.

Mean

S.D.

00000000-

0.518 0.895 1.202 1.447 1.693 1.895 2.035 2.228

1.228 1.669 1.952 2.170 2.611 2.879 3.168 3.353

0.178 0.446 0.614 0.881 1.089 1.238 1.446 1.564

0.456 0.900 1.122 1.380 1.638 1.817 2.012 2.123

3 months 6 months 9 months 12 months 15 months 18 months 21 months 24 months

(n = 114)

Control (n = 101)

t

2.74”* 2.49* * 2.74** 2.31* 2.06* 2.02’ 1.65 1.75

344

in Table II, the difference in the means between the two groups decreased with time since entering the follow-up period. In terms of their blood alcohol level on admission, duration of hospitalisation or place of discharge for their first five admissions to Aston Hospital in the follow-up period the two groups were indistinguishabic for both the all patients (Table II) and patients contacted (Table III) analyses. This was quite an important finding, for had it seemed that the treatment group patients had had more admissions because they were presenting for detoxification at an earlier stage in drinking bouts, it could have been argued that the significant difference in mean number of admissions was not necessarily a negative finding. As regards their hospitalisation in Ord Street Hospital during the followup period, in terms of number of admissions, duration of first admission, or duration of all admissions the groups were not significantly different. Consequently the significant difference in number of admissions to Aston Hospital cannot be attributed to the control group benefiting, or the treatment group being adversely affected by, the Ord Street Hospital programme. During the follow-up period 18 of the patients in the treatment group were re-admitted to Quo Vadis Hospital at least once. The 942 days of hospitalisation represented a 12.1% increase in the number of bed-days the treatment group had at Quo Vadis Hospital when they entered the study. All these patients were interviewed. Alcohol consumption and related behaviour Table IV contains details of the mean daily intake of alcohol in the week

TABLE IV FREQUENCY

DISTRIBUTION

Mean daily intake

OF MEAN DAILY ALCOHOL INTAKE! Week prior to interview

Typical Week

Treatment

Treatment

(9) Control

35 26 24 25 110

21 16 43 11 91

0 l- 120 121- 500 .501+ Total

49 26 19 16 110

Average daily intake in g/person

149.1

143.0

235.1

224.6

S.D. of mean daily intake

234.8

216.2

306.6

262.7

f

36 18 35 4 93

Control

0.19

0.26

345

prior to interview and in a typical week. In all cases there was a tendency for the treatment group to have a higher intake. With reference to Table lV it can be seen that while slightly more respondents in the treatment group were abstinent in both the week prior to interview and in a typical week, more treatment group respondents were also in the 501+ grams category. The phenomenon of the distribution of scores on a dependent variable being increased as a result of treatment, but the mean remaining the same, is not new [211 and is regarded by many researchers as a standard effect of treatment. For both the week prior to interview and in a typical week the treatment and control groups were very similar with respect to usual place of drinking on both weekdays and weekend days, type of beverage consumed, and number of weekdays and weekend days on which alcohol was consumed. However, respondents in the control group were significantly more likely to drink in company. The two groups were also compared on a number of variables for alcohol related behaviour. Respondents in the treatment group were significantly more likely to have had lapses or blackouts in the week prior to interview, a not unexpected finding in view of the difference in the number of persons in the 501 or more grams of alcohol daily category in Table Iv. No other alcohol related variable differentiated between the two groups (Table VJ. As part of the interview respondents were asked to describe their present drinking pattern and what they intended to do with regard to their drinking during the next few months. Respondents in both groups gave a similar description of their present drinking pattern, but significantly more of the treatment group respondents regarded their drinking problem as serious rather than moderate. Such a finding is of course consistent with the alcohol intake figures in Table IV. When asked about their intentions for the next few months, the treatment group respondents showed that they were significantly more likely than the control group respondents to know what they should be doing even if they were not doing it. Other outcome variables At the time of interview there were no significant differences between the two groups with respect to type of residence, type of tennant arrangements, regular or temporary residence, or number of different places lived in during the follow-up period. With respect to years since became separated, widowed or divorced, ever been married or current marital status, the two groups were also very similar. Highest level of education and type of vocational training also failed to discriminate between the two groups. The respondents in the two groups were compared on six variables relating to employment history during the follow-up period. As a result of participation in the Quo Vadis Hospital work re-training programme the employment pro&e&s of the treatment group were apparently not enhanced (Table IQ.

346 TABLE

V

COMPARISGN OF THE COME VARIARLES Figures in brackets

TREATMENT

AND

CONTROL

GROUPS

ON SELECTED

OUT-

next to the means are S.D. *P < 0.05; **P < 0.01; ***P < 0.001. Treatment

Control

No. of days worked in 30 days prior to interview

6.18 (11.20) (N = 111)

8.4lf13.09) (N = 93)

No. of jobs in the follow-up

1.39 (1.87) (N = 112) 4.94 (5.63) (N = 111)

0.94 (1.39) (N = 92) 4.41(5.66) (N = 93)

23 0 81

30 2 61

57 54

51 42

57 54 0.3lCO.85) (N = 112)

38 55 0.27 (0.55) (N = 93)

26 10 31 9 20 0.13 (0.62) (N = 112) 0.39 (1.56) (N = 109) 1.30 (1.94) (N = 109) 0.09 (0.37) (N = 111) 1.62 (5.63) (N = 110) 6.37 (14.66) (N = 110) 20.57 (45.10) (N = 112) 0.45 (0.67) (N = 112) 6.22 (2.52) (N = 107)

18 10 29 19 14 0.26 (0.97) (N = 93) 0.44 (1.54) (N = 92) 1.07 (1.83) (N = 92) 0.20 (0.56) (N = 92) 2.79 (10.88) (N = 93) 15.09 (26.63)** * (N = 93) 20.99 (46.44) (N=94) 0.26 (0.62)* (N = 94) 5.99 (2.56) (N = 87)

period

Months employed during the follow-up period Type of employment at time of interview Full-time Part-time Not employed Attended AA before the study Yes No Attended AA in the follow-up period Yes No No. of clubs, societies and organizations a member of Frequency of sporting activities or hobbies Every day 2 - 3 times per week Once a week Less than once a week None Times psychiatric inpatient Times psychiatric

outpatient

No. of admissions to a public hospital No. of admissions to a private hospital Attendances at hospital emergency departments G. P. consultations Total no. of times visited Voluntary Agencies No. of Agencies going to at time of interview Self-esteem score

347 TABLE

V (Co&d.)

BAL at time of interview BAL at time of interview for persons not interviewed in hospital or prison Dollars spent on alcohol in a typical week Week prior to in&-view Times missed a meal because of drinking Times had a drink as soon as woke up Times quarrelled while drinking Times drunk No. of lapses or blackouts Times had the ‘shakes’ Longest period without a drink in days No. of drink-driving convictions No. of convictions for dnmkeness and disorderly conduct

Treatment

Control

0.031% (0.069) (N = 91)

0.060% t0.088) (N = 70)

0.031% (0.070) (N = 62) 39.0 (48.9) (N = 112)

0.068% (0.090) (N = 55) 41.0 (40.4) (N = 85)

7.07 (9.24) (N = 109) 2.72 (3.62) (N = 108) 1.05 (2.32) (N = 109) 2.44 (3.38) (N = 109)

5.59 (8.11) (N = 94) 2.19 (3.31) (N = 93) 0.70 (1.94) (N = 93) 1.77 (3.06) (N = 91) 0.64 (1.76)” (N = 88) 1.93 (3.21) (N = 93) 3.36 (3.02) (N=84) 0.05 (0.23) tN = 91) 0.84 (1.87) (N = 93)

1.27 (2.37) (N = 101) 2.32 (3.39) (N = 108) 3.00 (2.80) (N = 82) 0.12 (0.42) (N = 112) 0.61(1.67) (N = 111)

Approximately the same proportions of respondents in both groups had attended AA before and after inclusion in the study. Men who attented AA in the after period were likely to have had a prior AA history. The two groups were not significantly different with respect to number of clubs, societies and organizations a member of, frequency of sporting activities or hobbies, company while participating in recreational activities, car ownership or possession of a driver’s licence. The two groups were very similar with respect to their usage of a wide range of health care facilities during the follow-up period and as to whether they saw their psychiatric and hospital admissions as related to drinking. In the alcohol treatment field it is virt_ually impossible to have a no treatment control group due to the large number of voluntary agencies from which a person may seek treatment. It was, therefore, necessary to monitor the extent to which persons in the treatment and control groups received such treatment to ensure that no confounding of the results occurred. At

time of interview respondents in the treatment group had greater usage of the services provided by voluntary agencies, but in terms of the total number of times respondents in both groups had visited voluntary agencies during the follow-up period there was no difference. Consistent with the hypothesis that the Quo Vadis Hospital programme was not achieving its goals, the self-esteem scale failed to discriminate between the two groups. At time of interview the treatment group respondents had significantly lower blood alcohol levels. However, as a greater proportion of the treatment group were interviewed in hospital or prison, where one would expect to obtain a zero reading, the comparison appeared to be invalid. When readings taken from respondents in hospitals and prisons were excluded, the treatment group still had a significantly lower mean blood alcohol level at time of interview, but as a result of the exclusion the value of the analysis was again open to question. For instance, of the 44 respondents excluded some 33 (65%) had a zero reading, and approximately two-thirds of the respondents excluded were in the treatment group. Duration of treatment group hospitalisation at Quo Vadis Hospital As some of the patients in the treatment group had relatively short periods of hospitalisation at Quo Vadis Hospital, it could be argued that it was unrealistic to expect the programme to benefit them and that the above negative findings were at least partly due to patients not completing the twelve week programme. The 11Bpatients in the treatment group who were contacted during the follow-up survey were therefore classified as to whether they spent more or less than 60 days at Quo Vadis Hospital. The short stay group had 63 patients, while the long stay group had 51 patients. Comparison of the two sub-groups indicated that on 43 out of the 44 variables recorded in the Authority’s records there were no significant differences on pre-follow-up period data. No support was found for the hypothesis that patients’ premature departure from Quo Vadis Hospital may have contributed to the negative results obtained. Indeed, quite the reverse was the case as the long stay group had, on the average, 24% more readmissions to Aston Hospital (mean 2.49, S.D. 4.05 compared to mean 2.02, S.D. 2.68). Most of this difference can be attributed to the considerably higher rate of admission during the three months following discharge from Quo Vadis Hospital (mean 0.87, S.D. 1.68 compared to mean 0.38, S.D. 0.66). However, neither of these differences was significant. Characteristics of patients who ‘improved’ It was anticipated that the overall result for the evaluative study would be positive, but that a small number of patients would have made a disproportionate contribution [2]. The implication of such a result would be that the programme was benefiting some patients and, were entry to be restricted to such patients, the overall effectiveness of the programme would

be increased perhaps considerably. A comparison was made of patients contacted in the treatment group who improved or did not improve. The same analyses were also conducted for the control group to identify the characteristics of patients who would have improved without participation in the Quo Vadis Hospital programme. As all patients had been admitted to Aston Hospital immediately prior to inclusion in the study, for the purpose of the comparisons ‘improvement’ was defined as having no admissions to Aston Hospital during the follow-up period. For both the treatment and control groups the number of admissions to Aston Hospital prior to entering the study consistently discriminated between persons who were or were not re-admitted. To a lesser extent, number of prior admissions to Ord Street Hospital also discriminated in a similar manner. No other variable in the Authority’s records yielded any useful information. Discriminant function analyses confirmed the findings from the t-tests and x2&e&s. It was concluded that using admissions to Aston Hospital as the index of improvement, no particular group of patients was likely to benefit from the Quo Vadis Hospital programme. Patients excluded from the treatment group During 1978 and 1979 a total of 55 male; non-aboriginal alcohol dependent patients were first admitted to Quo Vadis Hospital, but were excluded from the treatment group as they had not been discharged from Aston Hospital within the seven days prior to their admission. With a view to determining whether the above results were only applicable to patients admitted to Quo Vadis Hospital within one week of detoxification, the 55 patients were compared to the patients in the treatment and control groups on both background and official records follow-up data. It was concluded that in terms of background data the 55 patients were very similar to the patients in the treatment and control groups, but that on official records follow-up data the excluded group was like the treatment rather than the control group. The findings of this evaluative study were apparently not restricted to patients admitted to Quo Vadis Hospital within seven days of detoxification. DISCUSSION

Sobell and Sobell 1221reviewed the methodological issues which should be addressed in alcohol treatment outcome evaluation. They then ascertained the extent to which 37 peer-reviewed publications of alcohol treatment outcome evaluations that appeared in the literature during the period from 1976 to early 1980 fulfilled these criteria. The reviewers noted that their paper had a selection of studies biased towards the well designed outcome evaluation. Of the 50 items considered to be important, the 37 studies reviewed satisfied, on the average, 49%. By contrast, the Quo Vadis Hospital evaluation study satisfied 78%.

350

Although considerable effort was taken to obtain a control group of patients very similar to the treatment group, the selection procedure is not beyond criticism. Attention was drawn above to a number of differences between the treatment and control groups on information recorded in the Authority’s records. Differences on most of these variables were apparently of little or no importance, as indicated by previous research and the comparisons of the improved and not improved patients. However, on the variables relating to prior treatment at Aston and Ord Street Hospitals, the differences were clearly of importance. Furthermore, it seems that for at least the February - September, 1979 period stall at Aston Hospital encouraged the Better’ patients to go to Quo Vadis Hospital. It appears that the outcome data for the control group should be viewed as a minimum statement of the extent to which the treatment group could have been expected to improve without participation in the Quo Vadis Hospital programme. A reviewer of the original report suggested that some of the increased re-admission rate of the treatment group may have been due to a ‘dependency’ factor. Those people who accepted referral to Quo Vadis Hospital from Aston Hospital could have been a more dependent group, more inclined to leave their problem to be managed and dealt with by other people. This dependency could also have contributed to the development of their dependence on alcohol. However, the fact that the two groups were matched on age, yet patients in the control group had significantly more admissions to Ord Street Hospital and 9% more admissions to Aston Hospital before entering the study, is inconsistent with a ‘dependency’ factor explanation of the negative results. Contrary to the suggestion of Sobell et al. 1231.a number of respondents were interviewed when they had positive blood alcohol levels. For the 30 respondents with a blood alcohol level of 0.001- 0.100% the accuracy of their self-reports was probably not adversely aifected. Consideration was given to re-interviewing the 30 respondents with a blood alcohol level of 0.101% or more, but was decided against as the probability of these respondents having a lower blood alcohol level at time of re-interview was close to zero due to their often very high daily levels of alcohol intake. Exclusion of the data for these respondents would have resulted in the results analysis being performed on a biased sample. Even after allowing for the methodological shortcomings noted above, the findings indicate that the rehabilitation programme operating at Quo Vadis Hospital during 1978 and 1979 was not achieving its goals. For a large number of variables covering a wide range of outcome areas no significant differences were found between the treatment and control groups. Furthermore, as in many cases the difference was in favour of the control group, the point cannot be made that had more patients been included in the study, the results might have been statistically significant. To the extent that patients in the treatment group were significantly more likely to

351

be re-admitted to Aston Hospital during the follow-up period, it also appears that the Quo Vadis programme was having an unintentional deleterious effect on some patients. Due to limited resources, during 1978 and 1979 entry to the Quo Vadis Hospital programme was preceded by only a basic assessment and there was little opportunity for patients to participate in activities appropriate to the special needs of individual patients. Re-entry and after-care services were also restricted for the same reason. Consequently, considerable caution should be exercised in applying the findings of the study to other programmes, especially those in which clients are given a detailed assessment and are matched’ to a treatment programme, either inpatient or outpatient, abstinence or controlled drinking oriented, relevant to their individual needs and which contains both re-entry and aftercare components. The results of this study are in accordance with the findings of evaluative studies conducted in other countries. In a number of reviews [1,21 the ineffectiveness of overseas alcohol treatment programmes has been commented on and there is even a precedent for negative findings [241. The outcome of the Quo Vadis Hospital study highlights the importance of evaluation research and the need for such research to be an integral part of all alcohol treatment programmes. ACKNOWLEDGEMENT

This study was supported by a grant from the TVW Telethon Foundation of Western Australia. REFERENCES

1 D.J. Armor, J.M. Polich and H.B. Stambul, Alcoholism and Treatment, Rand Corporation, Santa Monica, 1976. 2 C.D. Emrick, J. Stud. Alcohol., 36 (1975) 66. 3 M.A. Schuckit and D. Cahalan, in: W.J. Filstead, J.J. Rossi and M. Keller fEds.1, Alcohol and Alcohol Problems: New Thinking and New Directions, Ballinger, Cambridge, Massachusetts, 1976. 4 F. Baekeland, L. Lundwall and B. Kissen, in: R.J. Gibbin et al. @Us.), Research Advances in Alcohol and Drug Problems, Vol. 2, Wiley, New York, 1975. 5 H.M. Annis and C.B. Liban, J. Stud. Alcohol., 40 (1979) 63. 6 D.T. Campbell and J.C. Stanley, Experimental and Quasi-experimental Designs for Research, Rand McNally, Chicago, 1976. 7 M.B. Sobell and L.C. Sobell, Behav. Res. Ther., 36 (1975) 66. 8 D. Black, J. R. Coll. Physicians London, 13 (1979) 57. 9 Leading Article, Br. Med. J., 2 (1979) 1244. 10 L. Cranberg, Br. Med. J., 2 (1979) 1265. 11 G.I. Athey and L. Coyne, Alcoholism, 3 (1979) 341. 12 E. Gottheil, in: M. Galanter (Ed.), Currents in Alcoholism, Vol. VII. Recent Advances in Research and Treatment, Grune and Stratton, New York, 1979. 13 A.A. CongaRon, Occupational Status in Australia, Studies in Sociology, No. 3, The University of N.S.W., 1967.

352

14 N.H. Nie et al., Statistical Package for the Social Sciences, 2nd edn., McGraw-Hill, New York, 1975. 15 R. Holland, R.C. Eveneon and B. Izadi, J. Stud. Alcohol., 40 (1979) 914. 16 M. Rosenberg, Society and the Adolescent Self Image, Princeton University Press, New Jersey, 1965. 17 K.D. Cheralampous, B.K. Ford and T.J. Skinner, J. Stud. Alcohol., 37 (1976) 990. 18 Australian Associated Brewers, Comm. Hlth. St., 2 (1978) 120. 19 A.C. Ogborne, Acta Psychiat. &and., Suppl. 284,62 (1980) 66. 20 R. Moos and F. Bliss, J. Stud. Alcohol., 39 (1978) 473. 21 F.B. Glaaer, in: G. Edwards and M. Grant (F&I.), Alcoholism Treatment in Transition, Croom Helm, London, 1980. 22 L.C. Sobell and M.B. Sobell, Alcohol Treatment Outcome Evaluation. Substudy No. 1132, Alcoholism and Drug Addiction Research Foundation, Toronto, 1980. 23 M.B. Sobell, L.C. Sobell and R. Vanderspek, J. Consult. Clin. Peychol., 47 (1979) 204. 24 A.C. Ogbome and G. Clare, Br. J. Addict., 74 (1979) 283.