- Email: [email protected]
Effectiveness of cognitive therapy training
Journal of Behavior Therapy and Experimental Psychiatry 30 (1999) 81}92
E!ectiveness of cognitive therapy training Derek L. Milne *, Chris Baker, Ivy-Marie Blackburn, Ian James, Katharina Reichelt Centre for Applied Psychology, University of Newcastle upon Tyne, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK Northumberland Mental Health NHS Trust, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK Newcastle City Health Trust, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK
Abstract Despite growing demands for e!ective training there remains a lack of convincing evaluations. Reviews of the training literature specify the need for multiple measures, homogeneous therapies, generalization assessments and longitudinal designs. The present analysis responds by evaluating the training of 20 mental health professionals in cognitive therapy (CT). The evaluation focussed on changes in the professionals' competence and its generalization to their patients' coping strategies (N"20). Results indicated that a 40 day training programme led to signi"cantly higher post-training competence scores and to improved patient coping. The implications for professional training and its evaluation are discussed. 1999 Elsevier Science Ltd. All rights reserved.
1. E4ectiveness of training in cognitive therapy Evidence-based practice has been described as &the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients' (Sacket, Richardson, Rosenberg, & Haynes, 1997). The growing demand for managed care, evidence-based practice and short training courses (Roth & Fonagy, 1996) means that the need for carefully designed psychotherapy training programmes, such as cognitive therapy (CT), has increased latterly (Freiheit & Overholser, 1997). Therapy training courses must teach trainee therapists how to integrate their
* Corresponding author. Tel.: #0191-2226000 Ext. 7912. E-mail address: [email protected] (D.L. Milne) 0005-7916/99/$ - see front matter 1999 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 7 9 1 6 ( 9 9 ) 0 0 0 1 1 - 7
82
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
individual expertise with the best available clinical evidence from systematic research by: converting clinical issues arising from individual patients into answerable questions, "nding the best evidence with which to answer these, appraising the evidence for its validity and usefulness in the speci"c clinical context, applying the results of the appraisal in their clinical work, and evaluating the outcome. In addition, there is a need to demonstrate that such training programmes are e!ective, regardless of the speci"c therapy being taught, in order to ensure good use of scarce sta! resources. The evidence available is yet to provide convincing data on the e!ectiveness of training courses in psychotherapy generally (Bootzin & Ruggill, 1988; Edelstein, 1985; Beutler & Kendall, 1995). In particular, there is a need for: (1) more re"ned approaches to measurement, in particular the inclusion of expert raters and multiple measures (Alberts & Edelstein, 1990); (2) for su$cient training to develop measurable competence in newly learned techniques (Binder, 1993); (3) a clear focus on an homogenous therapy and the measurement of the consequences of competence in terms of patient outcome (Beutler & Kendall, 1995); and (4) for longitudinal analyses which examine these outcomes within experimental or quasi-experimental designs (Ellis, Ladany, Krengel, & Schult, 1996). CT is a suitable candidate for this re"ned evaluation of the e!ectiveness of training. It is an explicitly empirical approach in which adherence to therapy protocols and assessing the outcome of therapy are integral and it is well supported by the evidence on clinical outcome (Beck, 1991; Roth & Fonagy, 1996). The e!ectiveness of cognitive therapy training and supervision has not been addressed adequately in the cognitive therapy literature. Shaw and Wilson-Smith (1988) describe an intensive training and monitoring programme with the aims of developing therapists' skills in adhering to a cognitive therapy research protocol in the treatment of patients presenting with depression. This study assessed the application of CT over time. Theoretical and practical aspects of the approach were taught, demonstrated and practised, and monitoring and case supervision continued for up to 24 months. Independent expert raters used the cognitive therapy scale (CTS: Young & Beck, 1980, 1988) which yielded signi"cant training e!ects over time on the overall CTS scores, on the conceptualisation sub-scale, and on scores of the structure of CT (i.e. agenda setting, pacing, and homework). Unfortunately, the impact of the progress of the training was not measured in relation to the patients involved in this project. Frank, Kupfer, Wagner, McEachran, and Cornes (1991) studied long-term maintenance therapy for depression and found that those patients who rated their therapists as competent had longer periods of bene"t from therapy than those patients who rated their therapists as providing poor quality therapy (two years compared to four months, respectively). Williams, Moorey, and Cobb (1991) examined the impact of a training course on competence in CT by rating participants' audio-taped CT sessions at the beginning and the end of the course. Two raters, blind as to the order of the tapes, scored each tape using a shortened form of the CTS. Eight out of 11 trainees showed improvements on the CTS, but a comparison of mean scores failed to reach statistical signi"cance. Again, the impact of competence on patient outcome was not evaluated in this pilot study.
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
83
Kingdon, Tyrer, Seivewright, Ferguson, and Murphy (1996) re-analysed the data from a prior and negative clinical outcome evaluation of CT in terms of the supervisors' perceptions of the competence of the eleven therapists. This sub-analysis indicated that the competent therapists (N"4) obtained better clinical outcomes than those judged to be of &uncertain' or &doubtful' competence (N"7). The authors concluded that competence should be measured objectively and that the CT should be of greater than four sessions duration. More recently, Freiheit and Overholser (1997) examined the impact of therapists' variables, including theoretical backgrounds and gains made in CT training. Forty clinical psychologists participated in a year long training course, involving both teaching and supervised application of CT. Trainees' knowledge, attitude to CT and application of behavioural and cognitive techniques were measured before and after the training. Statistically signi"cant gains were reported in trainees' knowledge, positive appraisal of CT and CT ideology, and the practice of behavioural and cognitive techniques at the end of the course. There was also a signi"cant reduction in negative appraisal of CT, but no di!erences between trainees of di!erent theoretical orientation. However, the study relied on self report data rather than employing independent raters, and once more the impact on patients was not evaluated. In addition to these training evaluations, useful reports have been published about teaching and supervision in CT, but many are descriptive, raising awareness of issues, rather than reporting empirical data (e.g. Perris, 1993). For example, Padesky (1996) pointed out that teaching in CT in the 1990s had become increasingly demanding, with speci"c conceptualisations and treatment approaches being developed for a wide range of psychiatric disorders. She provided a helpful summary of considerations in the teaching and supervision process in CT training. Finally, Perris (1993) drew attention to the limited published work that addresses supervisor training and supervision processes in cognitive therapy, and went on to examine some of the stumbling blocks. In summary, the literature on the e!ectiveness of CT training is limited and relatively unre"ned, yet the available evidence supports the e!ectiveness of CT training in improving the competence of trainees. Furthermore, reports of the contents of such training programmes indicate that theoretical and didactic components can be combined with clinical supervision to improve trainees' knowledge and skills, in line with the development of evidence-based practice. More re"ned research, designed to evaluate the training of competences of CT and the contribution of such competences to the outcome of therapy would provide valuable information to trainers and supervisors. The purpose of the present study is to de"ne and describe a regional training programme in CT and to evaluate its impact on the competence of the mental health professionals undertaking the training within a longitudinal design. The most distinctive feature of this study is the relationship of competence to the clinical outcomes achieved by patients, perhaps the best test of competence (Roth & Fonagy, 1996). This is the "rst such in-depth evaluation of training in the UK (Williams, Moorey, & Cobb, 1991), referred to their own work as a pilot study) and the "rst to use an improved measure of competence (the CTS-R: (Blackburn, Milne,
84
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
& James, 1997). It was predicted that the training program would result in signi"cantly raised competence amongst the trainee cognitive therapists, which would in turn generalise to signi"cant improvements in the clinical status of their patients.
2. Methods 2.1. Training programme The training programme in CT was run in a well-established centre in the UK, the Newcastle Cognitive Therapy Centre. The course consisted of an intensive induction week, followed by 35 training days spread over three university terms (nine month duration). Each day consisted of three hours of large group didactic instruction by presenters of national and international repute, two hours of clinical work and one and a half to two hours of supervision in pairs. Supervisors were either experienced cognitive therapists in NHS settings who had attended a cognitive therapy supervisors course or were members of the specialist Cognitive Therapy Centre. All attended regular supervisor training meetings to maintain supervisory skills. Trainees routinely videotaped their therapy sessions and these tapes were reviewed by supervisors and used in supervision. Supervisors were encouraged to emphasise the development of cognitive formulations, linking structure, process and theoretical models in therapy. It was expected that supervisory strategies such as role-play and modelling would feature in supervision sessions. Supervisors had, in previous years, been expected to submit an audiotape or videotape of a supervision session on two occasions, in order that the quality of supervision be examined by course organisers. Trainees were formally evaluated through the course on the competence ratings of two therapy videotapes submitted in the second and third terms, as well as through two essays and two case studies. 2.2. Therapists Entry to the programme was by open competition amongst professionally quali"ed mental health sta! from the UK. The present analysis focuses on one intake of 20 professionals, consisting of clinical psychologists (N"6), psychiatrists (N"6) and mental health nurses (N"8). There were 11 females and 9 males in this sample, with an average age of 39.0 years (SD 7.0). Most were Caucasian, with two representatives of two ethnic minorities. 2.3. Patients The patients (N"20) had an average age of 37.7 (SD 13.2) with a range of 19}70 years of age. Eleven were female and nine were male. Thirteen were married or cohabiting and seven were single, separated or divorced. All su!ered from an Axis I disorder, usually depression or an anxiety disorder. Patients were selected by supervisors from routine referrals to the Newcastle Cognitive Therapy Centre
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
85
(a tertiary referral centre), having already been screened by members of the centre, or were selected from supervisors' NHS waiting lists. Patients provided informed consent to be seen by cognitive therapists in training and for video recordings of their therapy and psychometric data to be used in this study. 2.4. Raters There were four raters. All were trained supervisors for the training course and were considered to be experts in CT and in the use of the CTS-R.
3. Methods and procedure 3.1. Measurement of competence The CTS-R is based on the original CTS (Young & Beck, 1980, 1988) and was developed by experienced cognitive therapists and researchers (Blackburn, Milne, & James, 1997; Milne, Blackburn, & James, 1997). The revised CTS has 14 items retaining items of the original scale and adding additional items: (1) Agenda setting, (2) Eliciting feedback (emphasising two way feedback), (3) Collaboration, (4) Pacing and e$cient use of time, (5) Interpersonal e!ectiveness (incorporating Empathy and Professionalism), (6) Facilitation of emotional expression (new item), (7) Charisma and #air factor (new item), (8) Guided discovery, (9) Conceptualisation, (10) Key cognitions and emotions, (11) Application of cognitive techniques, (12) Application of behavioural techniques, (13) Use of homework, (14) Therapists non-verbal behaviour (new item). The 14th CTS-R item was not included in the present analysis. The items are carefully de"ned and rated on a revised and detailed six point competency scale. This unipolar Likert Scale extends from &incompetent' (level 0) to &expert' (level 5). The CTS-R has been shown to have adequate inter-rater reliability and high internal consistency (Blackburn et al., 1997). The competence of the CT trainees was assessed on the CTS-R at three points during a 12 week period of therapy by means of video tape recordings made in the early stages (sessions 1}4), in the middle (sessions 5}8), and towards the end of the therapy (sessions 9}12). The video tapes were independently double rated by the four expert raters who were blind to the sessions sampled. Competency was measured as the average of the two scores awarded to each of these three recordings. 3.2. Measurement of benexts to patients (generalisation) A general measure of clinical outcome was used to assess impact of CT (the coping responses inventory, CRI: Moos, 1990). The CRI is a measure of an individual's personal coping strategies and has been regarded as the fundamental mediator of therapeutic change (Moos, 1976; Lazarus & Folkman, 1984). It has eight dimensions, four addressing approach-based coping (logical analysis, positive appraisal, seeking guidance and taking problem solving action) and four covering avoidance-based
86
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
coping (cognitive avoidance, resignation, seeking alternative rewards and emotional discharge). The dimensions can also be broken down into cognitive (logical analysis, positive appraisal, cognitive avoidance, and acceptance) and behavioural coping strategies (support seeking, problem-solving, alternative rewards, emotional discharge). In general, the four approach based coping strategies tend to be more adaptive for patients and so therapists will typically try to enhance these, while encouraging their patients to decrease their use of avoidance-based coping strategies (Milne, 1992). Each of the CRI's 48 statements is evaluated on a four point frequency scale, ranging from &never' to &fairly often' and is related to a stressor described by the patient (e.g. unemployment). An example of an approach-based cognitive coping strategy is &think of di!erent ways to deal with the problem' (from the logical analysis dimension), while &keep away from people in general' is an example of an avoidancebased behavioural coping strategy (from the emotional discharge dimension). The CRI was administered at the beginning and end of treatment.
4. Design The research design was quasi experimental, utilising direct observations and self-report instruments before, during and after training (for the CT trainees) and therapy (for the patients). Although this was a longitudinal design with respect to the training phase, it did not include a post-training follow-up assessment point. Hypotheses: It was predicted that the training course would improve the therapists' competence leading to better coping pro"le for the patients. In particular, therapists were expected to improve most in terms of the speci"c cognitive therapy competencies (i.e. CTS-R items 8}13 } see Table 2), while the patients were expected to have higher CRI frequencies of approach coping strategies, and lower frequencies of the avoidance strategies.
5. Results 5.1. Change in competence following training Following training in CT, there was a highly signi"cant increase in overall therapist competence, as measured by expert raters on the CTS-R. (Repeated one-way ANOVA: F (2,38)"4.24, p(0.02.) Table 1 summarises this "nding, which represents an overall improvement in competence from &advanced beginner' to &competent' on the competence scale. The breakdown of the changes in CTS-R items between the "rst and last assessment point are presented in Table 2, indicating that there were signi"cant improvements in the therapists competence in eight of the 13 CTS-R subscales. Highly signi"cant changes (i.e. p(0.01) were obtained for the subscales &interpersonal e!ectiveness', &facilitation of emotional expression', &charisma and #air', &guided discovery', &conceptualisation' and &application of cognitive techniques'. Signi"cant changes were also
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
87
Table 1 E!ect of training on therapist competence (N"20), as measured by the cognitive therapy scale } revised (13-items } item 14 rating non-verbal behaviour excluded)
Mean total score SD Range
Time 1 Sessions 1}4
Time 2 Sessions 5}8
Time 3 Sessions 9}12
30.3 8.2 19.3}44.3
33.1 7.2 20.5}45.0
37.0 8.1 23.0}53.5
Table 2 Mean competence scores (SDs) on individual items of the CTS-R at the "rst and "nal assessment phases for the patients seen from the start of training (N"20) Items
1 2 3 4 5 6 7 8 9 10 11 12 13
Agenda setting Eliciting feedback Collaboration Pacing Interpersonal e!ectiveness Facilitation of emotional expression Charisma and #air Guided discovery Conceptualisation Focus on key cognitions Application of cognitive technique Application of behavioural techniques Use of homework
Beginning of treatment
End of treatment
¹
p
2.7 2.7 2.9 2.7 3.0 2.3 2.5 2.5 2.2 2.5 2.0 1.7
2.9 2.9 3.4 2.9 3.6 2.8 3.2 3.1 2.8 2.9 2.7 2.4
(1.0) (0.7) (0.8) (0.8) (0.9) (0.9) (1.0) (1.0) (0.8) (0.8) (0.8) (1.0)
0.94 1.19 2.51 0.97 2.6 2.58 2.87 2.79 3.00 1.68 3.21 1.95
NS NS p(0.05 NS p(0.01 p(0.01 p(0.005 p(0.01 p(0.005 NS p(0.005 p(0.05
3.1 (0.7)
0.92
NS
(1.0) (0.8) (0.9) (0.7) (0.9) (0.7) (1.0) (0.8) (0.7) (0.7) (0.8) (1.2)
3.0 (0.7)
obtained for &collaboration' and &application of behavioural techniques'. It should be noted, therefore, amongst these there were signi"cant improvements in three of the six speci"c CT techniques (i.e. CTS-R items 8, 9 and 11). As the inter-rater reliability and internal consistency of the CTS-R were high (Blackburn et al., 1997) and raters were blind to phase of training, these "ndings probably re#ect a real e!ect of training. As predicted, the results suggest that the training course was led to improved competence in trainees, and provide some more detailed information about those competencies that were most improved by training. 5.2. Changes in patients' coping strategies (generalisation) In terms of clinical outcome, it is evident that, as a group, the patients improved their coping pro"le to a signi"cant degree. Table 3 summarises the mean scores
88
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
Table 3 Comparison of patients' baseline and outcome scores (Coping response inventory; N"20) Coping response
Baseline
Outcome
P"
x
SD
x
SD
Approach coping strategies
Logical analysis Positive appraisal Problem solving Support seeking
45.3 41.8 41.6 46.8
11.6 9.8 11.7 11.3
47.5 44.7 45.4 49.1
9.5 8.8 11.0 8.8
NS 0.03 0.05 NS
Avoidance coping strategies
Cognitive avoidance Acceptance Alternative reward Emotional discharge
53.6 52.8 49.5 59.0
8.6 10.2 9.4 9.1
50.6 50.4 47.0 55.1
8.4 10.1 7.6 8.6
0.07 NS NS 0.02
obtained on the CRI before and after receiving cognitive therapy, indicating signi"cantly higher use of positive appraisal and problem solving, with signi"cantly less frequent use and emotional discharge. This represents a much-improved coping pro"le, two of the four speci"c predictions being upheld statistically (increased positive appraisal and problem solving). A third predicted change, decrease in cognitive avoidance, showed a trend towards signi"cance (p"0.07). These "ndings and the consistent, desirable e!ects across all the other CRI subscales indicate that CT had as marked an impact on behavioural and a!ective responding as it had on the patients' cognitions. In summary, these "ndings indicate that the 20 CT trainees became more competent following the training programme as measured by the CTS (R). This appeared to be associated with the provision of e!ective therapy to the 20 clients (CRI).
6. Discussion 6.1. Main xndings The results of this study indicate that a carefully designed training programme signi"cantly improved competence in cognitive therapy in the group of 20 therapists, raising it from the &advanced beginner' level to that of &competent'. Experts blind to the phase of training independently rated tapes from the beginning, middle and end of therapy, giving signi"cantly higher CTS-R ratings to the therapy provided at the end of training. The breakdown of the CTS-R into its components o!ers valuable insight into the e!ectiveness of the course in training therapists in adherence to speci"ed therapeutic procedures and in skilfulness in the application of such procedures. It is satisfying to note that there were improvements in therapists' collaborative style and interpersonal e!ectiveness in therapy sessions, and increased adherence and skill in the use of guided discovery and cognitive techniques, and in the incorporation of
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
89
behavioural techniques. These "ndings suggest that therapists can develop key competences in CT, the dominant form of evidence-based practice (Roth & Fonagy, 1996), strengthening the evidence from earlier research (Shaw & Wilson, 1988; Williams et al., 1991; Freiheit & Overholser, 1997). An important feature of the present study was to demonstrate that delivery of competent cognitive therapy was mirrored in general improvements in the patients' coping repertoires. The improved coping was not restricted to changes in the CRI's cognitive coping strategies, but rather was evenly distributed across cognitive, behavioural and a!ective responding. This indicates that successful cognitive therapy brings changes in the patients' maladaptive thinking patterns as well as to their behavioural and emotional expression. As expected, competent CT was associated with a trend of increased approach and decreased avoidance coping strategies, a "nding that is in keeping with both the transactional stress theory (Lazarus & Folkman, 1984; Moos, 1976) and previous clinical research (e.g. Milne & Souter, 1988). Taken together, these "ndings suggest that relatively short training courses in evidence-based practice can be e!ective, and provide data that have been sadly wanting (Bootzin & Ruggill, 1988; Beutler & Kendall, 1995). Most therapists in this study were reaching competent levels of ability in all areas measured by the CTS-R, except in the use of behavioural techniques. In this particular item there was a signi"cant gain, but a low starting point. This is re#ected in qualitative information from the raters, suggesting that the therapists relied too heavily on a limited number of verbal restructuring techniques, in particular on reviewing evidence for and against negative automatic thoughts and the use of alternative, less dysfunctional responses. The relative lack of behavioural methods was of particular concern. Wells (1997) has demonstrated the importance of such methods in eliciting patients' experience, socialising patients to the CT model, reattributing unhelpful cognitions, and modifying a!ect. In terms of this feature, the present therapists barely escaped from the novice categorisation by the end of the course. This is an area for increased emphasis in this training programme, perhaps particularly in supervision, along with extending trainees' repertoire and skills in evaluation techniques. The items concerning therapeutic structure, namely agenda setting, pacing, setting homework and seeking feedback showed no signi"cant gain. However, the initial scores for these items revealed reasonable levels of competence from the outset. A possible explanation for this "nding is that the training provided during the week long intensive induction course was su$cient to enable trainees to adhere to an adequate CT structure for a therapy session. As a consequence, the competence ratings from the "rst videotapes may have re#ected a training outcome, rather than a true baseline value for competence in these aspects of CT. 6.2. Methodological critique While these data represent the most systematic evaluation of CT training to date, there are nonetheless ways in which the study could have been strengthened. Firstly, future research should measure the training itself, providing a manipulation check
90
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
analogous to the competence check carried out for therapy (Waltz, Addis, Koesner, & Jacobson, 1993). Prior research has shown how a training intervention, in the same way as therapy, can be operationalized in terms of type (e.g. experiential or didactic), pro"ciency (e.g. &novice' or &master') and interpersonal e!ectiveness (Ashworth & Milne, 1996). Just as in therapy, variations in these dimensions of the training intervention may lead to signi"cantly di!erent learning outcomes. Similarly, careful assessment of these training variables over time can provide informational feedback, a powerful basis for improving the competence of the trainers. For example, careful assessment, feedback and participative decision-making on such aspects as the involvement of the participants or the match between the training method and learning styles can optimise the e!ectiveness of training (Milne & Noone, 1996). A second major weakness of the present study was its quasi-experimental research design. Future research could strengthen our pre-post design by adding a control condition, in routine practice perhaps, best achieved by implementing a double baseline (waiting list) assessment for both trainees and patients. This would enable greater con"dence in drawing conclusions about the causal relationship between therapists' competency and therapy outcome. As things stand, we cannot exclude the possibility that it was improvements in the patients that led to enhanced competence in the trainees. A further re"nement would be to extend our generalisation analysis to include evaluation of the therapists' learned skills to their routine work with patients after training. This could also cover generalisation across time (a post-training follow-up assessment), across settings (i.e. to the trainee's normal work environment) and possibly also across behaviours (i.e. do the trainees extend their use of the trained skills so as to develop novel variants of the therapy or to address new clinical presentations?). Analysis of these forms of generalisation have typically indicated major limitations to training (Milne, 1986) and a concomitant need to focus on the organisational factors that can powerfully in#uence generalisation (Corrigan & McCracken, 1997). These are important topics for future evaluations of CT training.
7. Conclusions A 40 day training course in cognitive therapy appeared to result in signi"cantly improved competence amongst 20 cognitive therapy trainees and this was associated with improvement in coping in 20 patients. This kind of generalisation assessment is both the scienti"c criterion for successful therapist training and essential for a demonstration that training in evidence-based practice results in patient bene"t (Roth & Fonagy, 1996). This study illustrates how some prior limitations in the therapist training literature could be redressed (especially in terms of de"ning and measuring competence in a homogenous therapy). However, future research should add controls, include multidimensional measurement of the training intervention and assess generalisation across responses, settings and time.
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
91
References Alberts, G., & Edelstein, B. (1990). Therapist training: A critical review of skill training studies. Clinical Psychology Review, 10, 497}511. Ashworth, P., & Milne, D. L. (1996). Learning from the MASTER: An instrument for measuring interactions in teaching Psychology. Clinical Psychology and Psychotherapy, 3, 103}108. Beck, A. T. (1991). Cognitive therapy: A 30 year perspective. American Psychologist, 46, 368}375. Beutler, L. E., & Kendall, P. C. (1995). Introduction to the special section: The case for training in the provision of psychological therapy. Journal of Consulting and Clinical Psychology, 63, 179}181. Binder, J. L. (1993). Is it time to improve psychotherapy training?. Clinical Psychology Review, 13, 301}318. Blackburn, I. M., Milne, D. L., & James, I. A. (1997). How are therapeutic competencies evaluated? Heriot-Watt University; Edinburgh. Annual conference of the British Psychological Society (Division of Clinical Psychology). Bootzin, R. R., & Ruggill, J. S. (1988). Training issues in behaviour therapy. Journal of Consulting and Clinical Psychology, 55(5), 703}709. Corrigan, P. W., & McCracken, S. G. (1997). Interactive staw training: Rehabilitation teams that work. New York: Plenum Press. Edelstein, B. A. (1985). Empirical evaluation of clinical training. The Behaviour Therapist, 8, 67}70. Ellis, M. V., Ladany, N., Krengel, M., & Schult, D. (1996). Clinical supervision research from 1981 to 1993: A methodological critique. Journal of Counselling Psychology, 43, 35}50. Frank, E., Kupfer, D. J., Wagner, E. F., McEachran, A. B., & Cornes, C. (1991). E$cacy of interpersonal therapy as a maintenance treatment of recurrent depression. Archives of General Psychiatry, 48, 1053}1059. Freiheit, S. R., & Overholser, J. C. (1997). Training issues in cognitive-behavioural psychotherapy. Journal of Behaviour Therapy & Experimental Psychiatry, 28, 79}86. Kingdon, D., Tyrer, P., Seivewright, N., Ferguson, B., & Murphy, S. (1996). The Nottingham study of neurotic disorder: In#uence of cognitive therapists on outcome. British Journal of Psychiatry, 169, 93}97. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal & coping. New York: Springer. Milne, D. L. (1986). Training Behaviour Therapists: Methods, evaluation and implementation with parents, nurses & teachers. London: Croom-Helm. Milne, D. L. (1992). Assessment: A mental health portfolio. Windsor: NFER-Nelson. Milne, D. L., Blackburn, I. M., & James, I. (1997). Patient and therapist characteristics and competence. Paper presented at the annual conference of the British psychological society, Heriot Watt University, Edinburgh. Milne, D. L., Claydon, T., James, I., & Blackburn, I. -M. (1997). The importance of charisma and #air in competence. Heriot Watt University, Edinburgh. Annual conference of the British Psychological Society (Division of Clinical Psychology). Milne, D. L., & Noone, S. (1996). Teaching and training for non-teachers. Leicester: BPS Books. Milne, D. L., Souter, K. M. (1988). A re-evaluation of the clinical psychologist in general practice. Journal of the Royal College of General Practitioners, 38, 457}460. Moos, R. H. (Ed.) (1976). Human Adaptation: Coping with life crises. Lexington, MA: D. C. Health & Co. Moos, R. H. (1990). In D. L. Milne, Coping responses inventory. Windsor: NFER-Nelson. Padesky, C. A. (1996). Developing cognitive therapist competency: Teaching and supervision models. In P. M. Salkovskis, Frontiers of cognitive therapy. London: The Guildford Press. Perris, C. (1993). Stumbling blocks in the supervision of cognitive psychotherapy. Journal Clinical Psychology and Psychotherapy, 1, 29}43. Roth, A., & Fonagy, P. (1996). What works for whom? London: Guildford Press. Sackett, D. L., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (1997). Evidence-based medicine. How to practice and teach EBM. London: Churchhill Livingstone. Shaw, B. F., & Wilson-Smith, D. (1988). Training therapists in cognitive-behaviour therapy. In C. Perris, & I. M. Blackburn, Cognitive therapy in practice. Heidelberg: Springer.
92
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
Waltz, J., Addis, M. E., Koesner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620}630. Wells, A. (1997). Cognitive therapy of anxiety disorders. A practice manual and conceptual guide. Chichester: Wiley. Williams, R. M., Moorey, S., & Cobb, J. (1991). Training in cognitive-behaviour therapy: Pilot evaluation of a training course using the cognitive therapy scale. Behavioural Psychotherapy, 19, 373}376. Young, J. E., & Beck, A. T. (1980 and 1988). Cognitive therapy scale: Rating manual. Unpublished manuscript, Centre for Cognitive Therapy, University of Pennsylvania, Room 602, 133 South 36th Street, PA 19104, USA.
E!ectiveness of cognitive therapy training Derek L. Milne *, Chris Baker, Ivy-Marie Blackburn, Ian James, Katharina Reichelt Centre for Applied Psychology, University of Newcastle upon Tyne, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK Northumberland Mental Health NHS Trust, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK Newcastle City Health Trust, Ridley Building, Newcastle upon Tyne, NE1 7LP, UK
Abstract Despite growing demands for e!ective training there remains a lack of convincing evaluations. Reviews of the training literature specify the need for multiple measures, homogeneous therapies, generalization assessments and longitudinal designs. The present analysis responds by evaluating the training of 20 mental health professionals in cognitive therapy (CT). The evaluation focussed on changes in the professionals' competence and its generalization to their patients' coping strategies (N"20). Results indicated that a 40 day training programme led to signi"cantly higher post-training competence scores and to improved patient coping. The implications for professional training and its evaluation are discussed. 1999 Elsevier Science Ltd. All rights reserved.
1. E4ectiveness of training in cognitive therapy Evidence-based practice has been described as &the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients' (Sacket, Richardson, Rosenberg, & Haynes, 1997). The growing demand for managed care, evidence-based practice and short training courses (Roth & Fonagy, 1996) means that the need for carefully designed psychotherapy training programmes, such as cognitive therapy (CT), has increased latterly (Freiheit & Overholser, 1997). Therapy training courses must teach trainee therapists how to integrate their
* Corresponding author. Tel.: #0191-2226000 Ext. 7912. E-mail address: [email protected] (D.L. Milne) 0005-7916/99/$ - see front matter 1999 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 7 9 1 6 ( 9 9 ) 0 0 0 1 1 - 7
82
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
individual expertise with the best available clinical evidence from systematic research by: converting clinical issues arising from individual patients into answerable questions, "nding the best evidence with which to answer these, appraising the evidence for its validity and usefulness in the speci"c clinical context, applying the results of the appraisal in their clinical work, and evaluating the outcome. In addition, there is a need to demonstrate that such training programmes are e!ective, regardless of the speci"c therapy being taught, in order to ensure good use of scarce sta! resources. The evidence available is yet to provide convincing data on the e!ectiveness of training courses in psychotherapy generally (Bootzin & Ruggill, 1988; Edelstein, 1985; Beutler & Kendall, 1995). In particular, there is a need for: (1) more re"ned approaches to measurement, in particular the inclusion of expert raters and multiple measures (Alberts & Edelstein, 1990); (2) for su$cient training to develop measurable competence in newly learned techniques (Binder, 1993); (3) a clear focus on an homogenous therapy and the measurement of the consequences of competence in terms of patient outcome (Beutler & Kendall, 1995); and (4) for longitudinal analyses which examine these outcomes within experimental or quasi-experimental designs (Ellis, Ladany, Krengel, & Schult, 1996). CT is a suitable candidate for this re"ned evaluation of the e!ectiveness of training. It is an explicitly empirical approach in which adherence to therapy protocols and assessing the outcome of therapy are integral and it is well supported by the evidence on clinical outcome (Beck, 1991; Roth & Fonagy, 1996). The e!ectiveness of cognitive therapy training and supervision has not been addressed adequately in the cognitive therapy literature. Shaw and Wilson-Smith (1988) describe an intensive training and monitoring programme with the aims of developing therapists' skills in adhering to a cognitive therapy research protocol in the treatment of patients presenting with depression. This study assessed the application of CT over time. Theoretical and practical aspects of the approach were taught, demonstrated and practised, and monitoring and case supervision continued for up to 24 months. Independent expert raters used the cognitive therapy scale (CTS: Young & Beck, 1980, 1988) which yielded signi"cant training e!ects over time on the overall CTS scores, on the conceptualisation sub-scale, and on scores of the structure of CT (i.e. agenda setting, pacing, and homework). Unfortunately, the impact of the progress of the training was not measured in relation to the patients involved in this project. Frank, Kupfer, Wagner, McEachran, and Cornes (1991) studied long-term maintenance therapy for depression and found that those patients who rated their therapists as competent had longer periods of bene"t from therapy than those patients who rated their therapists as providing poor quality therapy (two years compared to four months, respectively). Williams, Moorey, and Cobb (1991) examined the impact of a training course on competence in CT by rating participants' audio-taped CT sessions at the beginning and the end of the course. Two raters, blind as to the order of the tapes, scored each tape using a shortened form of the CTS. Eight out of 11 trainees showed improvements on the CTS, but a comparison of mean scores failed to reach statistical signi"cance. Again, the impact of competence on patient outcome was not evaluated in this pilot study.
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
83
Kingdon, Tyrer, Seivewright, Ferguson, and Murphy (1996) re-analysed the data from a prior and negative clinical outcome evaluation of CT in terms of the supervisors' perceptions of the competence of the eleven therapists. This sub-analysis indicated that the competent therapists (N"4) obtained better clinical outcomes than those judged to be of &uncertain' or &doubtful' competence (N"7). The authors concluded that competence should be measured objectively and that the CT should be of greater than four sessions duration. More recently, Freiheit and Overholser (1997) examined the impact of therapists' variables, including theoretical backgrounds and gains made in CT training. Forty clinical psychologists participated in a year long training course, involving both teaching and supervised application of CT. Trainees' knowledge, attitude to CT and application of behavioural and cognitive techniques were measured before and after the training. Statistically signi"cant gains were reported in trainees' knowledge, positive appraisal of CT and CT ideology, and the practice of behavioural and cognitive techniques at the end of the course. There was also a signi"cant reduction in negative appraisal of CT, but no di!erences between trainees of di!erent theoretical orientation. However, the study relied on self report data rather than employing independent raters, and once more the impact on patients was not evaluated. In addition to these training evaluations, useful reports have been published about teaching and supervision in CT, but many are descriptive, raising awareness of issues, rather than reporting empirical data (e.g. Perris, 1993). For example, Padesky (1996) pointed out that teaching in CT in the 1990s had become increasingly demanding, with speci"c conceptualisations and treatment approaches being developed for a wide range of psychiatric disorders. She provided a helpful summary of considerations in the teaching and supervision process in CT training. Finally, Perris (1993) drew attention to the limited published work that addresses supervisor training and supervision processes in cognitive therapy, and went on to examine some of the stumbling blocks. In summary, the literature on the e!ectiveness of CT training is limited and relatively unre"ned, yet the available evidence supports the e!ectiveness of CT training in improving the competence of trainees. Furthermore, reports of the contents of such training programmes indicate that theoretical and didactic components can be combined with clinical supervision to improve trainees' knowledge and skills, in line with the development of evidence-based practice. More re"ned research, designed to evaluate the training of competences of CT and the contribution of such competences to the outcome of therapy would provide valuable information to trainers and supervisors. The purpose of the present study is to de"ne and describe a regional training programme in CT and to evaluate its impact on the competence of the mental health professionals undertaking the training within a longitudinal design. The most distinctive feature of this study is the relationship of competence to the clinical outcomes achieved by patients, perhaps the best test of competence (Roth & Fonagy, 1996). This is the "rst such in-depth evaluation of training in the UK (Williams, Moorey, & Cobb, 1991), referred to their own work as a pilot study) and the "rst to use an improved measure of competence (the CTS-R: (Blackburn, Milne,
84
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
& James, 1997). It was predicted that the training program would result in signi"cantly raised competence amongst the trainee cognitive therapists, which would in turn generalise to signi"cant improvements in the clinical status of their patients.
2. Methods 2.1. Training programme The training programme in CT was run in a well-established centre in the UK, the Newcastle Cognitive Therapy Centre. The course consisted of an intensive induction week, followed by 35 training days spread over three university terms (nine month duration). Each day consisted of three hours of large group didactic instruction by presenters of national and international repute, two hours of clinical work and one and a half to two hours of supervision in pairs. Supervisors were either experienced cognitive therapists in NHS settings who had attended a cognitive therapy supervisors course or were members of the specialist Cognitive Therapy Centre. All attended regular supervisor training meetings to maintain supervisory skills. Trainees routinely videotaped their therapy sessions and these tapes were reviewed by supervisors and used in supervision. Supervisors were encouraged to emphasise the development of cognitive formulations, linking structure, process and theoretical models in therapy. It was expected that supervisory strategies such as role-play and modelling would feature in supervision sessions. Supervisors had, in previous years, been expected to submit an audiotape or videotape of a supervision session on two occasions, in order that the quality of supervision be examined by course organisers. Trainees were formally evaluated through the course on the competence ratings of two therapy videotapes submitted in the second and third terms, as well as through two essays and two case studies. 2.2. Therapists Entry to the programme was by open competition amongst professionally quali"ed mental health sta! from the UK. The present analysis focuses on one intake of 20 professionals, consisting of clinical psychologists (N"6), psychiatrists (N"6) and mental health nurses (N"8). There were 11 females and 9 males in this sample, with an average age of 39.0 years (SD 7.0). Most were Caucasian, with two representatives of two ethnic minorities. 2.3. Patients The patients (N"20) had an average age of 37.7 (SD 13.2) with a range of 19}70 years of age. Eleven were female and nine were male. Thirteen were married or cohabiting and seven were single, separated or divorced. All su!ered from an Axis I disorder, usually depression or an anxiety disorder. Patients were selected by supervisors from routine referrals to the Newcastle Cognitive Therapy Centre
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
85
(a tertiary referral centre), having already been screened by members of the centre, or were selected from supervisors' NHS waiting lists. Patients provided informed consent to be seen by cognitive therapists in training and for video recordings of their therapy and psychometric data to be used in this study. 2.4. Raters There were four raters. All were trained supervisors for the training course and were considered to be experts in CT and in the use of the CTS-R.
3. Methods and procedure 3.1. Measurement of competence The CTS-R is based on the original CTS (Young & Beck, 1980, 1988) and was developed by experienced cognitive therapists and researchers (Blackburn, Milne, & James, 1997; Milne, Blackburn, & James, 1997). The revised CTS has 14 items retaining items of the original scale and adding additional items: (1) Agenda setting, (2) Eliciting feedback (emphasising two way feedback), (3) Collaboration, (4) Pacing and e$cient use of time, (5) Interpersonal e!ectiveness (incorporating Empathy and Professionalism), (6) Facilitation of emotional expression (new item), (7) Charisma and #air factor (new item), (8) Guided discovery, (9) Conceptualisation, (10) Key cognitions and emotions, (11) Application of cognitive techniques, (12) Application of behavioural techniques, (13) Use of homework, (14) Therapists non-verbal behaviour (new item). The 14th CTS-R item was not included in the present analysis. The items are carefully de"ned and rated on a revised and detailed six point competency scale. This unipolar Likert Scale extends from &incompetent' (level 0) to &expert' (level 5). The CTS-R has been shown to have adequate inter-rater reliability and high internal consistency (Blackburn et al., 1997). The competence of the CT trainees was assessed on the CTS-R at three points during a 12 week period of therapy by means of video tape recordings made in the early stages (sessions 1}4), in the middle (sessions 5}8), and towards the end of the therapy (sessions 9}12). The video tapes were independently double rated by the four expert raters who were blind to the sessions sampled. Competency was measured as the average of the two scores awarded to each of these three recordings. 3.2. Measurement of benexts to patients (generalisation) A general measure of clinical outcome was used to assess impact of CT (the coping responses inventory, CRI: Moos, 1990). The CRI is a measure of an individual's personal coping strategies and has been regarded as the fundamental mediator of therapeutic change (Moos, 1976; Lazarus & Folkman, 1984). It has eight dimensions, four addressing approach-based coping (logical analysis, positive appraisal, seeking guidance and taking problem solving action) and four covering avoidance-based
86
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
coping (cognitive avoidance, resignation, seeking alternative rewards and emotional discharge). The dimensions can also be broken down into cognitive (logical analysis, positive appraisal, cognitive avoidance, and acceptance) and behavioural coping strategies (support seeking, problem-solving, alternative rewards, emotional discharge). In general, the four approach based coping strategies tend to be more adaptive for patients and so therapists will typically try to enhance these, while encouraging their patients to decrease their use of avoidance-based coping strategies (Milne, 1992). Each of the CRI's 48 statements is evaluated on a four point frequency scale, ranging from &never' to &fairly often' and is related to a stressor described by the patient (e.g. unemployment). An example of an approach-based cognitive coping strategy is &think of di!erent ways to deal with the problem' (from the logical analysis dimension), while &keep away from people in general' is an example of an avoidancebased behavioural coping strategy (from the emotional discharge dimension). The CRI was administered at the beginning and end of treatment.
4. Design The research design was quasi experimental, utilising direct observations and self-report instruments before, during and after training (for the CT trainees) and therapy (for the patients). Although this was a longitudinal design with respect to the training phase, it did not include a post-training follow-up assessment point. Hypotheses: It was predicted that the training course would improve the therapists' competence leading to better coping pro"le for the patients. In particular, therapists were expected to improve most in terms of the speci"c cognitive therapy competencies (i.e. CTS-R items 8}13 } see Table 2), while the patients were expected to have higher CRI frequencies of approach coping strategies, and lower frequencies of the avoidance strategies.
5. Results 5.1. Change in competence following training Following training in CT, there was a highly signi"cant increase in overall therapist competence, as measured by expert raters on the CTS-R. (Repeated one-way ANOVA: F (2,38)"4.24, p(0.02.) Table 1 summarises this "nding, which represents an overall improvement in competence from &advanced beginner' to &competent' on the competence scale. The breakdown of the changes in CTS-R items between the "rst and last assessment point are presented in Table 2, indicating that there were signi"cant improvements in the therapists competence in eight of the 13 CTS-R subscales. Highly signi"cant changes (i.e. p(0.01) were obtained for the subscales &interpersonal e!ectiveness', &facilitation of emotional expression', &charisma and #air', &guided discovery', &conceptualisation' and &application of cognitive techniques'. Signi"cant changes were also
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
87
Table 1 E!ect of training on therapist competence (N"20), as measured by the cognitive therapy scale } revised (13-items } item 14 rating non-verbal behaviour excluded)
Mean total score SD Range
Time 1 Sessions 1}4
Time 2 Sessions 5}8
Time 3 Sessions 9}12
30.3 8.2 19.3}44.3
33.1 7.2 20.5}45.0
37.0 8.1 23.0}53.5
Table 2 Mean competence scores (SDs) on individual items of the CTS-R at the "rst and "nal assessment phases for the patients seen from the start of training (N"20) Items
1 2 3 4 5 6 7 8 9 10 11 12 13
Agenda setting Eliciting feedback Collaboration Pacing Interpersonal e!ectiveness Facilitation of emotional expression Charisma and #air Guided discovery Conceptualisation Focus on key cognitions Application of cognitive technique Application of behavioural techniques Use of homework
Beginning of treatment
End of treatment
¹
p
2.7 2.7 2.9 2.7 3.0 2.3 2.5 2.5 2.2 2.5 2.0 1.7
2.9 2.9 3.4 2.9 3.6 2.8 3.2 3.1 2.8 2.9 2.7 2.4
(1.0) (0.7) (0.8) (0.8) (0.9) (0.9) (1.0) (1.0) (0.8) (0.8) (0.8) (1.0)
0.94 1.19 2.51 0.97 2.6 2.58 2.87 2.79 3.00 1.68 3.21 1.95
NS NS p(0.05 NS p(0.01 p(0.01 p(0.005 p(0.01 p(0.005 NS p(0.005 p(0.05
3.1 (0.7)
0.92
NS
(1.0) (0.8) (0.9) (0.7) (0.9) (0.7) (1.0) (0.8) (0.7) (0.7) (0.8) (1.2)
3.0 (0.7)
obtained for &collaboration' and &application of behavioural techniques'. It should be noted, therefore, amongst these there were signi"cant improvements in three of the six speci"c CT techniques (i.e. CTS-R items 8, 9 and 11). As the inter-rater reliability and internal consistency of the CTS-R were high (Blackburn et al., 1997) and raters were blind to phase of training, these "ndings probably re#ect a real e!ect of training. As predicted, the results suggest that the training course was led to improved competence in trainees, and provide some more detailed information about those competencies that were most improved by training. 5.2. Changes in patients' coping strategies (generalisation) In terms of clinical outcome, it is evident that, as a group, the patients improved their coping pro"le to a signi"cant degree. Table 3 summarises the mean scores
88
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
Table 3 Comparison of patients' baseline and outcome scores (Coping response inventory; N"20) Coping response
Baseline
Outcome
P"
x
SD
x
SD
Approach coping strategies
Logical analysis Positive appraisal Problem solving Support seeking
45.3 41.8 41.6 46.8
11.6 9.8 11.7 11.3
47.5 44.7 45.4 49.1
9.5 8.8 11.0 8.8
NS 0.03 0.05 NS
Avoidance coping strategies
Cognitive avoidance Acceptance Alternative reward Emotional discharge
53.6 52.8 49.5 59.0
8.6 10.2 9.4 9.1
50.6 50.4 47.0 55.1
8.4 10.1 7.6 8.6
0.07 NS NS 0.02
obtained on the CRI before and after receiving cognitive therapy, indicating signi"cantly higher use of positive appraisal and problem solving, with signi"cantly less frequent use and emotional discharge. This represents a much-improved coping pro"le, two of the four speci"c predictions being upheld statistically (increased positive appraisal and problem solving). A third predicted change, decrease in cognitive avoidance, showed a trend towards signi"cance (p"0.07). These "ndings and the consistent, desirable e!ects across all the other CRI subscales indicate that CT had as marked an impact on behavioural and a!ective responding as it had on the patients' cognitions. In summary, these "ndings indicate that the 20 CT trainees became more competent following the training programme as measured by the CTS (R). This appeared to be associated with the provision of e!ective therapy to the 20 clients (CRI).
6. Discussion 6.1. Main xndings The results of this study indicate that a carefully designed training programme signi"cantly improved competence in cognitive therapy in the group of 20 therapists, raising it from the &advanced beginner' level to that of &competent'. Experts blind to the phase of training independently rated tapes from the beginning, middle and end of therapy, giving signi"cantly higher CTS-R ratings to the therapy provided at the end of training. The breakdown of the CTS-R into its components o!ers valuable insight into the e!ectiveness of the course in training therapists in adherence to speci"ed therapeutic procedures and in skilfulness in the application of such procedures. It is satisfying to note that there were improvements in therapists' collaborative style and interpersonal e!ectiveness in therapy sessions, and increased adherence and skill in the use of guided discovery and cognitive techniques, and in the incorporation of
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
89
behavioural techniques. These "ndings suggest that therapists can develop key competences in CT, the dominant form of evidence-based practice (Roth & Fonagy, 1996), strengthening the evidence from earlier research (Shaw & Wilson, 1988; Williams et al., 1991; Freiheit & Overholser, 1997). An important feature of the present study was to demonstrate that delivery of competent cognitive therapy was mirrored in general improvements in the patients' coping repertoires. The improved coping was not restricted to changes in the CRI's cognitive coping strategies, but rather was evenly distributed across cognitive, behavioural and a!ective responding. This indicates that successful cognitive therapy brings changes in the patients' maladaptive thinking patterns as well as to their behavioural and emotional expression. As expected, competent CT was associated with a trend of increased approach and decreased avoidance coping strategies, a "nding that is in keeping with both the transactional stress theory (Lazarus & Folkman, 1984; Moos, 1976) and previous clinical research (e.g. Milne & Souter, 1988). Taken together, these "ndings suggest that relatively short training courses in evidence-based practice can be e!ective, and provide data that have been sadly wanting (Bootzin & Ruggill, 1988; Beutler & Kendall, 1995). Most therapists in this study were reaching competent levels of ability in all areas measured by the CTS-R, except in the use of behavioural techniques. In this particular item there was a signi"cant gain, but a low starting point. This is re#ected in qualitative information from the raters, suggesting that the therapists relied too heavily on a limited number of verbal restructuring techniques, in particular on reviewing evidence for and against negative automatic thoughts and the use of alternative, less dysfunctional responses. The relative lack of behavioural methods was of particular concern. Wells (1997) has demonstrated the importance of such methods in eliciting patients' experience, socialising patients to the CT model, reattributing unhelpful cognitions, and modifying a!ect. In terms of this feature, the present therapists barely escaped from the novice categorisation by the end of the course. This is an area for increased emphasis in this training programme, perhaps particularly in supervision, along with extending trainees' repertoire and skills in evaluation techniques. The items concerning therapeutic structure, namely agenda setting, pacing, setting homework and seeking feedback showed no signi"cant gain. However, the initial scores for these items revealed reasonable levels of competence from the outset. A possible explanation for this "nding is that the training provided during the week long intensive induction course was su$cient to enable trainees to adhere to an adequate CT structure for a therapy session. As a consequence, the competence ratings from the "rst videotapes may have re#ected a training outcome, rather than a true baseline value for competence in these aspects of CT. 6.2. Methodological critique While these data represent the most systematic evaluation of CT training to date, there are nonetheless ways in which the study could have been strengthened. Firstly, future research should measure the training itself, providing a manipulation check
90
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
analogous to the competence check carried out for therapy (Waltz, Addis, Koesner, & Jacobson, 1993). Prior research has shown how a training intervention, in the same way as therapy, can be operationalized in terms of type (e.g. experiential or didactic), pro"ciency (e.g. &novice' or &master') and interpersonal e!ectiveness (Ashworth & Milne, 1996). Just as in therapy, variations in these dimensions of the training intervention may lead to signi"cantly di!erent learning outcomes. Similarly, careful assessment of these training variables over time can provide informational feedback, a powerful basis for improving the competence of the trainers. For example, careful assessment, feedback and participative decision-making on such aspects as the involvement of the participants or the match between the training method and learning styles can optimise the e!ectiveness of training (Milne & Noone, 1996). A second major weakness of the present study was its quasi-experimental research design. Future research could strengthen our pre-post design by adding a control condition, in routine practice perhaps, best achieved by implementing a double baseline (waiting list) assessment for both trainees and patients. This would enable greater con"dence in drawing conclusions about the causal relationship between therapists' competency and therapy outcome. As things stand, we cannot exclude the possibility that it was improvements in the patients that led to enhanced competence in the trainees. A further re"nement would be to extend our generalisation analysis to include evaluation of the therapists' learned skills to their routine work with patients after training. This could also cover generalisation across time (a post-training follow-up assessment), across settings (i.e. to the trainee's normal work environment) and possibly also across behaviours (i.e. do the trainees extend their use of the trained skills so as to develop novel variants of the therapy or to address new clinical presentations?). Analysis of these forms of generalisation have typically indicated major limitations to training (Milne, 1986) and a concomitant need to focus on the organisational factors that can powerfully in#uence generalisation (Corrigan & McCracken, 1997). These are important topics for future evaluations of CT training.
7. Conclusions A 40 day training course in cognitive therapy appeared to result in signi"cantly improved competence amongst 20 cognitive therapy trainees and this was associated with improvement in coping in 20 patients. This kind of generalisation assessment is both the scienti"c criterion for successful therapist training and essential for a demonstration that training in evidence-based practice results in patient bene"t (Roth & Fonagy, 1996). This study illustrates how some prior limitations in the therapist training literature could be redressed (especially in terms of de"ning and measuring competence in a homogenous therapy). However, future research should add controls, include multidimensional measurement of the training intervention and assess generalisation across responses, settings and time.
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
91
References Alberts, G., & Edelstein, B. (1990). Therapist training: A critical review of skill training studies. Clinical Psychology Review, 10, 497}511. Ashworth, P., & Milne, D. L. (1996). Learning from the MASTER: An instrument for measuring interactions in teaching Psychology. Clinical Psychology and Psychotherapy, 3, 103}108. Beck, A. T. (1991). Cognitive therapy: A 30 year perspective. American Psychologist, 46, 368}375. Beutler, L. E., & Kendall, P. C. (1995). Introduction to the special section: The case for training in the provision of psychological therapy. Journal of Consulting and Clinical Psychology, 63, 179}181. Binder, J. L. (1993). Is it time to improve psychotherapy training?. Clinical Psychology Review, 13, 301}318. Blackburn, I. M., Milne, D. L., & James, I. A. (1997). How are therapeutic competencies evaluated? Heriot-Watt University; Edinburgh. Annual conference of the British Psychological Society (Division of Clinical Psychology). Bootzin, R. R., & Ruggill, J. S. (1988). Training issues in behaviour therapy. Journal of Consulting and Clinical Psychology, 55(5), 703}709. Corrigan, P. W., & McCracken, S. G. (1997). Interactive staw training: Rehabilitation teams that work. New York: Plenum Press. Edelstein, B. A. (1985). Empirical evaluation of clinical training. The Behaviour Therapist, 8, 67}70. Ellis, M. V., Ladany, N., Krengel, M., & Schult, D. (1996). Clinical supervision research from 1981 to 1993: A methodological critique. Journal of Counselling Psychology, 43, 35}50. Frank, E., Kupfer, D. J., Wagner, E. F., McEachran, A. B., & Cornes, C. (1991). E$cacy of interpersonal therapy as a maintenance treatment of recurrent depression. Archives of General Psychiatry, 48, 1053}1059. Freiheit, S. R., & Overholser, J. C. (1997). Training issues in cognitive-behavioural psychotherapy. Journal of Behaviour Therapy & Experimental Psychiatry, 28, 79}86. Kingdon, D., Tyrer, P., Seivewright, N., Ferguson, B., & Murphy, S. (1996). The Nottingham study of neurotic disorder: In#uence of cognitive therapists on outcome. British Journal of Psychiatry, 169, 93}97. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal & coping. New York: Springer. Milne, D. L. (1986). Training Behaviour Therapists: Methods, evaluation and implementation with parents, nurses & teachers. London: Croom-Helm. Milne, D. L. (1992). Assessment: A mental health portfolio. Windsor: NFER-Nelson. Milne, D. L., Blackburn, I. M., & James, I. (1997). Patient and therapist characteristics and competence. Paper presented at the annual conference of the British psychological society, Heriot Watt University, Edinburgh. Milne, D. L., Claydon, T., James, I., & Blackburn, I. -M. (1997). The importance of charisma and #air in competence. Heriot Watt University, Edinburgh. Annual conference of the British Psychological Society (Division of Clinical Psychology). Milne, D. L., & Noone, S. (1996). Teaching and training for non-teachers. Leicester: BPS Books. Milne, D. L., Souter, K. M. (1988). A re-evaluation of the clinical psychologist in general practice. Journal of the Royal College of General Practitioners, 38, 457}460. Moos, R. H. (Ed.) (1976). Human Adaptation: Coping with life crises. Lexington, MA: D. C. Health & Co. Moos, R. H. (1990). In D. L. Milne, Coping responses inventory. Windsor: NFER-Nelson. Padesky, C. A. (1996). Developing cognitive therapist competency: Teaching and supervision models. In P. M. Salkovskis, Frontiers of cognitive therapy. London: The Guildford Press. Perris, C. (1993). Stumbling blocks in the supervision of cognitive psychotherapy. Journal Clinical Psychology and Psychotherapy, 1, 29}43. Roth, A., & Fonagy, P. (1996). What works for whom? London: Guildford Press. Sackett, D. L., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (1997). Evidence-based medicine. How to practice and teach EBM. London: Churchhill Livingstone. Shaw, B. F., & Wilson-Smith, D. (1988). Training therapists in cognitive-behaviour therapy. In C. Perris, & I. M. Blackburn, Cognitive therapy in practice. Heidelberg: Springer.
92
D.L. Milne et al. / J. Behav. Ther. & Exp. Psychiat. 30 (1999) 81}92
Waltz, J., Addis, M. E., Koesner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620}630. Wells, A. (1997). Cognitive therapy of anxiety disorders. A practice manual and conceptual guide. Chichester: Wiley. Williams, R. M., Moorey, S., & Cobb, J. (1991). Training in cognitive-behaviour therapy: Pilot evaluation of a training course using the cognitive therapy scale. Behavioural Psychotherapy, 19, 373}376. Young, J. E., & Beck, A. T. (1980 and 1988). Cognitive therapy scale: Rating manual. Unpublished manuscript, Centre for Cognitive Therapy, University of Pennsylvania, Room 602, 133 South 36th Street, PA 19104, USA.