- Email: [email protected]
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
Nurse Education Today (2008) 28, 15–23
Nurse Education Today intl.elsevierhealth.com/journals/nedt
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland Gillian Knowles a,*,1, Catherine Hutchison b,2, Graeme Smith c, Iona D. Philp a, Katrina McCormick a, Elizabeth Preston a a
Edinburgh Cancer Centre, Western General Hospital, Crewe Road, Edinburgh EH4 2XU, United Kingdom North Glasgow University Hospitals, NHS Division, Administration Building, Western Infirmary, Dumbarton Road, Glasgow G11 6NT, United Kingdom c School of Health, University of Edinburgh, Teviot Place, Edinburgh, United Kingdom b
Accepted 4 February 2007
KEYWORDS
Summary This paper describes the results of an evaluation of a nursing education programme for nurses caring for patients with colorectal cancer. A sample of 67 registered nurses from 6 location sites within the 3 Regional Cancer Networks in Scotland were recruited to the pilot. The programme was adapted from an evidence-based education manual for nurses in the management of colorectal cancer developed by the European Oncology Nursing Society as part of a Nursing in Colorectal Cancer Initiative (NICCI) [Hawthorn, J., Redmond, K., 1999. A Guide to Colorectal Cancer. AstraZeneca Oncology, UK]. The format for evaluating the programme was based on the TELER method of treatment evaluation [Le Roux, A.A., 1995. TELER: the concept. Physiotherapy 79 (11), 755–758] that had previously been developed along side the training manual [Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K, 2001a. Nursing in colorectal cancer initiative – the audit phase. Part 1. Development of the audit tool. European Journal of Oncology Nursing 5 (2), 100–111; Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001b. Nursing in colorectal cancer initiative: the audit phase. Part 2. Content validity of the audit tool and implications of the standards set for clinical practice. European Journal of Oncology Nursing 5 (3), 165–173] for cytotoxic
Nurse education; Cancer; Colorectal cancer; Evaluation; Clinical outcomes
* Corresponding author. Tel.: +44 131 244 3291. E-mail addresses: [email protected] (G. Knowles), [email protected] (C. Hutchison). 1 Currently on secondment as Nurse Consultant, Scottish Executive, St Andrew’s House, Edinburgh. 2 Tel.: +44 141 211 2344.
0260-6917/$ - see front matter c 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.nedt.2007.02.002
16
G. Knowles et al. chemotherapy to provide the NICCI Audit Tool (Le Roux, 2003). This model was developed further in the current study to include the domains of: Disease, Diagnosis and Staging, Treatment, Nursing Issues and General Issues. Data were analysed descriptively and are discussed. Overall the results from this study demonstrate a statistically significant improvement in disease-related knowledge (p = <0.001) and in the best practice statements for nursing issues (p = <0.001) and general issues (including attitudes) (p = 0.023) that were maintained at four months post completion of the course. c 2007 Elsevier Ltd. All rights reserved.
Introduction Colorectal cancer constitutes one of the most common cancers and is the second cause of cancer death in Scotland (Information and Statistical Division, 2000). The pathway for patients is complex and care is delivered in many different settings. Therefore educational preparation in the management of colorectal cancer is important and relevant for nurses working within various hospital departments and community settings. The aim of service provision is to ensure the smooth pathway for patients across the various care settings and the delivery of high quality care. For this to happen effectively nurses need to have core knowledge and skills in the management of the disease and a clear understanding of the interlinked components of service delivery. This paper describes the implementation and evaluation of a pilot education programme for nurses caring for patients with colorectal cancer across 6 location sites in Scotland.
Background/literature One of the key drivers for health policy in the NHS is to ensure consistent and high quality care for patients with cancer (Scottish Executive Health Department, 2004a, 2001a; NHS Education Scotland, 2003a; Scottish Executive Health Department, 2005). There has been a major shift in how we manage cancer services in Scotland through the development of Managed Clinical Networks, bringing together healthcare professional and patient representatives with the aim of ensuring consistent and seamless care for patients with specific tumour types (Scottish Executive Health Department, 2001a). These key policy documents in addition to clinical and professional guidelines that focus on best practice are influencing ways in which we support health professionals to develop the necessary knowledge and skills to meet the challenges of a modern NHS in Scotland. With this in mind, education and continuing professional development for all healthcare professionals are
therefore very important. However how do we ensure that an intended education programme is effective in influencing nursing practice? Evaluating clinical effectiveness as a result of an educational intervention is inherently difficult due to the numerous variables that can influence clinical outcomes (Jordon et al., 1999). However, with an increasing emphasis on continuing professional development stakeholders need reliable evidence that their investment has been worthwhile. While there remains a lack of empirical data supporting the relationship between education and clinical effectiveness there have been a number of general cancer nursing studies demonstrating improvements in cancer-related knowledge and attitudes as a result of education (Corner and Wilson-Barnett, 1992; Corner, 1993; Hooker and Milburn, 2000; Lasch et al., 2000; Loftus and Thompson, 2002; McClement et al., 2005). In addition, a number of studies have been conducted to identify the participants’ perceptions of how their practice changed as a result of an educational intervention and have been evaluated positively (Steginga et al., 2005; Howell et al., 2000). The evaluations of education and training programmes in cancer nursing have been approached in a variety of ways. Primarily through the use of pre and post-test questionnaires, cancer attitudes scales and focus groups interviews. Many of these studies have taken one aspect of cancer or palliative care to conduct and evaluate an intervention, for example pain management (Howell et al., 2000; Plymale et al., 2001; Dalton et al., 1995). However as yet there are no published data specifically evaluating education interventions aimed at providing nurses with a improved understanding of the interlinked components of service delivery in colorectal cancer management. With greater emphasis on sub-specialisation and cross boundary working (Scottish Executive Health Department, 2005), the aim of the current study was to pilot and evaluate an education programme for nurses caring for patients with colorectal cancer across different care settings throughout Scotland. Identifying an evaluation strategy that would be sensitive
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
to measuring changes in knowledge and understanding of the pathway of care for patients with bowel cancer was an important consideration and proved challenging. There were no ‘off-the shelf’ tools available with the exception of a recent audit tool that had been developed to measure standards of care in relation to the delivery of cytotoxic chemotherapy to patients with advanced colorectal cancer (Grocott et al., 2001a; Grocott et al., 2001b) using the TELER system of audit (Le Roux, 1995). Initially developed to measure the effectiveness of the treatment received by a patient by determining the clinical and statistical significance of the improvement experienced by the patient, the TELER methodology was further developed by Grocott et al. (2001a,b) to measure the clinical performance of a nurse in the delivery of cytotoxic chemotherapy with the potential to measure changes in performance when repeated. Expansion and adaptation of the TELER framework to include all aspects of colorectal cancer management was considered to be an appropriate approach in the evaluation of the present study (personnel correspondence with Patricia Grocott and AA Le Roux, April 2002).
Study objectives The overall aim of the study was to improve the understanding of the care pathway for patients with colorectal cancer and the interlinked components of service delivery in a group of registered nurses. The specific objectives were: To provide a comprehensive educational basis for registered nurses in the management of colorectal cancer through the implementation of an evidence-based education programme. To evaluate knowledge, general cancer nursing issues and best practice statements prior to, and after the implementation of the programme. To evaluate how site-specific cancer nursing education impacts on nurses perceptions of changes in their practice 4 weeks and 4 months post course completion (based on the literature and timeframe for the study).
Methods Design and procedure The study was designed around an evidence-based education manual for nurses in the management of colorectal cancer developed by the European Oncology Nursing Society as part of a Nursing in
17
Colorectal Cancer Initiative (NICCI) (Hawthorn and Redmond, 1999). The original material, referred to as the NICCI manual, was developed in an open-learning format, divided into four chapters to cover the topics of colorectal disease, diagnosis and staging, treatment, and nursing care. Each section contains learning objectives and a number of learning activities that the readers are encouraged to undertake as they progress through the manual. Permission to use and adapt the NICCI manual for the purpose of the current study was obtained from the European Oncology Nursing Society. The manual was supplemented with prepared up-to-date research evidence and each section was introduced through a series of four education study days over a five-month period. The self-directed component of the programme required the participants to work through each section of the manual and supplementary material, setting their own personal objectives. Management approval was granted from the 6 location sites to enable the participants to participate in the pilot, attend the four study days along with two hours per week for personal study. A Steering Group consisting of three Lead Colorectal Cancer Nurse Specialists, a Nurse Consultant, an Assistant General Manager, a Nurse Educationalist, a Colorectal Surgeon and a Professor of Medical Oncology managed the overall study. Two project managers managed the day-to-day organisation of the study.
Sample and setting A total sample of 67 registered nurses from 6 location sites within the 3 Regional Cancer Networks in Scotland were recruited to the pilot. As bowel cancer is managed in both acute and primary care and across surgical, medical and oncology settings, the location sites were selected for inclusion in the project to reflect this. The participants were selfselecting following a series of advertisements and through communication from local planning teams comprising of the Colorectal Clinical Nurse Specialists based in each of the location sites. There were no exclusion criteria with the exception that the participants required to be working with patients with colorectal cancer. It was also made explicit in the advertisement that the course was aimed at providing core level knowledge to inform the pathway of care.
Instruments As part of the Nursing in Colorectal Cancer Initiative the TELER audit was developed along side
18
G. Knowles et al.
the training manual (Grocott et al., 2001a,b). As previously discussed the audit tool was developed to measure standards of nursing care in relation to the delivery of cytotoxic chemotherapy to patients with advanced colorectal cancer. The TELER methodology provides an established format for the construction of the evaluation questions, the TELER indicator, which reduces random response error (Le Roux, 1995; Le Roux, 2003). Permission to use and develop the TELER framework was granted under licence to the author, A.A. Le Roux. Development of pre and post education intervention quiz-style questionnaires included the following domains: The Disease (incidence, aetiology, pathology, genetics, prevention and screening). Diagnosis and Staging. Treatment. Nursing Issues (colorectal cancer best practice statements). General Issues. The questionnaires covered each chapter of the manual and along with best practice statements. These statements related to the participants’ perceptions of how comfortable and confident they felt in communicating, informing and supporting patients with colorectal cancer throughout the pathway and their role and contribution within the multidisciplinary team. In addition, cancer attitude questions were incorporated in the General Issues audit questionnaire adapted from the Cancer Attitude Scale (Haley et al., 1968; Haley et al., 1977). Corner (1993) suggests that attitudes may affect the quality of care that nurses provide to patients, with positive attitude scores linked to the nurse’s ability to effectively care for patients with cancer and their family
Table 1
members. The aims of the pre- and post-test questionnaires were to evaluate the impact of the course on participant’s knowledge, attitudes and practice once the relevant study days and self-directed reading were complete (see Table 1). The audit tools were developed by a Colorectal Cancer Nurse Specialist and Oncology Nurse Consultant and subject to peer review for content validity by four Lead Colorectal Nurse Specialists, a Nurse Researcher and a Nurse Educationalist. A demographic questionnaire was devised specifically for the study to document participant details. In addition, a programme monitoring questionnaire was completed to assess the participants overall evaluation of the course.
Data analysis Using TELER methodology the level of statistical significance was set at <0.05. Responses to the quiz style statements were scored as follows: Correct = 6, Don’t know = 4, Incorrect = O. TELER methodology states a correct response to a statement is made by a respondent who possesses the appropriate knowledge, and is aware of the extent and limitations of that knowledge (score of 6). A neutral response to a statement is made by a respondent who does not possess the appropriate knowledge and is aware of the extent and limitation of that knowledge. A neutral response implies that the respondent is unlikely to act in a manner that would impede the effectiveness of a patient’s treatment or care and is therefore weighted highly (score of 4). Finally, an incorrect response to a statement is made by a respondent who does not possess the appropriate knowledge and is unaware of the extent and limitations of that lack of knowl-
Time points for completion of quiz style audit questionnaire
Time points
Questionnaire completion
Study Day 1 – The disease
General issues (pre-test) The disease (pre-test) Nursing issues (pre-test) The disease (post-test) Diagnosis & Staging (pre-test) Diagnosis & Staging (post-test) Treatment (pre-test) Treatment (post-test) General issues (post-test) Nursing issues (post-test) General issues (post-test) Nursing issues (post-test)
Study Day 2 – Diagnosis & Staging Study Day 3 – Treatment Study Day 4 – Nursing issues 4 weeks post course completion 4 months post course completion
19
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
edge. Consequently an incorrect response is awarded a score of 0. In other words, an incorrect response is expected to be equal and opposite to that of a correct response and potential effect of a ‘don’t know’ response is considered to be negligible (Le Roux, 2003). Individual participant scores for each audit questionnaire were awarded an unacceptable, acceptable or excellent score. From these scores a target of 90% index of achievement was set for the group. The index of achievement determines the percentage of the group that attain ‘at least’ acceptable score for each questionnaire rather than individual scores. The research team agreed the outcome of interest was change in the number of questions answered correctly for each of the five pre and post audit questionnaires and also individual correct responses. Using Minitab further data analysis was conducted by an independent statistician where scoring included ‘correct’ and ‘incorrect’ responses only. Comparisons between results were made using analysis of variance (ANOVA) where there were more than two groups and two-sample t-test where there were only two groups. Both these methods determine if there are statistically significant differences between the means of either two groups (t-test) or between the means of all the group (ANOVA).
Results A total of 67 registered nurses were recruited to the study. Fifty-two participants completed the course and 47 participants successfully returned their follow-up evaluation audit questionnaire at 4 months post course completion. Of the 53 participants who completed the course, 2 considered the course to be at too low a level to meet their requirements, 10 were unable to get time off to attend and in 3 cases the reasons were not documented. Only 9% (n = 6) were male; 62.1% (n = 41) of the sample were in full-time occupation; median
Table 2
number of years qualified was 12 (inter-quartile range 5, 23); and years spent in present post 3 (inter-quartile range 1, 7.25). The distribution of grade and professional qualification are seen in Table 2. Only 13.4% (n = 9) of the participants had a cancer-related qualification. Forty-one percent (n = 28) were educated to degree level. Table 3 provides the index of achievement scores in relation to the five identified domains. These scores are presented as pre and post scores for the study population. It is important to note that the percentages are not measuring differences between pre- and post-test scores but rather the percentage of the group that have reached an ‘acceptable or above’ level of achievement at that given time. TELER methodology does not allow for changes in the individual number of questions answered correctly pre- and post-test. As this was the main outcome of interest to this study further analysis was conducted. Table 4 shows the mean (SD) of the number of correct answers in each section for all cases overall, along with the minimum, median and inter-quartile range. Looking at the results overall to see if there is a significant ‘Post–Pre’ change, paired t-tests have been used. Table 5 summaries the results. From this it can be seen there was no significant change in the number of correct answers given for either treatment or diagnosis and staging. However, for disease, nursing issues and the second general issues questionnaire there was a significant improvement. Table 6 provides the summary results from the programme monitoring questionnaire on completion of the course (n = 52 returns). Looking at the results, all of the participants enjoyed the programme, reporting that the content was relevant and that they would utilize the knowledge gained in their own area of practice. Finally, there were no differences in the change in number of correct answers by nursing grade and
Distribution of grade and professional qualification
Nursing grade
Number (n = 67)
%
Professional qualification
Number (n = 67)
%
EN S/N D Grade S/N E Grade S/N F Grade Charge Nurse
3 29 9 10 16
4.45% 43.3% 13.4% 14.9% 23.9%
RGN Diploma BSc BA MSc Cancer-related qualification*
23 16 19 8 1 9
34.3% 23.9% 28.4% 11.9% 1.5% 13.4%
*
Note as some nurses have multiple qualifications percentages may not total 100.
20
G. Knowles et al.
Table 3
Index of achievement
Index of achievement: (required 90% of group)
N
Pre-test (%)
N
Post-test (%)
N
2nd Retest
The disease Diagnosis and staging Treatment General issues Nursing issues
67 61 61 67 67
80 72 98 98 23
61 59 58 51 51
84 60 98 99 81
N/A N/A N/A 47 47
N/A N/A N/A 99 91
Table 4
Descriptive statistics of number of correct answers for each section
Number of correct answers for each section General issues
Nursing issues
N
Mean
SD
Minimum
Maximum
Median
Inter-quartile rangea
Pre Post Post Post Post Post
1 1–Pre 2 2–Pre 2–Post 1
67 51 50 47 46 44
10.69 10.73 0.18 11.04 0.50 0.41
1.21 1.04 1.61 0.91 1.49 1.15
7 7 4 9 2 2
12 12 4 12 4 4
11 11 0 11 0 0
(10, 12) (10, 12) ( 1, 1) (10, 12) ( 0.25, 1.25) (0, 1)
Pre Post Post Post Post Post
1 1–Pre 2 2–Pre 2–Post 1
67 51 50 47 46 44
5.93 9.14 3.24 9.66 3.89 0.55
2.40 1.77 2.46 1.51 2.25 1.61
1 6 3 6 0 5
11 12 10 12 10 4
6 9 3 10 4 1
(4, 7) (8, 11) (2, 5) (8, 11) (2.75, 5.00) (0, 1) (9, 10) (9, 10) ( 0.25, 1.00)
Treatment
Pre Post Post–Pre
61 58 58
9.361 9.431 0.086
1.225 0.975 1.328
5 7 3
11 11 3
10 10 0
Disease
Pre Post Post–Pre
67 61 61
7.46 8.49 1.08
1.55 1.16 1.82
3 4 4
11 11 5
7 9 1
(6, 9) (8, 9) (0, 2)
Diagnosis
Pre Post Post–Pre
61 59 58
7.18 7.46 0.29
1.51 1.32 1.85
4 3 4
10 9 3
7 8 0
(6, 9) (7, 8) ( 1, 2)
a
Note: A positive number for ‘Post–Pre’ indicates there was an improvement in scores following training. Inter-quartile range give values for the 1st and 3rd quartiles of the data.
Table 5
Significant Post–Pre differences
Sections
Timepoint
Difference in means
95% CI of difference
P-value
General issues
Post 1–Pre Post 2–Pre Post 2–Post 1
0.18 0.50 0.41
( 0.28, 0.64) (0.06, 0.94) (0.06, 0.76)
Nursing issues
Post 1–Pre Post 2–Pre Post 2–Post 1
3.24 3.89 0.55
(2.54, 3.94) (3.22, 4.56) (0.06, 1.03)
<0.001 <0.001 0.029
Treatment Disease Diagnosis
Post–Pre Post–Pre Post–Pre
0.09 1.08 0.29
( 0.26, 0.44) (0.62, 1.55) ( 0.19, 0.78)
0.623 <0.001 0.231
0.434 0.023 0.023
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
Table 6
21
Programme monitoring responses
Did you enjoy the course Was the course content relevant to your area of practice Do you feel you will be able to utilise the knowledge gained from the course in your own practice Did you feel adequately supported by your line manager Did you feel there was sufficient study time allocated to the course Did you have enough time to complete personal objectives
professional qualification (using ANOVA) nor those with and without a cancer-related qualification (using two-sample t-tests).
Discussion The aim of this pilot study was to assess the benefits of an education programme for nurses involved in the care of patients with colorectal cancer. The results from the evaluation are mixed in terms of overall improvement. However, positive changes were identified in relation to the number of correct responses to the best practice statements which reached statistical significance. While caution is needed in claiming statistical significance given the small sample size the results are encouraging and were sustained at one and four months after completion of the course. In recent years, one of the key quality drivers for cancer service improvement is to focus on the whole patient journey, demonstrate what needs to change at different stages, and target these areas (NHS Modernisation Agency, 2005; Scottish Executive, 2005). To implement change we need to understand the ‘pathway’. The results from the current study can be viewed as heartening with the participants reporting a greater pathway understanding. Further fieldwork would be important to be able to substantiate these findings. A further encouraging finding from this study was the positive attitudes towards cancer displayed by the participants throughout and up to four months post completion of the programme, reaching statistical significance. This may be due to the fact this was a self-select group and therefore the participants were generally more positive about the disease and proactive in their learning. However, these results are in contrast to previous work on nurses’ attitudes towards cancer, in particular the seminal work conducted by Corner (1993) on newly registered nurses’ attitudes to cancer prior to an educational intervention. In more
Yes n (%)
No n (%)
52 (100%) 50 (96%) 51 (98%)
2 (4%) 1 (2%)
46 (88%) 36 (69%)
6 (12%) 16 (32%)
28 (54%)
21 (40%)
Don’t know n (%)
3 (6%)
recent years there has been a shift in how cancer is perceived with a much greater emphasis on cancer being viewed as a chronic rather than acute illness, with people living longer. In addition there is better media coverage on the benefits of cancer treatments. Equally patients themselves are taking a much more active role in decisions about their management and are working in partnership with healthcare professionals to shape cancer services. This positive shift may explain why the participants in the present study held more positive attitudes to cancer in general. The results from the knowledge quiz style audit questionnaires did not produce the same significant improvements and were mixed. There was a statistically significant improvement in diseaserelated knowledge but no significant change in the number of correct answers given in the sections ‘treatment’ and ‘diagnosis and staging’. Furthermore there were no significant differences in the change of correct answers by nursing grade, professional qualification or cancer-related qualification. There are a number of possible explanations for this. First, on the whole the participants started out with relatively high knowledge levels. They were all working in an area where patients with bowel cancer were being managed (i.e. endoscopy, surgery etc) and had a median of 12 years (inter-quartile range 5, 23) clinical experience. One could argue that this is encouraging and that the study sample was a particularly well informed group. Second, the participants were self-selecting and, while no assumptions can be made, were motivated and had been proactive in their learning before the course commenced. However, a further explanation for this mixed response may in fact relate to the TELER methodology. As previously discussed a ‘don’t know’ response is heavily weighted with a score of 4 as a neutral response implies that the respondent does not have appropriate knowledge but is unlikely to act in a manner that would impede clinical effectiveness. When considering the group index
22 of achievement scores for ‘diagnosis and staging’ shown in Table 2 at first glance the participants ‘index of achievement’ scores appear to have declined post intervention. When the researchers then reviewed the raw data for patterns of change, the individual number of ‘correct’ responses had in-fact marginally increased from 438 pre-test to 440 post-test. In addition, the total number of ‘don’t know’ responses pre test (n = 89) dropped to 19, while the total number of ‘incorrect’ responses increased from 82 pretest to 130 post-test, giving an increase of 48 responses receiving no score post-test. Arguably this might explain the potential reason why the index of achievement score dropped in this section. It also raises the question as to whether this methodology was appropriate for this study. In Grocott et al.’s (2001a,b) papers she highlights that documentary evidence is essential to differentiate between intentions and assumptions about the delivery of care. Behavioral skills were not evaluated in the current study and therefore the researchers could not assess whether changes in knowledge related to either an increase or decline in skill level. Whist this was a study limitation, this is the first documented attempt at applying the TELER method as a repeated measure following a cancer educational programme. Furthermore, when analysing the ‘correct’ responses only, there was a significant improvement in best practice statements for nursing, general issues and the disease sections. This improvement in nursing and general issues was maintained four months after completion of the course and highlights the potential impact this education programme has on nurses’ practice. While the authors acknowledge that by conducting an independent analysis of ‘correct’ responses only there is a potential to introduce bias the justification for this relates to the original question of interest, namely whether the participants knowledge-base on colorectal cancer improved following the programme. It must also be acknowledged this was a pilot and future studies could extend the work to include assessing clinical skills through documented evidence, repeated measures of audit and observation of practice. Finally, the feedback from the participants was very positive. It is widely recognized that many nurses wishing to access education and training courses can find it difficult to agree time out from the work place (RCN 2003) and there is a need for more flexible approaches to learning (Scottish Executive Health Department, 1999; Levett-Jones, 2005). The majority of the participants in the present study were able to attend
G. Knowles et al. all four study days although it is acknowledged that 32% (n = 16) were unable to utilise the 2 h work-time period and 40% (n = 21) did not feel they had sufficient time to complete their personal objectives by the end of the programme. Certainly the participants in this study positively evaluated the balance between self-directed learning, insofar as managing their time and setting their own objectives, but also benefited from the study days giving them the opportunity for discussion and networking with colleagues throughout Scotland. Consequently this educational approach could be considered and applied for other site-specific patient groups in the future both Nationally and Internationally.
Conclusion The aim of this pilot study was to prepare nurses in the management of colorectal cancer and their understanding of the pathway of care. While the systematic audit has produced mixed results, improvements in the participants knowledge-base of the care pathway for this patient group were identified. While we cannot say whether this improvement led to progression in skill, the participants’ knowledge level was maintained four months after completion of the course and this provides us with a greater understanding of the potential impact that education programmes have on nurses’ clinical practice. Furthermore, this study has shown that nurses’ attitudes towards cancer are more positive than have been previously reported and this in itself may impact on their approach to practice. In carrying out this study it has again highlighted the inherent difficulties in trying to evaluate outcomes of education programmes in terms of clinical outcomes. However, this is the first attempt at evaluating the benefits of a nursing education programme concerned with colorectal cancer care across the whole patient pathway and indeed the first study to apply TELER methodology as a repeated measure of audit. Extension of this work and it’s application in clinical practice should be considered. Finally, the flexibility of this education programme provided the participants with a balance between self-directed and formal learning and was highly evaluated. The authors recommended this approach as a possible framework for other site-specific cancer groups interested in developing a foundation level programme for nurses that could be extended to include allied health professions.
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
Acknowledgement The authors would like to thank the following people for the invaluable support: All the participants in the study. Big Lottery for funding the project. Steering Group Members. Lead Colorectal Cancer Nurses Group. Patriona Grocott for her advice at the development stage. Mr AA Le Roux for his advice and support with the TELER system of audit. Cat Graham Wellcome Trust, Edinburgh for her statistical support.
References Corner, J., 1993. The impact of nurses’ encounters with cancer on their attitudes towards cancer. J. Clin. Nurs. 2, 363–372. Corner, J., Wilson-Barnett, J., 1992. The newly registered nurse and the cancer patient: an educational evaluation. Int. J. Nurs. Stud. 29 (2), 177–190. Dalton, J.A., Bernard, S., Blau, W., Johnston, C., Mann, J.D., Germino, B., Pierce, S., Toomey, T., Mooney, D., Carlson, J., 1995. Managing cancer pain: content and scope of an educational program for nurses who work in predominantly rural areas. J. Pain Sympt. Manage. 10 (3), 214–220. Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001a. Nursing in colorectal cancer initiative – the audit phase. Part 1. Development of the audit tool. Eur. J. Oncol. Nurs. 5 (2), 100–111. Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001b. Nursing in colorectal cancer initiative: the audit phase. Part 2. Content validity of the audit tool and implications of the standards set for clinical practice. Eur. J. Oncol. Nurs. 5 (3), 165–173. Haley, H.B., Juan, I.R., Galen, J.F., 1968. Factor-analytic approach to attitude scale construction. J. Med. Educ. 43 (3), 331–336. Haley, H.B., Huynh, H., Paiva, R.E.A., Juan, I.R., 1977. Student attitudes towards cancer changes in medical school. J. Med. Educ. 52 (6), 500–507. Hawthorn, J., Redmond, K., 1999. A Guide to Colorectal Cancer. AstraZeneca Oncology, UK. Hooker, L., Milburn, M., 2000. Taking practice forward in paediatric oncology: the impact of a newly developed education programme for nurses working in shared-care hospitals. Eur. J. Oncol. Nurs. 4 (1), 48–54. Howell, D., Butler, L., Vincent, L., Watt-Watson, J., Stearns, N., 2000. Influencing nurses’ knowledge, atti-
23
tudes, and practice in cancer pain management. Cancer Nurs. 23 (1), 55–63. Information and Statistical Division, 2000. Trends in Cancer Survival in Scotland 1971–1995. Information and Statistical Division, Edinburgh. Jorden, S., Colemen, M., Hardy, B., Hughes, D., 1999. Assessing educational effectiveness the impact of a specialist course on the delivery of care. J. Adv. Nurs. 30 (4), 796–807. Lasch, K.E., Wilkes, G., Lee, J., Blanchard, R., 2000. Is hands-on experience more effective than didactic workshops in postgraduate cancer pain education? J. Cancer Educ. 15, 218–222. Le Roux, A.A., 1995. TELER: the concept. Physiotherapy 79 (11), 755–758. Le Roux, A.A., 2003. The TELER Handbook. TELER Ltd., Sheffield. Levett-Jones, T.L., 2005. Self directed learning implications and limitations for undergraduate nursing education. Nurs. Educ. Today 25 (5), 363–368. Loftus, L.A., Thompson, E., 2002. An evaluation of a palliative care course for generic nurses. Int. J. Palliative Nurs. 8, 354– 360. McClement, S.E., Care, D., Dean, R., 2005. Evaluation of Education in Palliative Care: Determining the effects on nurses’ knowledge and attitudes. J. Palliative Care 21 (1), 44–48. NHS Education Scotland, 2003a. Assessment of the Education and Training Needs Within Cancer Clinical Networks in Scotland. NES. NHS Modernisation Agency, 2005. Applying High Impact Changes to Cancer Care. Cancer Services Collaborative Improvement Partnership. NHS Modernisation Agency, Leicester. Plymale, M.A., Sloan, P.A., Johnson, M., LaFountain, P., Snapp, J., Vanderveer, B., Sloan, D.A., 2001. Cancer pain education: a structured clinical instruction modules for hospice nurses. Cancer Nurs. 24 (6), 424–429. Executive, Scottish, 2005. Illustrating ‘‘How to’’ Implement the Top 20 Actions for Change. Cancer Service Improvement Programme. Scottish Executive, Edinburgh. Scottish Executive Health Department, 1999. Learning Together: A Strategy for Education, Training and Lifelong Learning for all in the National Health Service in Scotland. SEHD, Edinburgh. Scottish Executive Health Department, 2001a. Cancer in Scotland: Action for Change. SEHD, Edinburgh. Scottish Executive Health Department, 2004a. Nursing People with Cancer in Scotland: A Framework. SEHD, Edinburgh. Scottish Executive Health Department, 2005. A National Framework for Service Changes in the NHS in Scotland: Building a Health Service Fit for the Future. SEHD, Edinburgh. Steginga, S.K., Dunn, J., Dewar, A.M., McCarthy, A., Yates, P., Beadle, G., 2005. Impact of an intensive nursing education course on nurses’ knowledge, confidence, attitudes, and perceived skills in the care of patients with cancer. Oncol. Nurs. Forum 32 (2), 375–381.
Available online at www.sciencedirect.com
Nurse Education Today intl.elsevierhealth.com/journals/nedt
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland Gillian Knowles a,*,1, Catherine Hutchison b,2, Graeme Smith c, Iona D. Philp a, Katrina McCormick a, Elizabeth Preston a a
Edinburgh Cancer Centre, Western General Hospital, Crewe Road, Edinburgh EH4 2XU, United Kingdom North Glasgow University Hospitals, NHS Division, Administration Building, Western Infirmary, Dumbarton Road, Glasgow G11 6NT, United Kingdom c School of Health, University of Edinburgh, Teviot Place, Edinburgh, United Kingdom b
Accepted 4 February 2007
KEYWORDS
Summary This paper describes the results of an evaluation of a nursing education programme for nurses caring for patients with colorectal cancer. A sample of 67 registered nurses from 6 location sites within the 3 Regional Cancer Networks in Scotland were recruited to the pilot. The programme was adapted from an evidence-based education manual for nurses in the management of colorectal cancer developed by the European Oncology Nursing Society as part of a Nursing in Colorectal Cancer Initiative (NICCI) [Hawthorn, J., Redmond, K., 1999. A Guide to Colorectal Cancer. AstraZeneca Oncology, UK]. The format for evaluating the programme was based on the TELER method of treatment evaluation [Le Roux, A.A., 1995. TELER: the concept. Physiotherapy 79 (11), 755–758] that had previously been developed along side the training manual [Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K, 2001a. Nursing in colorectal cancer initiative – the audit phase. Part 1. Development of the audit tool. European Journal of Oncology Nursing 5 (2), 100–111; Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001b. Nursing in colorectal cancer initiative: the audit phase. Part 2. Content validity of the audit tool and implications of the standards set for clinical practice. European Journal of Oncology Nursing 5 (3), 165–173] for cytotoxic
Nurse education; Cancer; Colorectal cancer; Evaluation; Clinical outcomes
* Corresponding author. Tel.: +44 131 244 3291. E-mail addresses: [email protected] (G. Knowles), [email protected] (C. Hutchison). 1 Currently on secondment as Nurse Consultant, Scottish Executive, St Andrew’s House, Edinburgh. 2 Tel.: +44 141 211 2344.
0260-6917/$ - see front matter c 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.nedt.2007.02.002
16
G. Knowles et al. chemotherapy to provide the NICCI Audit Tool (Le Roux, 2003). This model was developed further in the current study to include the domains of: Disease, Diagnosis and Staging, Treatment, Nursing Issues and General Issues. Data were analysed descriptively and are discussed. Overall the results from this study demonstrate a statistically significant improvement in disease-related knowledge (p = <0.001) and in the best practice statements for nursing issues (p = <0.001) and general issues (including attitudes) (p = 0.023) that were maintained at four months post completion of the course. c 2007 Elsevier Ltd. All rights reserved.
Introduction Colorectal cancer constitutes one of the most common cancers and is the second cause of cancer death in Scotland (Information and Statistical Division, 2000). The pathway for patients is complex and care is delivered in many different settings. Therefore educational preparation in the management of colorectal cancer is important and relevant for nurses working within various hospital departments and community settings. The aim of service provision is to ensure the smooth pathway for patients across the various care settings and the delivery of high quality care. For this to happen effectively nurses need to have core knowledge and skills in the management of the disease and a clear understanding of the interlinked components of service delivery. This paper describes the implementation and evaluation of a pilot education programme for nurses caring for patients with colorectal cancer across 6 location sites in Scotland.
Background/literature One of the key drivers for health policy in the NHS is to ensure consistent and high quality care for patients with cancer (Scottish Executive Health Department, 2004a, 2001a; NHS Education Scotland, 2003a; Scottish Executive Health Department, 2005). There has been a major shift in how we manage cancer services in Scotland through the development of Managed Clinical Networks, bringing together healthcare professional and patient representatives with the aim of ensuring consistent and seamless care for patients with specific tumour types (Scottish Executive Health Department, 2001a). These key policy documents in addition to clinical and professional guidelines that focus on best practice are influencing ways in which we support health professionals to develop the necessary knowledge and skills to meet the challenges of a modern NHS in Scotland. With this in mind, education and continuing professional development for all healthcare professionals are
therefore very important. However how do we ensure that an intended education programme is effective in influencing nursing practice? Evaluating clinical effectiveness as a result of an educational intervention is inherently difficult due to the numerous variables that can influence clinical outcomes (Jordon et al., 1999). However, with an increasing emphasis on continuing professional development stakeholders need reliable evidence that their investment has been worthwhile. While there remains a lack of empirical data supporting the relationship between education and clinical effectiveness there have been a number of general cancer nursing studies demonstrating improvements in cancer-related knowledge and attitudes as a result of education (Corner and Wilson-Barnett, 1992; Corner, 1993; Hooker and Milburn, 2000; Lasch et al., 2000; Loftus and Thompson, 2002; McClement et al., 2005). In addition, a number of studies have been conducted to identify the participants’ perceptions of how their practice changed as a result of an educational intervention and have been evaluated positively (Steginga et al., 2005; Howell et al., 2000). The evaluations of education and training programmes in cancer nursing have been approached in a variety of ways. Primarily through the use of pre and post-test questionnaires, cancer attitudes scales and focus groups interviews. Many of these studies have taken one aspect of cancer or palliative care to conduct and evaluate an intervention, for example pain management (Howell et al., 2000; Plymale et al., 2001; Dalton et al., 1995). However as yet there are no published data specifically evaluating education interventions aimed at providing nurses with a improved understanding of the interlinked components of service delivery in colorectal cancer management. With greater emphasis on sub-specialisation and cross boundary working (Scottish Executive Health Department, 2005), the aim of the current study was to pilot and evaluate an education programme for nurses caring for patients with colorectal cancer across different care settings throughout Scotland. Identifying an evaluation strategy that would be sensitive
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
to measuring changes in knowledge and understanding of the pathway of care for patients with bowel cancer was an important consideration and proved challenging. There were no ‘off-the shelf’ tools available with the exception of a recent audit tool that had been developed to measure standards of care in relation to the delivery of cytotoxic chemotherapy to patients with advanced colorectal cancer (Grocott et al., 2001a; Grocott et al., 2001b) using the TELER system of audit (Le Roux, 1995). Initially developed to measure the effectiveness of the treatment received by a patient by determining the clinical and statistical significance of the improvement experienced by the patient, the TELER methodology was further developed by Grocott et al. (2001a,b) to measure the clinical performance of a nurse in the delivery of cytotoxic chemotherapy with the potential to measure changes in performance when repeated. Expansion and adaptation of the TELER framework to include all aspects of colorectal cancer management was considered to be an appropriate approach in the evaluation of the present study (personnel correspondence with Patricia Grocott and AA Le Roux, April 2002).
Study objectives The overall aim of the study was to improve the understanding of the care pathway for patients with colorectal cancer and the interlinked components of service delivery in a group of registered nurses. The specific objectives were: To provide a comprehensive educational basis for registered nurses in the management of colorectal cancer through the implementation of an evidence-based education programme. To evaluate knowledge, general cancer nursing issues and best practice statements prior to, and after the implementation of the programme. To evaluate how site-specific cancer nursing education impacts on nurses perceptions of changes in their practice 4 weeks and 4 months post course completion (based on the literature and timeframe for the study).
Methods Design and procedure The study was designed around an evidence-based education manual for nurses in the management of colorectal cancer developed by the European Oncology Nursing Society as part of a Nursing in
17
Colorectal Cancer Initiative (NICCI) (Hawthorn and Redmond, 1999). The original material, referred to as the NICCI manual, was developed in an open-learning format, divided into four chapters to cover the topics of colorectal disease, diagnosis and staging, treatment, and nursing care. Each section contains learning objectives and a number of learning activities that the readers are encouraged to undertake as they progress through the manual. Permission to use and adapt the NICCI manual for the purpose of the current study was obtained from the European Oncology Nursing Society. The manual was supplemented with prepared up-to-date research evidence and each section was introduced through a series of four education study days over a five-month period. The self-directed component of the programme required the participants to work through each section of the manual and supplementary material, setting their own personal objectives. Management approval was granted from the 6 location sites to enable the participants to participate in the pilot, attend the four study days along with two hours per week for personal study. A Steering Group consisting of three Lead Colorectal Cancer Nurse Specialists, a Nurse Consultant, an Assistant General Manager, a Nurse Educationalist, a Colorectal Surgeon and a Professor of Medical Oncology managed the overall study. Two project managers managed the day-to-day organisation of the study.
Sample and setting A total sample of 67 registered nurses from 6 location sites within the 3 Regional Cancer Networks in Scotland were recruited to the pilot. As bowel cancer is managed in both acute and primary care and across surgical, medical and oncology settings, the location sites were selected for inclusion in the project to reflect this. The participants were selfselecting following a series of advertisements and through communication from local planning teams comprising of the Colorectal Clinical Nurse Specialists based in each of the location sites. There were no exclusion criteria with the exception that the participants required to be working with patients with colorectal cancer. It was also made explicit in the advertisement that the course was aimed at providing core level knowledge to inform the pathway of care.
Instruments As part of the Nursing in Colorectal Cancer Initiative the TELER audit was developed along side
18
G. Knowles et al.
the training manual (Grocott et al., 2001a,b). As previously discussed the audit tool was developed to measure standards of nursing care in relation to the delivery of cytotoxic chemotherapy to patients with advanced colorectal cancer. The TELER methodology provides an established format for the construction of the evaluation questions, the TELER indicator, which reduces random response error (Le Roux, 1995; Le Roux, 2003). Permission to use and develop the TELER framework was granted under licence to the author, A.A. Le Roux. Development of pre and post education intervention quiz-style questionnaires included the following domains: The Disease (incidence, aetiology, pathology, genetics, prevention and screening). Diagnosis and Staging. Treatment. Nursing Issues (colorectal cancer best practice statements). General Issues. The questionnaires covered each chapter of the manual and along with best practice statements. These statements related to the participants’ perceptions of how comfortable and confident they felt in communicating, informing and supporting patients with colorectal cancer throughout the pathway and their role and contribution within the multidisciplinary team. In addition, cancer attitude questions were incorporated in the General Issues audit questionnaire adapted from the Cancer Attitude Scale (Haley et al., 1968; Haley et al., 1977). Corner (1993) suggests that attitudes may affect the quality of care that nurses provide to patients, with positive attitude scores linked to the nurse’s ability to effectively care for patients with cancer and their family
Table 1
members. The aims of the pre- and post-test questionnaires were to evaluate the impact of the course on participant’s knowledge, attitudes and practice once the relevant study days and self-directed reading were complete (see Table 1). The audit tools were developed by a Colorectal Cancer Nurse Specialist and Oncology Nurse Consultant and subject to peer review for content validity by four Lead Colorectal Nurse Specialists, a Nurse Researcher and a Nurse Educationalist. A demographic questionnaire was devised specifically for the study to document participant details. In addition, a programme monitoring questionnaire was completed to assess the participants overall evaluation of the course.
Data analysis Using TELER methodology the level of statistical significance was set at <0.05. Responses to the quiz style statements were scored as follows: Correct = 6, Don’t know = 4, Incorrect = O. TELER methodology states a correct response to a statement is made by a respondent who possesses the appropriate knowledge, and is aware of the extent and limitations of that knowledge (score of 6). A neutral response to a statement is made by a respondent who does not possess the appropriate knowledge and is aware of the extent and limitation of that knowledge. A neutral response implies that the respondent is unlikely to act in a manner that would impede the effectiveness of a patient’s treatment or care and is therefore weighted highly (score of 4). Finally, an incorrect response to a statement is made by a respondent who does not possess the appropriate knowledge and is unaware of the extent and limitations of that lack of knowl-
Time points for completion of quiz style audit questionnaire
Time points
Questionnaire completion
Study Day 1 – The disease
General issues (pre-test) The disease (pre-test) Nursing issues (pre-test) The disease (post-test) Diagnosis & Staging (pre-test) Diagnosis & Staging (post-test) Treatment (pre-test) Treatment (post-test) General issues (post-test) Nursing issues (post-test) General issues (post-test) Nursing issues (post-test)
Study Day 2 – Diagnosis & Staging Study Day 3 – Treatment Study Day 4 – Nursing issues 4 weeks post course completion 4 months post course completion
19
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
edge. Consequently an incorrect response is awarded a score of 0. In other words, an incorrect response is expected to be equal and opposite to that of a correct response and potential effect of a ‘don’t know’ response is considered to be negligible (Le Roux, 2003). Individual participant scores for each audit questionnaire were awarded an unacceptable, acceptable or excellent score. From these scores a target of 90% index of achievement was set for the group. The index of achievement determines the percentage of the group that attain ‘at least’ acceptable score for each questionnaire rather than individual scores. The research team agreed the outcome of interest was change in the number of questions answered correctly for each of the five pre and post audit questionnaires and also individual correct responses. Using Minitab further data analysis was conducted by an independent statistician where scoring included ‘correct’ and ‘incorrect’ responses only. Comparisons between results were made using analysis of variance (ANOVA) where there were more than two groups and two-sample t-test where there were only two groups. Both these methods determine if there are statistically significant differences between the means of either two groups (t-test) or between the means of all the group (ANOVA).
Results A total of 67 registered nurses were recruited to the study. Fifty-two participants completed the course and 47 participants successfully returned their follow-up evaluation audit questionnaire at 4 months post course completion. Of the 53 participants who completed the course, 2 considered the course to be at too low a level to meet their requirements, 10 were unable to get time off to attend and in 3 cases the reasons were not documented. Only 9% (n = 6) were male; 62.1% (n = 41) of the sample were in full-time occupation; median
Table 2
number of years qualified was 12 (inter-quartile range 5, 23); and years spent in present post 3 (inter-quartile range 1, 7.25). The distribution of grade and professional qualification are seen in Table 2. Only 13.4% (n = 9) of the participants had a cancer-related qualification. Forty-one percent (n = 28) were educated to degree level. Table 3 provides the index of achievement scores in relation to the five identified domains. These scores are presented as pre and post scores for the study population. It is important to note that the percentages are not measuring differences between pre- and post-test scores but rather the percentage of the group that have reached an ‘acceptable or above’ level of achievement at that given time. TELER methodology does not allow for changes in the individual number of questions answered correctly pre- and post-test. As this was the main outcome of interest to this study further analysis was conducted. Table 4 shows the mean (SD) of the number of correct answers in each section for all cases overall, along with the minimum, median and inter-quartile range. Looking at the results overall to see if there is a significant ‘Post–Pre’ change, paired t-tests have been used. Table 5 summaries the results. From this it can be seen there was no significant change in the number of correct answers given for either treatment or diagnosis and staging. However, for disease, nursing issues and the second general issues questionnaire there was a significant improvement. Table 6 provides the summary results from the programme monitoring questionnaire on completion of the course (n = 52 returns). Looking at the results, all of the participants enjoyed the programme, reporting that the content was relevant and that they would utilize the knowledge gained in their own area of practice. Finally, there were no differences in the change in number of correct answers by nursing grade and
Distribution of grade and professional qualification
Nursing grade
Number (n = 67)
%
Professional qualification
Number (n = 67)
%
EN S/N D Grade S/N E Grade S/N F Grade Charge Nurse
3 29 9 10 16
4.45% 43.3% 13.4% 14.9% 23.9%
RGN Diploma BSc BA MSc Cancer-related qualification*
23 16 19 8 1 9
34.3% 23.9% 28.4% 11.9% 1.5% 13.4%
*
Note as some nurses have multiple qualifications percentages may not total 100.
20
G. Knowles et al.
Table 3
Index of achievement
Index of achievement: (required 90% of group)
N
Pre-test (%)
N
Post-test (%)
N
2nd Retest
The disease Diagnosis and staging Treatment General issues Nursing issues
67 61 61 67 67
80 72 98 98 23
61 59 58 51 51
84 60 98 99 81
N/A N/A N/A 47 47
N/A N/A N/A 99 91
Table 4
Descriptive statistics of number of correct answers for each section
Number of correct answers for each section General issues
Nursing issues
N
Mean
SD
Minimum
Maximum
Median
Inter-quartile rangea
Pre Post Post Post Post Post
1 1–Pre 2 2–Pre 2–Post 1
67 51 50 47 46 44
10.69 10.73 0.18 11.04 0.50 0.41
1.21 1.04 1.61 0.91 1.49 1.15
7 7 4 9 2 2
12 12 4 12 4 4
11 11 0 11 0 0
(10, 12) (10, 12) ( 1, 1) (10, 12) ( 0.25, 1.25) (0, 1)
Pre Post Post Post Post Post
1 1–Pre 2 2–Pre 2–Post 1
67 51 50 47 46 44
5.93 9.14 3.24 9.66 3.89 0.55
2.40 1.77 2.46 1.51 2.25 1.61
1 6 3 6 0 5
11 12 10 12 10 4
6 9 3 10 4 1
(4, 7) (8, 11) (2, 5) (8, 11) (2.75, 5.00) (0, 1) (9, 10) (9, 10) ( 0.25, 1.00)
Treatment
Pre Post Post–Pre
61 58 58
9.361 9.431 0.086
1.225 0.975 1.328
5 7 3
11 11 3
10 10 0
Disease
Pre Post Post–Pre
67 61 61
7.46 8.49 1.08
1.55 1.16 1.82
3 4 4
11 11 5
7 9 1
(6, 9) (8, 9) (0, 2)
Diagnosis
Pre Post Post–Pre
61 59 58
7.18 7.46 0.29
1.51 1.32 1.85
4 3 4
10 9 3
7 8 0
(6, 9) (7, 8) ( 1, 2)
a
Note: A positive number for ‘Post–Pre’ indicates there was an improvement in scores following training. Inter-quartile range give values for the 1st and 3rd quartiles of the data.
Table 5
Significant Post–Pre differences
Sections
Timepoint
Difference in means
95% CI of difference
P-value
General issues
Post 1–Pre Post 2–Pre Post 2–Post 1
0.18 0.50 0.41
( 0.28, 0.64) (0.06, 0.94) (0.06, 0.76)
Nursing issues
Post 1–Pre Post 2–Pre Post 2–Post 1
3.24 3.89 0.55
(2.54, 3.94) (3.22, 4.56) (0.06, 1.03)
<0.001 <0.001 0.029
Treatment Disease Diagnosis
Post–Pre Post–Pre Post–Pre
0.09 1.08 0.29
( 0.26, 0.44) (0.62, 1.55) ( 0.19, 0.78)
0.623 <0.001 0.231
0.434 0.023 0.023
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
Table 6
21
Programme monitoring responses
Did you enjoy the course Was the course content relevant to your area of practice Do you feel you will be able to utilise the knowledge gained from the course in your own practice Did you feel adequately supported by your line manager Did you feel there was sufficient study time allocated to the course Did you have enough time to complete personal objectives
professional qualification (using ANOVA) nor those with and without a cancer-related qualification (using two-sample t-tests).
Discussion The aim of this pilot study was to assess the benefits of an education programme for nurses involved in the care of patients with colorectal cancer. The results from the evaluation are mixed in terms of overall improvement. However, positive changes were identified in relation to the number of correct responses to the best practice statements which reached statistical significance. While caution is needed in claiming statistical significance given the small sample size the results are encouraging and were sustained at one and four months after completion of the course. In recent years, one of the key quality drivers for cancer service improvement is to focus on the whole patient journey, demonstrate what needs to change at different stages, and target these areas (NHS Modernisation Agency, 2005; Scottish Executive, 2005). To implement change we need to understand the ‘pathway’. The results from the current study can be viewed as heartening with the participants reporting a greater pathway understanding. Further fieldwork would be important to be able to substantiate these findings. A further encouraging finding from this study was the positive attitudes towards cancer displayed by the participants throughout and up to four months post completion of the programme, reaching statistical significance. This may be due to the fact this was a self-select group and therefore the participants were generally more positive about the disease and proactive in their learning. However, these results are in contrast to previous work on nurses’ attitudes towards cancer, in particular the seminal work conducted by Corner (1993) on newly registered nurses’ attitudes to cancer prior to an educational intervention. In more
Yes n (%)
No n (%)
52 (100%) 50 (96%) 51 (98%)
2 (4%) 1 (2%)
46 (88%) 36 (69%)
6 (12%) 16 (32%)
28 (54%)
21 (40%)
Don’t know n (%)
3 (6%)
recent years there has been a shift in how cancer is perceived with a much greater emphasis on cancer being viewed as a chronic rather than acute illness, with people living longer. In addition there is better media coverage on the benefits of cancer treatments. Equally patients themselves are taking a much more active role in decisions about their management and are working in partnership with healthcare professionals to shape cancer services. This positive shift may explain why the participants in the present study held more positive attitudes to cancer in general. The results from the knowledge quiz style audit questionnaires did not produce the same significant improvements and were mixed. There was a statistically significant improvement in diseaserelated knowledge but no significant change in the number of correct answers given in the sections ‘treatment’ and ‘diagnosis and staging’. Furthermore there were no significant differences in the change of correct answers by nursing grade, professional qualification or cancer-related qualification. There are a number of possible explanations for this. First, on the whole the participants started out with relatively high knowledge levels. They were all working in an area where patients with bowel cancer were being managed (i.e. endoscopy, surgery etc) and had a median of 12 years (inter-quartile range 5, 23) clinical experience. One could argue that this is encouraging and that the study sample was a particularly well informed group. Second, the participants were self-selecting and, while no assumptions can be made, were motivated and had been proactive in their learning before the course commenced. However, a further explanation for this mixed response may in fact relate to the TELER methodology. As previously discussed a ‘don’t know’ response is heavily weighted with a score of 4 as a neutral response implies that the respondent does not have appropriate knowledge but is unlikely to act in a manner that would impede clinical effectiveness. When considering the group index
22 of achievement scores for ‘diagnosis and staging’ shown in Table 2 at first glance the participants ‘index of achievement’ scores appear to have declined post intervention. When the researchers then reviewed the raw data for patterns of change, the individual number of ‘correct’ responses had in-fact marginally increased from 438 pre-test to 440 post-test. In addition, the total number of ‘don’t know’ responses pre test (n = 89) dropped to 19, while the total number of ‘incorrect’ responses increased from 82 pretest to 130 post-test, giving an increase of 48 responses receiving no score post-test. Arguably this might explain the potential reason why the index of achievement score dropped in this section. It also raises the question as to whether this methodology was appropriate for this study. In Grocott et al.’s (2001a,b) papers she highlights that documentary evidence is essential to differentiate between intentions and assumptions about the delivery of care. Behavioral skills were not evaluated in the current study and therefore the researchers could not assess whether changes in knowledge related to either an increase or decline in skill level. Whist this was a study limitation, this is the first documented attempt at applying the TELER method as a repeated measure following a cancer educational programme. Furthermore, when analysing the ‘correct’ responses only, there was a significant improvement in best practice statements for nursing, general issues and the disease sections. This improvement in nursing and general issues was maintained four months after completion of the course and highlights the potential impact this education programme has on nurses’ practice. While the authors acknowledge that by conducting an independent analysis of ‘correct’ responses only there is a potential to introduce bias the justification for this relates to the original question of interest, namely whether the participants knowledge-base on colorectal cancer improved following the programme. It must also be acknowledged this was a pilot and future studies could extend the work to include assessing clinical skills through documented evidence, repeated measures of audit and observation of practice. Finally, the feedback from the participants was very positive. It is widely recognized that many nurses wishing to access education and training courses can find it difficult to agree time out from the work place (RCN 2003) and there is a need for more flexible approaches to learning (Scottish Executive Health Department, 1999; Levett-Jones, 2005). The majority of the participants in the present study were able to attend
G. Knowles et al. all four study days although it is acknowledged that 32% (n = 16) were unable to utilise the 2 h work-time period and 40% (n = 21) did not feel they had sufficient time to complete their personal objectives by the end of the programme. Certainly the participants in this study positively evaluated the balance between self-directed learning, insofar as managing their time and setting their own objectives, but also benefited from the study days giving them the opportunity for discussion and networking with colleagues throughout Scotland. Consequently this educational approach could be considered and applied for other site-specific patient groups in the future both Nationally and Internationally.
Conclusion The aim of this pilot study was to prepare nurses in the management of colorectal cancer and their understanding of the pathway of care. While the systematic audit has produced mixed results, improvements in the participants knowledge-base of the care pathway for this patient group were identified. While we cannot say whether this improvement led to progression in skill, the participants’ knowledge level was maintained four months after completion of the course and this provides us with a greater understanding of the potential impact that education programmes have on nurses’ clinical practice. Furthermore, this study has shown that nurses’ attitudes towards cancer are more positive than have been previously reported and this in itself may impact on their approach to practice. In carrying out this study it has again highlighted the inherent difficulties in trying to evaluate outcomes of education programmes in terms of clinical outcomes. However, this is the first attempt at evaluating the benefits of a nursing education programme concerned with colorectal cancer care across the whole patient pathway and indeed the first study to apply TELER methodology as a repeated measure of audit. Extension of this work and it’s application in clinical practice should be considered. Finally, the flexibility of this education programme provided the participants with a balance between self-directed and formal learning and was highly evaluated. The authors recommended this approach as a possible framework for other site-specific cancer groups interested in developing a foundation level programme for nurses that could be extended to include allied health professions.
Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland
Acknowledgement The authors would like to thank the following people for the invaluable support: All the participants in the study. Big Lottery for funding the project. Steering Group Members. Lead Colorectal Cancer Nurses Group. Patriona Grocott for her advice at the development stage. Mr AA Le Roux for his advice and support with the TELER system of audit. Cat Graham Wellcome Trust, Edinburgh for her statistical support.
References Corner, J., 1993. The impact of nurses’ encounters with cancer on their attitudes towards cancer. J. Clin. Nurs. 2, 363–372. Corner, J., Wilson-Barnett, J., 1992. The newly registered nurse and the cancer patient: an educational evaluation. Int. J. Nurs. Stud. 29 (2), 177–190. Dalton, J.A., Bernard, S., Blau, W., Johnston, C., Mann, J.D., Germino, B., Pierce, S., Toomey, T., Mooney, D., Carlson, J., 1995. Managing cancer pain: content and scope of an educational program for nurses who work in predominantly rural areas. J. Pain Sympt. Manage. 10 (3), 214–220. Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001a. Nursing in colorectal cancer initiative – the audit phase. Part 1. Development of the audit tool. Eur. J. Oncol. Nurs. 5 (2), 100–111. Grocott, P., Richardson, A., Ambaum, B., Kearney, N., Redmond, K., 2001b. Nursing in colorectal cancer initiative: the audit phase. Part 2. Content validity of the audit tool and implications of the standards set for clinical practice. Eur. J. Oncol. Nurs. 5 (3), 165–173. Haley, H.B., Juan, I.R., Galen, J.F., 1968. Factor-analytic approach to attitude scale construction. J. Med. Educ. 43 (3), 331–336. Haley, H.B., Huynh, H., Paiva, R.E.A., Juan, I.R., 1977. Student attitudes towards cancer changes in medical school. J. Med. Educ. 52 (6), 500–507. Hawthorn, J., Redmond, K., 1999. A Guide to Colorectal Cancer. AstraZeneca Oncology, UK. Hooker, L., Milburn, M., 2000. Taking practice forward in paediatric oncology: the impact of a newly developed education programme for nurses working in shared-care hospitals. Eur. J. Oncol. Nurs. 4 (1), 48–54. Howell, D., Butler, L., Vincent, L., Watt-Watson, J., Stearns, N., 2000. Influencing nurses’ knowledge, atti-
23
tudes, and practice in cancer pain management. Cancer Nurs. 23 (1), 55–63. Information and Statistical Division, 2000. Trends in Cancer Survival in Scotland 1971–1995. Information and Statistical Division, Edinburgh. Jorden, S., Colemen, M., Hardy, B., Hughes, D., 1999. Assessing educational effectiveness the impact of a specialist course on the delivery of care. J. Adv. Nurs. 30 (4), 796–807. Lasch, K.E., Wilkes, G., Lee, J., Blanchard, R., 2000. Is hands-on experience more effective than didactic workshops in postgraduate cancer pain education? J. Cancer Educ. 15, 218–222. Le Roux, A.A., 1995. TELER: the concept. Physiotherapy 79 (11), 755–758. Le Roux, A.A., 2003. The TELER Handbook. TELER Ltd., Sheffield. Levett-Jones, T.L., 2005. Self directed learning implications and limitations for undergraduate nursing education. Nurs. Educ. Today 25 (5), 363–368. Loftus, L.A., Thompson, E., 2002. An evaluation of a palliative care course for generic nurses. Int. J. Palliative Nurs. 8, 354– 360. McClement, S.E., Care, D., Dean, R., 2005. Evaluation of Education in Palliative Care: Determining the effects on nurses’ knowledge and attitudes. J. Palliative Care 21 (1), 44–48. NHS Education Scotland, 2003a. Assessment of the Education and Training Needs Within Cancer Clinical Networks in Scotland. NES. NHS Modernisation Agency, 2005. Applying High Impact Changes to Cancer Care. Cancer Services Collaborative Improvement Partnership. NHS Modernisation Agency, Leicester. Plymale, M.A., Sloan, P.A., Johnson, M., LaFountain, P., Snapp, J., Vanderveer, B., Sloan, D.A., 2001. Cancer pain education: a structured clinical instruction modules for hospice nurses. Cancer Nurs. 24 (6), 424–429. Executive, Scottish, 2005. Illustrating ‘‘How to’’ Implement the Top 20 Actions for Change. Cancer Service Improvement Programme. Scottish Executive, Edinburgh. Scottish Executive Health Department, 1999. Learning Together: A Strategy for Education, Training and Lifelong Learning for all in the National Health Service in Scotland. SEHD, Edinburgh. Scottish Executive Health Department, 2001a. Cancer in Scotland: Action for Change. SEHD, Edinburgh. Scottish Executive Health Department, 2004a. Nursing People with Cancer in Scotland: A Framework. SEHD, Edinburgh. Scottish Executive Health Department, 2005. A National Framework for Service Changes in the NHS in Scotland: Building a Health Service Fit for the Future. SEHD, Edinburgh. Steginga, S.K., Dunn, J., Dewar, A.M., McCarthy, A., Yates, P., Beadle, G., 2005. Impact of an intensive nursing education course on nurses’ knowledge, confidence, attitudes, and perceived skills in the care of patients with cancer. Oncol. Nurs. Forum 32 (2), 375–381.
Available online at www.sciencedirect.com