- Email: [email protected]
Evaluating the impact of continual professional education (ENB 941)
NurrrEducafron T&y fQ Longman Group
02606917190/0010-04281f10.00
(1990) 10,428-4% UK Ltd 1990
Evaluating the impact of continual professional education (ENB 941) Pat Hughes
It is often assumed that participation in continuing professional education (CPE) activities necessarily results in maintenance of competence and improvements in nursing practice. A literature review revealed that evaluation of CPE activities is frequently limited to the completion of forms at the conclusion of the offering, indicating the extent to which the objectives have been achieved and the general level of satisfaction of the participants. Very little attention seems to have been paid to the longer term effects of the CPE activity on the individual participant or on their ability to provide an enhanced level of nursing practice - the raison d’etre of CPE. This project was designed to explore the impact of ENB 941 on course participants. A group of 11 nurses and their six clinical managers were interviewed 10 months following course completion to discuss the effects of the course on the professional, educational and individual development of course participants and on their ability to improve nursing practice. Analysis of the interviews suggests that all group members used some of the opportunities available in the course to further their development. All clinical managers agreed that despite staffing and financial implications, the course provided value for money in terms of staff development and enhanced nursing practice.
carried
BACKGROUND ENB 941 Caring course
for Older
for qualified
nurses.
People is a 22-day This
course
was
developed as a joint venture between two health authorities and consists of a Briefing Day, two 2-week blocks and a Recall Day over a 4-month period. Evaluation group
and
of the extent individual
to which
objectives
are
course, met
Pat Hughes RGN Dip N PGCEA RNT RCNT Senior Nurse Manager/Education Bromley and Camberwell Department of Nursing Studies, 13 Grace Avenue, Maidstone, Kent, ME16 OBS (Requests for offprints to PH) Manuscript accepted 21 March 1990 428
out
evaluation
is
in a variety of each
block is completed This
assists the
development
of ways.
A written
day’s experience
of study
by every
member.
course
course
tutors
by providing
in curriculum
useful information
about both content and process at the end of each block. It also facilitates course members in clarifying the extent to which they are meeting their personal Evaluation throughout the course
objectives. also takes
place
informally
the blocks. At the end of each block members work in small groups to
evaluate the extent to which the course objectives are being achieved and to identify positive and negative aspects of the experience. The feedback from each small group is discussed and the comments
collated
to provide
a report
to the
NURSE EDUCATION
Curriculum Development Group. Using these approaches the course has been consistently well evaluated in terms of process and outcome with positive comments far outweighing any negative ones. The negative comments tend to reflect individuals’ responses to particular teaching styles and content. Despite the very positive light in which the course is viewed and the interest and enthusiasm it generates, a major question which has not been addressed using such methods of course evaluation is what impact, if any, the course has on the professional practice of course members. Such type of valuation has been variously identified as follow up evaluation, post implementation evaluation, behaviour evaluation and impact evaluation (Heick 198 1). Gosnell (1984) suggests that a major impediment to the implementation of this type of evaluation is the absence of an identifiable conceptual framework or model.
LITERATURE
REVIEW
CPE can encompass a variety of activities such as study days, conferences, short courses e.g. ENB 94 1, role development courses and other longer courses leading to recognised academic awards. The American Nurses Association suggests that: ‘Continuing education in nursing consists of planned learning experiences beyond a basic educational programme. These nursing experiences are designed to promote the development of knowledge skills and attitudes for the enhancement of nursing practice, thus improving health care to the public.’ (Cited NBS 1985) Ferrell(1988) suggests that CPE is of particular relevance to nursing ‘since the half life of nursing knowledge ranges from two to five years.’ Basic professional education is thus no longer sufficient for a life time of practice. Social, demographic e.g. the increasing proportion of elderly people in the population, epidemiological and technological changes dictate the necessity for continuing education. Whilst there is, as yet, no mandatory requirement in this country for nurses to participate in
TODAY
429
specific educational activities, the UKCC issued a Code of Professional Conduct in 1984 which states that Every nurse, midwife or health visitor is accountable for his or her own practice and in the exercise of professional accountability shall: 3) take every reasonable opportunity to maintain and improve professional knowledge and competence. Work is currently under way to determine the extent to which participation in educational activities should become a periodic mandatory requirement, thus implying that CPE will result in maintaining competence and improving practice. However, in Hutton’s 1987 review of the American literature she concluded that the data were inconclusive and that the central question of the relationship between continuing education and competence remains unanswered. In an attempt to identify an appropriate model for use with ENB 941 it was necessary to remain cognizant of Heick’s contention that: ‘whenever impact evaluation integral part of the totality development process and appendage or extension of gram.’ (Heick 198 1)
is used, it is an of the program not simply an any given pro-
That statement notwithstanding, it was decided to seek to identify a model which appeared to either complement the current evaluation process or to inform future initiatives in developing a more comprehensive framework for evaluation. To this end two specific models were considered - a four stage model described by Gosnell (Fig. 1) and a four component model described by Crotty and Bignell (1987) and Bignell and Crotty (1988). Within the model proposed by Gosnell, Stage I easily encompassed the extant evaluations pertaining to the participants’ perceptions, opinions and attitudes about ENB 941. Stage II could be accommodated by developing a more formalized approach to the assessment of course assignments. Stages III and IV however, would have required a level of sophisticated planning beyond the resources available for this study.
430
NURSE EDUCATION
TODAY
Outcome
-
Ftesulls
Evalualion
Fig 1 Conceptual model for evaluation of continuing nursing education
The model could more easily be used to provide
the
valuable information and insight if it was developed in conjuction with a particular CPE
improve
offering
i.e. if it had been incorporated
planning The
into the
of ENB 941.
model
proposed
was then considered modified form (Fig.
by Crotty
and Bignell
and selected for use in a 2). The model is under-
pinned by a theory which suggests that CPE programmes are comprised of four main components
-
professional,
individual development, ability to improve nursing ‘An improved knowledge educational development which in turn individual
educational
and the practice.
base will lead to the of the individual,
will lead to professional
development.
and
enhanced
These
of
practice.’
four
components
used by Crotty and Bignell of course planning
the
individual
(Crotty
to
& Bignell
appear
to have been
as an integral
thus facilitating
aspect
the develop-
ment of an evaluation pro-forma based on identified units of learning. Their evaluation was undertaken
6
months
completion
and
consisted
based on statements the course content.
following
course
of a questionnaire
of critical
functions
Each participant
from
was asked to
assess any changes they had noted in their effectiveness and efficiency for each of the 23 critical functions. They were also asked to comment or not the course
Profe/sional
Fig 2 ENB course leading to improved care
ability
nursing
1987).
and
This should result in
enhanced
had influenced
Individual Develoomenc
on whether them
in the
NURSE EDL’CATION
four previously
identified
components.
The par-
ticipants’ senior nurses were also asked to evaluate changes
in the students
using the same criteria. As the four components cational
and
enhanced
of professional,
edu-
development
and
to improve
had not been explicitly
perceived
responding
to be a desirable
rather
in order
in what they than perhaps
an honest manner.
individual
ability
role performance
ible in her attitude toward the responses to prevent interviewees
43 1
TODAY
nursing
practice
considered
in the plann-
the development
of an appro-
Data collection
was
Of the 13 course members, 11 agreed to participate in a taped interview. One had left the NHS 3
decided, therefore, to select a group who had completed the course within the previous year
months following course completion and could not be contacted and 1 had moved out of the area
and
to take up promotion
ing of the course, priate
pro
forma
to conduct
member
was
a taped
of the group
an unstructured
interview
with each
covering
each
(Cohen & Mannion
It was anticipated useful
It
and their manager
format
four components provide
problematic.
that
this
1985: 293).
approach
information
would
to illuminate
impact of the course on these participants inform
future
course
ods of evaluation. pleted the course
using of the
developments
and meth-
The group chosen 10 months
the
and to
from staff nurse to ward
sister on an elderly care ward. Of the 7 clinical nurse
managers
6 agreed
to participate
and
1
had left the area at the same time as her course member had changed jobs. Although some course members and some clinical managers were male, in the interests interviewees
of confidentiality
will be referred
all
to as she/her.
had com-
previously.
ANALYSIS Educational development
METHODOLOGY
The course
Pilot study
ledge
A pilot study was carried
out to test the appro-
priateness of the questions, to determine the time required for the interviews and to provide the interviewer
with an opportunity
and refine her technique. members
from other
together
groups
seconded
staff previously for this evaluation.
was indeed
use. The
Following
pertaining
but
in the group
an appropriate
questions
who had
and subsequently
study it was clear that an unstructured format
course
were interviewed
with a clinical nurse manager
whose staff were not represented chosen
to practice
Two previous
the pilot interview
approach
to
to professional
members
claimed
base had increased
particularly
in breadth.
forced
their
by
information
was a very significant reflects
The
feature
of professional
acute
and long
hospital members
for
the
worked
environments
stay elderly elderly
in modern
where
varying
degrees
examples this area.
course just reinforced All course members
highlighted interviewer
together to remain
the ENB discussion
with the need as non-committal
ill. Group
and
probing
importance of keeping firmly in mind during
and were
wards and a day
mentally
provide The course
members
and as the
drawn. The group included community nurses, nurses working on both day and night duty in
dial and therapeutic approaches practised. One course member
issues when
content,
backgrounds
clinical areas from which course
of
members
in discussion
development presented some difficulty to the previous course members so it was necessary to of professional
was rein-
breadth
by the course
not so much the course
wide range
and more
This statement
managers.
acquired
that their know-
in depth
traditional of custo-
to care were claimed ‘the
what I knew.’ identified further
edu-
941 was
cational input which they had sought as a direct
for the as poss-
of the members identified aspects of care which had been addressed in a necessarily limited
result of their participation
in the course.
Most
432
NURSE EDUCATlON
TODAY
manner on the course which they had followed up or were in the process of so doing e.g. by applying for courses and study days on topics such as Symptom Control and Counselling. TWO course members were continuing to seek out information and build up resource packs covering community facilities which are available for a particular client group and aids for people with physical disabilities. One course member was in the process of working her way through the Open University Package Caring for Older People P650 with a view to perhaps leading a group of junior staff through the package. One course member claimed to have gained considerably by learning how to use the library resources properly and during the course another member placed a regular order for a weekly nursing journal for the first time in her professional career. Some course members said their confidence in their ability to study had increased or been reinforced and this had stimulated them to apply for more academic courses in nursing e.g. Enrolled Nurse Conversion course, and for one person an Open University Foundation course. Four course members claimed that their experience of the course had increased their insight and abilities in helping learners. One course member had enthusiastically fulfilled the role of mentor to a member on a subsequent course and two other members declared their interest in acting as mentors in the future. One course member stated that the supportive non-judgemental milieu which surrounded the course was appropriate and much appreciated at the time but felt on reflection that more emphasis on grading and marking of assignments would have been helpful in terms of educational development. Two course members specifically mentioned an assignment which required them to carry out a teaching activity which was assessed by their peers, course tutors and their individual mentor as being particularly anxiety provoking at the time but of enormous value subsequently. Within this area of development the managers reinforced many of the comments which had been made and confirmed that course members had pursued further some areas of interest
apparently stimulated by the course. Two managers expressed interest in the value of exploring the response of their individual staff members to educational input. One manager emphasised the importance of incorporating the continuing identification of educational needs into the context of Individual Performance Review.
Professional
development
With reference to the impact of ENB 94 1 on the professional development of course members, the responses fell easily into four categories clarification of the role of the nurse in elderly care; effect on career plans; communication and relationship effects and knowledge of current professional issues. More than half of the group claimed to have achieved more clarity in their thinking about how the role of the qualified nurse in elderly care differed from that of other members of the caring team e.g. social workers or nursing auxilliaries. One clinical manager said that the clarification of roles that occurred in a clinical area following the course had had a major beneficial impact on nursing practice and on team spirit, as people acknowledged their differing contributions to patient care. Two group members who were RGN’s and have since commenced RMN training claimed to have been influenced in their decisions by their increased insight into the skills required to care for elderly people who are mentally ill. Some course members said that the course provided a useful base to progress their future career and promotion plans. One course member claimed not to have any career plans, but claimed increased commitment ‘to do what I’m doing very well’. All course members claimed to have developed greater understanding of their colleague’s roles. For some, this reinforced their previous choices e.g. the community nurses were clear that caring for patients in hospital held little attraction for them. None of the course members admitted to any decrease of interest or enthusiasm for working with elderly people. Four course members said they felt more confident of their contributions within the multi-
NURSE EDUCATION
disciplinary team. The course was also said to have had a positive effect on the relationship between course members and their managers. Some course members claimed to feel more confident and determined in communicating their needs and the needs of their patients to their managers. Three managers said that their course members had developed a more positive and creative approach to problem solving and demonstrated a deeper understanding of broader issues affecting the provision of services. One manager expressed concern over the apparent widespread acceptance by staff that there was only one system or channel of communication and that if that system failed no other approaches were possible. She stated that at least one course member had made significant progress in overcoming this attitude following the course. One member claimed to have felt particularly disconcerted on the course by the realisation that ‘those at the top don’t always know best and have clear views about what’s going on.’ Most course members claimed to have developed increased insight into the social context in which nursing care was being delivered. All course members commented on the opportunities presented to discuss relevant reports of the time e.g Griffiths report Community Care: Agenda for Action. One manger saw evidence of increased insight into and interest in planning for future services in one course member. ‘She seemed to suddenly grasp where we were coming from and trying to get to.’ One-manager claimed that a course member had demonstrated the use of research findings in clinical practice as a direct response to specific course content.
Individual development During the interviews three main issues came to the fore - the impact of the course on members as individuals; on their relationship with others and the problem of sustaining the positive effects of the course. All course members claimed to value the opportunity to take time out of the clinical areas ‘Just to stop and think’. The majority of
TODAY
433
members commented on how their communication skills had improved as a result of the course. One course member claimed to have ‘learned to sell my ideas better and, by giving praise, get the co-operation and increase the motivation of other staff.’ Another member said ‘I now hold my tongue a bit better and try other ways of getting what I want.’ More than half of the course members claimed to find it easier to discuss issues pertaining to elderly people with others at all levels of the organisation. One member said she values discussion and sharing of ideas with colleagues and rather than ‘try to score points’, she listens more. Most managers claimed to have seen evidence of more frequent constructive discussion among staff. One manager commented on a particular course member’s improved ability to follow ideas through into action. Eight course members claimed to feel more confident in their ability to contribute to patient care. All members claimed to feel less isolated and more motivated. One manager claimed that the course had ‘done wonders’ for a course member’s self-esteem and self-awareness because she felt she had done well on the course. Following on from these positive responses a major problem which was identified by all interviewees concerned the difficulties experienced in maintaining enthusiasm and motivation on return to clinical areas.
Nursing practice All interviewees claimed that a more individualised approach to patient care was evident to a greater or lesser degree in clinical practice. This manifested itself in a variety of ways including a re-organisation of nursing work. One member said that her ward ‘was now organised to meet the patients’ needs and not the whims of the hierarchy.’ One manager said that due to a less rigid, more flexible approach, ‘the patients are getting a look in, for once.’ Many course members identified specific areas of nursing practice which they had consciously improved upon, including, how and when restraint is used; really listening to patients and their relatives; assuring privacy for patients. A
434
NURSE EDUCATION
TODAY
number of course members identified instances in which they used their increased knowledge of available resources interviewees
to improve patient care. Most
commented
positively
on the con-
tinuing value to patients, relatives and staff of many of the projects which were initiated as course
assignments
e.g.
booklets and the introduction charts.
However,
of projects
appeared
to
little for a variety of reasons including
loss of interest on behalf of the course members and/or their managers.
features of the data analysis and the functions this type of evaluation. 1. The framework which was chosen
of
for this
study provided a useful guide to indicate areas to be evaluated. Had this framework informed course planning, professional
then the concepts
and individual
of education,
development
would
have been more clearly elucidated and the data yielded may have resulted in a more comprehensive evaluation interview
of the impact of the course.
format
which was chosen
have been most appropriate acquisition
The
was felt to
and resulted
in the
of a richer source of data than could
have been obtained tionnaire
An arbitrary
decision to conduct this type of following course 6 months
evaluation
through
or a structured
In some instances
the use of a ques-
interview.
the manager
may not have
been the most appropriate person to interview. For enrolled nurses and staff nurses it may be that the sister or charge nurse would be more aware of the impact of the course than the manager. It is suggested, therefore, that for future evaluations approach to sample
a more individualised selection would be appro-
priate. A major area of concern about the methodology employed highlights the difficulty of attributing particular aspects of change and development to a specific variable e.g. ENB 94 1, especially after a IO-month time span. It is recognised, however, that the incorporation of the evaluation framework into course develop-
relationship.
completion may reduce the potential variables affecting development. almost
all
variety of backgrounds being
number
of the data collected
interviewees
a particularly
identified of course
beneficial
of
was that the
wide
members aspect
of
as the
course. No-one identified the very different levels of experience and ability as an inhibiting factor.
This could be seen to support
(1984)
assumptions
that as a person
an increasing
learning
and his readiness
oriented
increasingly
of experi-
resource
for
to learn becomes
to the
developmental
tasks of his social roles.’ (Knowles Another
Knowles
matures
a growing reservoir
ence that becomes
Discussion about this study will focus on three main areas - the approach to evaluation, key
may assist
the cause and effect
‘he accumulates
DISCUSSION
of objectives
in clarifying
2. A key feature
information
of pain assessment
it was also clear from discuss-
ion that a small number contribute
ward
ment and identification
1984)
aspect of the course which appeared
have a major clarification
impact
was the process
to
of role
which took place. As Schon (1987:4)
suggests, the problems
of real world practice
not present
to practitioners
themselves
do
as prob-
lems at all but as messy, indeterminate situations. For at least one course member this period of reflection the
resulted in the naming and framing
problem
problem
of role
diffusion
thus
of
enabling
solving to take place. The value which
was placed on the opportunity
for reflection
by
all course members and some managers may indicate an area worthy of further study in terms of how such
opportunities
essential component
could
of nursing
become
an
practice.
An area of concern to all interviewees was the difficulty of applying learning in clinical areas and of maintaining ‘for continuing of
participants,
their motivation: education the
to change behaviour
participants
need
the
desire to learn, the ability to learn and then the right job climate for them to implement the new knowledge and skills into new practice. Change of behaviour will only occur and be maintained if the new behaviours are reinforced by reward, and recognition given by significant others such as nurses’ supervisors’. (Ferrell
1988)
NURSE EDUCATION
These issues might initially be addressed by such initiatives as the use of learning contracts for participants to indicate how the new concepts learned during the course will be applied in their clinical areas. ‘From an evaluative point of view, contracts provide an insight into what concepts in the continuing education offerings have been perceived by the participants as practical and applicable.’ (Ferrell 1988) Managers could ensure that secondment to a course such as ENB 94 1 forms part of a planned programme of development rather than an ad hoc occurrence. 3. Some of the factors influencing the use of this type of evaluation have been highlighted previously. It is likely, however, that such evaluations will become increasingly common for reasons other than educational ones i.e. social, economic and political factors will be influential. The prevailing value for money ethos demands accountability at all levels. Teachers need to demonstrate the value of their offerings, managers need to ensure that their investment in CPE yields measurable returns, and course participants will become more demanding if they are either being required to fund their own CPE or to meet certain objectives following participation in education programmes as part of their Individual Performance Review. Another function of this type of evaluation is the development of a deeper-understanding of the barriers to innovations and improvements in local areas. Identifying these barriers may then result in a reduction in burnout in clinical and teaching staff. Such an approach would be supported by Jarvis’ (1987) contention that ‘all teaching and learning should be in direct response to a practice related problem.’
LIMITATIONS
AND WEAKNESSES
In terms of providing a useful evaluation of the impact of ENB 941 on course participants the main weakness of this study has already been identified i.e. the fact that the evaluation
TODAY
435
approach was not an integral part of course design but was added later. However, this should not detract from the valuable information which was gained and which can usefully inform future curriculum development. Another potential weakness of the study was that it was carried out by the course tutor. Whilst her background knowledge enabled her to use to full advantage the unstructured interview format, the possibility exists rhat both the interviewer and interviewees over concentrated on the positive aspects of the course. If the interviewer had not been known to the interviewees it is conceivable that a more objective evaluation could have ensued.
CONCLUSION This study was designed to address the question of what impact, if any, ENB 941 had on the professional practice of course members. The data supports the contention that ‘Some nurses who participate in continuing education activites have been able to use what they have learned in their subsequent nursing practice. If the data collected do not indicate that continuing education necessarily has an effect on nursing practice, the data certainly indicate that it can.’ (Puetz 1979)
References Allen P, Jolley M 1987 The Curriculum in Nursing Education. Croom Helm Bignell A, Crotty M 1988 Continuing Education: Does it enhance patient care. Senior Nurse 8.4 Burrell T 1988 Curriculum design and development: A procedure manual for nurse education. Prentice Hall Cohen L, Mannion L 1985 Research Methods in Education (2nd ed), Croom Helm Crotty M, Bignell A 1987 Educating the Educators Senior Nurse 7, 4 ENB (1987) Managing Change in Nursing Education. ENB with Learning Materials Design Ferrell M J 1988 The relationship of continuing education offerings to self-reported change in behaviour. The Journal of Continuing Education in Nursing 19, 1 Gosnell D V 1984 Evaluating Continuing Nursing Education. The Journal of Continuing Education in Nursing 15, 1
436
NURSE EDUCATION
TODAY
Heick M A 198 1 Continuing Education Impact Evaluation. The Journal of Continuing Education in Nursing 12,4 Hutton C A 1987 Impact of Mandatory Continuing Education: A review of research on nurses’ attitudes and perceived outcomes. The Journal of Continuing Education in Nursing 18,6 Knowles M 1984 The Adult Learner: A Neglected Species. (3rd ed), Gulf Publishing, Houston Jarvis P 1985 Professional Education. Croom Helm Jarvis P 1985 The Sociology of Adult and Continuing Education. Croom Helm Jarvis P 1987 Adult Learning in the Social Context. Croom Helm Jarvis P 1987 Lifelong education and its relevance to nursing. Nurse Education Today 7, 1: 49-55
Kelly N V 1986 The Curriculum: Theory and Practice. Second Edition, Harper NBS National Board for Nursing, Midwifery and Health Visiting for Scotland 1985 Guidelines for Continuing Education (Professional Studies 1 & 2) Oppenheim A N 1966 Questionnaire Design and Attitude Measurement. Heinemann Peutz B E, Rytting M B 1979 Evaluation of the effect of continuing education in nursing on health care. The Journal of Continuing Education in Nursing 10,2 Schon D 1987 Educating the Reflective Practitioner Josey Bass, USA Tight M 1983 Education for Adults: Adult Learning and Education. Croom Helm Treece E, Treece J 1987 Elements of Research in Nursing. Fourth Edition, Mosley.
02606917190/0010-04281f10.00
(1990) 10,428-4% UK Ltd 1990
Evaluating the impact of continual professional education (ENB 941) Pat Hughes
It is often assumed that participation in continuing professional education (CPE) activities necessarily results in maintenance of competence and improvements in nursing practice. A literature review revealed that evaluation of CPE activities is frequently limited to the completion of forms at the conclusion of the offering, indicating the extent to which the objectives have been achieved and the general level of satisfaction of the participants. Very little attention seems to have been paid to the longer term effects of the CPE activity on the individual participant or on their ability to provide an enhanced level of nursing practice - the raison d’etre of CPE. This project was designed to explore the impact of ENB 941 on course participants. A group of 11 nurses and their six clinical managers were interviewed 10 months following course completion to discuss the effects of the course on the professional, educational and individual development of course participants and on their ability to improve nursing practice. Analysis of the interviews suggests that all group members used some of the opportunities available in the course to further their development. All clinical managers agreed that despite staffing and financial implications, the course provided value for money in terms of staff development and enhanced nursing practice.
carried
BACKGROUND ENB 941 Caring course
for Older
for qualified
nurses.
People is a 22-day This
course
was
developed as a joint venture between two health authorities and consists of a Briefing Day, two 2-week blocks and a Recall Day over a 4-month period. Evaluation group
and
of the extent individual
to which
objectives
are
course, met
Pat Hughes RGN Dip N PGCEA RNT RCNT Senior Nurse Manager/Education Bromley and Camberwell Department of Nursing Studies, 13 Grace Avenue, Maidstone, Kent, ME16 OBS (Requests for offprints to PH) Manuscript accepted 21 March 1990 428
out
evaluation
is
in a variety of each
block is completed This
assists the
development
of ways.
A written
day’s experience
of study
by every
member.
course
course
tutors
by providing
in curriculum
useful information
about both content and process at the end of each block. It also facilitates course members in clarifying the extent to which they are meeting their personal Evaluation throughout the course
objectives. also takes
place
informally
the blocks. At the end of each block members work in small groups to
evaluate the extent to which the course objectives are being achieved and to identify positive and negative aspects of the experience. The feedback from each small group is discussed and the comments
collated
to provide
a report
to the
NURSE EDUCATION
Curriculum Development Group. Using these approaches the course has been consistently well evaluated in terms of process and outcome with positive comments far outweighing any negative ones. The negative comments tend to reflect individuals’ responses to particular teaching styles and content. Despite the very positive light in which the course is viewed and the interest and enthusiasm it generates, a major question which has not been addressed using such methods of course evaluation is what impact, if any, the course has on the professional practice of course members. Such type of valuation has been variously identified as follow up evaluation, post implementation evaluation, behaviour evaluation and impact evaluation (Heick 198 1). Gosnell (1984) suggests that a major impediment to the implementation of this type of evaluation is the absence of an identifiable conceptual framework or model.
LITERATURE
REVIEW
CPE can encompass a variety of activities such as study days, conferences, short courses e.g. ENB 94 1, role development courses and other longer courses leading to recognised academic awards. The American Nurses Association suggests that: ‘Continuing education in nursing consists of planned learning experiences beyond a basic educational programme. These nursing experiences are designed to promote the development of knowledge skills and attitudes for the enhancement of nursing practice, thus improving health care to the public.’ (Cited NBS 1985) Ferrell(1988) suggests that CPE is of particular relevance to nursing ‘since the half life of nursing knowledge ranges from two to five years.’ Basic professional education is thus no longer sufficient for a life time of practice. Social, demographic e.g. the increasing proportion of elderly people in the population, epidemiological and technological changes dictate the necessity for continuing education. Whilst there is, as yet, no mandatory requirement in this country for nurses to participate in
TODAY
429
specific educational activities, the UKCC issued a Code of Professional Conduct in 1984 which states that Every nurse, midwife or health visitor is accountable for his or her own practice and in the exercise of professional accountability shall: 3) take every reasonable opportunity to maintain and improve professional knowledge and competence. Work is currently under way to determine the extent to which participation in educational activities should become a periodic mandatory requirement, thus implying that CPE will result in maintaining competence and improving practice. However, in Hutton’s 1987 review of the American literature she concluded that the data were inconclusive and that the central question of the relationship between continuing education and competence remains unanswered. In an attempt to identify an appropriate model for use with ENB 941 it was necessary to remain cognizant of Heick’s contention that: ‘whenever impact evaluation integral part of the totality development process and appendage or extension of gram.’ (Heick 198 1)
is used, it is an of the program not simply an any given pro-
That statement notwithstanding, it was decided to seek to identify a model which appeared to either complement the current evaluation process or to inform future initiatives in developing a more comprehensive framework for evaluation. To this end two specific models were considered - a four stage model described by Gosnell (Fig. 1) and a four component model described by Crotty and Bignell (1987) and Bignell and Crotty (1988). Within the model proposed by Gosnell, Stage I easily encompassed the extant evaluations pertaining to the participants’ perceptions, opinions and attitudes about ENB 941. Stage II could be accommodated by developing a more formalized approach to the assessment of course assignments. Stages III and IV however, would have required a level of sophisticated planning beyond the resources available for this study.
430
NURSE EDUCATION
TODAY
Outcome
-
Ftesulls
Evalualion
Fig 1 Conceptual model for evaluation of continuing nursing education
The model could more easily be used to provide
the
valuable information and insight if it was developed in conjuction with a particular CPE
improve
offering
i.e. if it had been incorporated
planning The
into the
of ENB 941.
model
proposed
was then considered modified form (Fig.
by Crotty
and Bignell
and selected for use in a 2). The model is under-
pinned by a theory which suggests that CPE programmes are comprised of four main components
-
professional,
individual development, ability to improve nursing ‘An improved knowledge educational development which in turn individual
educational
and the practice.
base will lead to the of the individual,
will lead to professional
development.
and
enhanced
These
of
practice.’
four
components
used by Crotty and Bignell of course planning
the
individual
(Crotty
to
& Bignell
appear
to have been
as an integral
thus facilitating
aspect
the develop-
ment of an evaluation pro-forma based on identified units of learning. Their evaluation was undertaken
6
months
completion
and
consisted
based on statements the course content.
following
course
of a questionnaire
of critical
functions
Each participant
from
was asked to
assess any changes they had noted in their effectiveness and efficiency for each of the 23 critical functions. They were also asked to comment or not the course
Profe/sional
Fig 2 ENB course leading to improved care
ability
nursing
1987).
and
This should result in
enhanced
had influenced
Individual Develoomenc
on whether them
in the
NURSE EDL’CATION
four previously
identified
components.
The par-
ticipants’ senior nurses were also asked to evaluate changes
in the students
using the same criteria. As the four components cational
and
enhanced
of professional,
edu-
development
and
to improve
had not been explicitly
perceived
responding
to be a desirable
rather
in order
in what they than perhaps
an honest manner.
individual
ability
role performance
ible in her attitude toward the responses to prevent interviewees
43 1
TODAY
nursing
practice
considered
in the plann-
the development
of an appro-
Data collection
was
Of the 13 course members, 11 agreed to participate in a taped interview. One had left the NHS 3
decided, therefore, to select a group who had completed the course within the previous year
months following course completion and could not be contacted and 1 had moved out of the area
and
to take up promotion
ing of the course, priate
pro
forma
to conduct
member
was
a taped
of the group
an unstructured
interview
with each
covering
each
(Cohen & Mannion
It was anticipated useful
It
and their manager
format
four components provide
problematic.
that
this
1985: 293).
approach
information
would
to illuminate
impact of the course on these participants inform
future
course
ods of evaluation. pleted the course
using of the
developments
and meth-
The group chosen 10 months
the
and to
from staff nurse to ward
sister on an elderly care ward. Of the 7 clinical nurse
managers
6 agreed
to participate
and
1
had left the area at the same time as her course member had changed jobs. Although some course members and some clinical managers were male, in the interests interviewees
of confidentiality
will be referred
all
to as she/her.
had com-
previously.
ANALYSIS Educational development
METHODOLOGY
The course
Pilot study
ledge
A pilot study was carried
out to test the appro-
priateness of the questions, to determine the time required for the interviews and to provide the interviewer
with an opportunity
and refine her technique. members
from other
together
groups
seconded
staff previously for this evaluation.
was indeed
use. The
Following
pertaining
but
in the group
an appropriate
questions
who had
and subsequently
study it was clear that an unstructured format
course
were interviewed
with a clinical nurse manager
whose staff were not represented chosen
to practice
Two previous
the pilot interview
approach
to
to professional
members
claimed
base had increased
particularly
in breadth.
forced
their
by
information
was a very significant reflects
The
feature
of professional
acute
and long
hospital members
for
the
worked
environments
stay elderly elderly
in modern
where
varying
degrees
examples this area.
course just reinforced All course members
highlighted interviewer
together to remain
the ENB discussion
with the need as non-committal
ill. Group
and
probing
importance of keeping firmly in mind during
and were
wards and a day
mentally
provide The course
members
and as the
drawn. The group included community nurses, nurses working on both day and night duty in
dial and therapeutic approaches practised. One course member
issues when
content,
backgrounds
clinical areas from which course
of
members
in discussion
development presented some difficulty to the previous course members so it was necessary to of professional
was rein-
breadth
by the course
not so much the course
wide range
and more
This statement
managers.
acquired
that their know-
in depth
traditional of custo-
to care were claimed ‘the
what I knew.’ identified further
edu-
941 was
cational input which they had sought as a direct
for the as poss-
of the members identified aspects of care which had been addressed in a necessarily limited
result of their participation
in the course.
Most
432
NURSE EDUCATlON
TODAY
manner on the course which they had followed up or were in the process of so doing e.g. by applying for courses and study days on topics such as Symptom Control and Counselling. TWO course members were continuing to seek out information and build up resource packs covering community facilities which are available for a particular client group and aids for people with physical disabilities. One course member was in the process of working her way through the Open University Package Caring for Older People P650 with a view to perhaps leading a group of junior staff through the package. One course member claimed to have gained considerably by learning how to use the library resources properly and during the course another member placed a regular order for a weekly nursing journal for the first time in her professional career. Some course members said their confidence in their ability to study had increased or been reinforced and this had stimulated them to apply for more academic courses in nursing e.g. Enrolled Nurse Conversion course, and for one person an Open University Foundation course. Four course members claimed that their experience of the course had increased their insight and abilities in helping learners. One course member had enthusiastically fulfilled the role of mentor to a member on a subsequent course and two other members declared their interest in acting as mentors in the future. One course member stated that the supportive non-judgemental milieu which surrounded the course was appropriate and much appreciated at the time but felt on reflection that more emphasis on grading and marking of assignments would have been helpful in terms of educational development. Two course members specifically mentioned an assignment which required them to carry out a teaching activity which was assessed by their peers, course tutors and their individual mentor as being particularly anxiety provoking at the time but of enormous value subsequently. Within this area of development the managers reinforced many of the comments which had been made and confirmed that course members had pursued further some areas of interest
apparently stimulated by the course. Two managers expressed interest in the value of exploring the response of their individual staff members to educational input. One manager emphasised the importance of incorporating the continuing identification of educational needs into the context of Individual Performance Review.
Professional
development
With reference to the impact of ENB 94 1 on the professional development of course members, the responses fell easily into four categories clarification of the role of the nurse in elderly care; effect on career plans; communication and relationship effects and knowledge of current professional issues. More than half of the group claimed to have achieved more clarity in their thinking about how the role of the qualified nurse in elderly care differed from that of other members of the caring team e.g. social workers or nursing auxilliaries. One clinical manager said that the clarification of roles that occurred in a clinical area following the course had had a major beneficial impact on nursing practice and on team spirit, as people acknowledged their differing contributions to patient care. Two group members who were RGN’s and have since commenced RMN training claimed to have been influenced in their decisions by their increased insight into the skills required to care for elderly people who are mentally ill. Some course members said that the course provided a useful base to progress their future career and promotion plans. One course member claimed not to have any career plans, but claimed increased commitment ‘to do what I’m doing very well’. All course members claimed to have developed greater understanding of their colleague’s roles. For some, this reinforced their previous choices e.g. the community nurses were clear that caring for patients in hospital held little attraction for them. None of the course members admitted to any decrease of interest or enthusiasm for working with elderly people. Four course members said they felt more confident of their contributions within the multi-
NURSE EDUCATION
disciplinary team. The course was also said to have had a positive effect on the relationship between course members and their managers. Some course members claimed to feel more confident and determined in communicating their needs and the needs of their patients to their managers. Three managers said that their course members had developed a more positive and creative approach to problem solving and demonstrated a deeper understanding of broader issues affecting the provision of services. One manager expressed concern over the apparent widespread acceptance by staff that there was only one system or channel of communication and that if that system failed no other approaches were possible. She stated that at least one course member had made significant progress in overcoming this attitude following the course. One member claimed to have felt particularly disconcerted on the course by the realisation that ‘those at the top don’t always know best and have clear views about what’s going on.’ Most course members claimed to have developed increased insight into the social context in which nursing care was being delivered. All course members commented on the opportunities presented to discuss relevant reports of the time e.g Griffiths report Community Care: Agenda for Action. One manger saw evidence of increased insight into and interest in planning for future services in one course member. ‘She seemed to suddenly grasp where we were coming from and trying to get to.’ One-manager claimed that a course member had demonstrated the use of research findings in clinical practice as a direct response to specific course content.
Individual development During the interviews three main issues came to the fore - the impact of the course on members as individuals; on their relationship with others and the problem of sustaining the positive effects of the course. All course members claimed to value the opportunity to take time out of the clinical areas ‘Just to stop and think’. The majority of
TODAY
433
members commented on how their communication skills had improved as a result of the course. One course member claimed to have ‘learned to sell my ideas better and, by giving praise, get the co-operation and increase the motivation of other staff.’ Another member said ‘I now hold my tongue a bit better and try other ways of getting what I want.’ More than half of the course members claimed to find it easier to discuss issues pertaining to elderly people with others at all levels of the organisation. One member said she values discussion and sharing of ideas with colleagues and rather than ‘try to score points’, she listens more. Most managers claimed to have seen evidence of more frequent constructive discussion among staff. One manager commented on a particular course member’s improved ability to follow ideas through into action. Eight course members claimed to feel more confident in their ability to contribute to patient care. All members claimed to feel less isolated and more motivated. One manager claimed that the course had ‘done wonders’ for a course member’s self-esteem and self-awareness because she felt she had done well on the course. Following on from these positive responses a major problem which was identified by all interviewees concerned the difficulties experienced in maintaining enthusiasm and motivation on return to clinical areas.
Nursing practice All interviewees claimed that a more individualised approach to patient care was evident to a greater or lesser degree in clinical practice. This manifested itself in a variety of ways including a re-organisation of nursing work. One member said that her ward ‘was now organised to meet the patients’ needs and not the whims of the hierarchy.’ One manager said that due to a less rigid, more flexible approach, ‘the patients are getting a look in, for once.’ Many course members identified specific areas of nursing practice which they had consciously improved upon, including, how and when restraint is used; really listening to patients and their relatives; assuring privacy for patients. A
434
NURSE EDUCATION
TODAY
number of course members identified instances in which they used their increased knowledge of available resources interviewees
to improve patient care. Most
commented
positively
on the con-
tinuing value to patients, relatives and staff of many of the projects which were initiated as course
assignments
e.g.
booklets and the introduction charts.
However,
of projects
appeared
to
little for a variety of reasons including
loss of interest on behalf of the course members and/or their managers.
features of the data analysis and the functions this type of evaluation. 1. The framework which was chosen
of
for this
study provided a useful guide to indicate areas to be evaluated. Had this framework informed course planning, professional
then the concepts
and individual
of education,
development
would
have been more clearly elucidated and the data yielded may have resulted in a more comprehensive evaluation interview
of the impact of the course.
format
which was chosen
have been most appropriate acquisition
The
was felt to
and resulted
in the
of a richer source of data than could
have been obtained tionnaire
An arbitrary
decision to conduct this type of following course 6 months
evaluation
through
or a structured
In some instances
the use of a ques-
interview.
the manager
may not have
been the most appropriate person to interview. For enrolled nurses and staff nurses it may be that the sister or charge nurse would be more aware of the impact of the course than the manager. It is suggested, therefore, that for future evaluations approach to sample
a more individualised selection would be appro-
priate. A major area of concern about the methodology employed highlights the difficulty of attributing particular aspects of change and development to a specific variable e.g. ENB 94 1, especially after a IO-month time span. It is recognised, however, that the incorporation of the evaluation framework into course develop-
relationship.
completion may reduce the potential variables affecting development. almost
all
variety of backgrounds being
number
of the data collected
interviewees
a particularly
identified of course
beneficial
of
was that the
wide
members aspect
of
as the
course. No-one identified the very different levels of experience and ability as an inhibiting factor.
This could be seen to support
(1984)
assumptions
that as a person
an increasing
learning
and his readiness
oriented
increasingly
of experi-
resource
for
to learn becomes
to the
developmental
tasks of his social roles.’ (Knowles Another
Knowles
matures
a growing reservoir
ence that becomes
Discussion about this study will focus on three main areas - the approach to evaluation, key
may assist
the cause and effect
‘he accumulates
DISCUSSION
of objectives
in clarifying
2. A key feature
information
of pain assessment
it was also clear from discuss-
ion that a small number contribute
ward
ment and identification
1984)
aspect of the course which appeared
have a major clarification
impact
was the process
to
of role
which took place. As Schon (1987:4)
suggests, the problems
of real world practice
not present
to practitioners
themselves
do
as prob-
lems at all but as messy, indeterminate situations. For at least one course member this period of reflection the
resulted in the naming and framing
problem
problem
of role
diffusion
thus
of
enabling
solving to take place. The value which
was placed on the opportunity
for reflection
by
all course members and some managers may indicate an area worthy of further study in terms of how such
opportunities
essential component
could
of nursing
become
an
practice.
An area of concern to all interviewees was the difficulty of applying learning in clinical areas and of maintaining ‘for continuing of
participants,
their motivation: education the
to change behaviour
participants
need
the
desire to learn, the ability to learn and then the right job climate for them to implement the new knowledge and skills into new practice. Change of behaviour will only occur and be maintained if the new behaviours are reinforced by reward, and recognition given by significant others such as nurses’ supervisors’. (Ferrell
1988)
NURSE EDUCATION
These issues might initially be addressed by such initiatives as the use of learning contracts for participants to indicate how the new concepts learned during the course will be applied in their clinical areas. ‘From an evaluative point of view, contracts provide an insight into what concepts in the continuing education offerings have been perceived by the participants as practical and applicable.’ (Ferrell 1988) Managers could ensure that secondment to a course such as ENB 94 1 forms part of a planned programme of development rather than an ad hoc occurrence. 3. Some of the factors influencing the use of this type of evaluation have been highlighted previously. It is likely, however, that such evaluations will become increasingly common for reasons other than educational ones i.e. social, economic and political factors will be influential. The prevailing value for money ethos demands accountability at all levels. Teachers need to demonstrate the value of their offerings, managers need to ensure that their investment in CPE yields measurable returns, and course participants will become more demanding if they are either being required to fund their own CPE or to meet certain objectives following participation in education programmes as part of their Individual Performance Review. Another function of this type of evaluation is the development of a deeper-understanding of the barriers to innovations and improvements in local areas. Identifying these barriers may then result in a reduction in burnout in clinical and teaching staff. Such an approach would be supported by Jarvis’ (1987) contention that ‘all teaching and learning should be in direct response to a practice related problem.’
LIMITATIONS
AND WEAKNESSES
In terms of providing a useful evaluation of the impact of ENB 941 on course participants the main weakness of this study has already been identified i.e. the fact that the evaluation
TODAY
435
approach was not an integral part of course design but was added later. However, this should not detract from the valuable information which was gained and which can usefully inform future curriculum development. Another potential weakness of the study was that it was carried out by the course tutor. Whilst her background knowledge enabled her to use to full advantage the unstructured interview format, the possibility exists rhat both the interviewer and interviewees over concentrated on the positive aspects of the course. If the interviewer had not been known to the interviewees it is conceivable that a more objective evaluation could have ensued.
CONCLUSION This study was designed to address the question of what impact, if any, ENB 941 had on the professional practice of course members. The data supports the contention that ‘Some nurses who participate in continuing education activites have been able to use what they have learned in their subsequent nursing practice. If the data collected do not indicate that continuing education necessarily has an effect on nursing practice, the data certainly indicate that it can.’ (Puetz 1979)
References Allen P, Jolley M 1987 The Curriculum in Nursing Education. Croom Helm Bignell A, Crotty M 1988 Continuing Education: Does it enhance patient care. Senior Nurse 8.4 Burrell T 1988 Curriculum design and development: A procedure manual for nurse education. Prentice Hall Cohen L, Mannion L 1985 Research Methods in Education (2nd ed), Croom Helm Crotty M, Bignell A 1987 Educating the Educators Senior Nurse 7, 4 ENB (1987) Managing Change in Nursing Education. ENB with Learning Materials Design Ferrell M J 1988 The relationship of continuing education offerings to self-reported change in behaviour. The Journal of Continuing Education in Nursing 19, 1 Gosnell D V 1984 Evaluating Continuing Nursing Education. The Journal of Continuing Education in Nursing 15, 1
436
NURSE EDUCATION
TODAY
Heick M A 198 1 Continuing Education Impact Evaluation. The Journal of Continuing Education in Nursing 12,4 Hutton C A 1987 Impact of Mandatory Continuing Education: A review of research on nurses’ attitudes and perceived outcomes. The Journal of Continuing Education in Nursing 18,6 Knowles M 1984 The Adult Learner: A Neglected Species. (3rd ed), Gulf Publishing, Houston Jarvis P 1985 Professional Education. Croom Helm Jarvis P 1985 The Sociology of Adult and Continuing Education. Croom Helm Jarvis P 1987 Adult Learning in the Social Context. Croom Helm Jarvis P 1987 Lifelong education and its relevance to nursing. Nurse Education Today 7, 1: 49-55
Kelly N V 1986 The Curriculum: Theory and Practice. Second Edition, Harper NBS National Board for Nursing, Midwifery and Health Visiting for Scotland 1985 Guidelines for Continuing Education (Professional Studies 1 & 2) Oppenheim A N 1966 Questionnaire Design and Attitude Measurement. Heinemann Peutz B E, Rytting M B 1979 Evaluation of the effect of continuing education in nursing on health care. The Journal of Continuing Education in Nursing 10,2 Schon D 1987 Educating the Reflective Practitioner Josey Bass, USA Tight M 1983 Education for Adults: Adult Learning and Education. Croom Helm Treece E, Treece J 1987 Elements of Research in Nursing. Fourth Edition, Mosley.