- Email: [email protected]
Problem-based learning in clinical practice: Employment and education as development partners
Nurse Education in Practice (2008) 8, 420–427
Nurse Education in Practice www.elsevier.com/nepr
Problem-based learning in clinical practice: Employment and education as development partners ¨¨ ¨nen b, Riitta-Liisa A ari a,*, Leena Elomaa a, Minna Ylo Mikko Saarikoski a a b
Turku University of Applied Sciences, Ruiskatu 8, 20720 Turku, Finland Turku Health Care Department, Luolavuorentie 2, 20700 Turku, Finland
Accepted 17 October 2007
KEYWORDS
Summary In this project tutorial small group working sessions of problem-based learning were carried out in the long-term care practice period of nursing students. The purpose was to describe nursing students’ learning when tutorial work was carried out during their first clinical practice period. The students had one PBL cycle during their four weeks’ clinical practice consisting of two tutorial sessions where clinical mentors and the teacher worked together as tutors. The students were pre-tested before the clinical practice and post-tested afterwards using a questionnaire. The questionnaire had been used in pilot projects during 2004–2005. Altogether, 40 nursing students participated in the study. There were statistically significant differences between pre- and post-test in ‘‘PBL-learning skills’’ (p < 0.001) and ‘‘interaction with old patients’’ (p < 0.001). The means of items ‘‘Interested in work in long-term care’’ and ‘‘Own attitude towards long-term care’’ had increased between pre- and post-test. In conclusion it can be suggested that carrying out tutorials in nursing students’ clinical practice is beneficial. c 2007 Elsevier Ltd. All rights reserved.
Clinical practice; Problem-based learning; Evidence based nursing; Long-term care
Introduction
* Corresponding author. Tel.: +358 50 365 2885; fax: +358 10 55 35 451. E-mail addresses: [email protected], leena.elomaa @turkuamk.fi, [email protected], [email protected]
This article describes a collaboration project between Turku University of Applied Sciences and Turku Health Care Department. Nursing education has been carried out based on a problem-based learning (PBL) curriculum since 2004. Therefore, there was also an evident need to develop the clinical learning
1471-5953/$ - see front matter c 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.nepr.2007.10.005
Problem-based learning in clinical practice: Employment and education as development partners environment and inform clinical mentor nurses of PBL. The aim of the project was to test tutorial small group work in clinical practice during the student nurses’ first clinical practice period (four weeks) in long-term care nursing. Long-term care is a part of health care services for the elderly in the Turku health care department. In long-term care patients need considerable help in their daily living activities. Long-term care consists of taking care of patients’ basic needs and nursing needs. The goal is on good patient oriented nursing which ensures quality of life and a safe environment during the patients’ stay in the ward. In this project ‘‘old patients’’ concept means older people who are patients in long-term care. The term ‘‘mentor’’ is used for a staff nurse working in long-term care (registered nurse or enrolled nurse). The students are nursing students in their first clinical practice period in long-term care and the term ‘‘teacher’’ is used to illustrate a teacher who is responsible both for theoretical and clinical teaching. Initially the clinical mentors of the nurse students participated in short clinical mentor’s training (20 h) where the basic principles of PBL and evidence based nursing (EBN) were dealt with. In long-term care nursing, the student nurses’ clinical mentors are registered nurses (RN) and enrolled nurses (EN). The clinical mentors who took part this project were also both RNs and ENs. The idea was that the clinical mentors and teacher worked together in small group tutorials to promote students’ learning. The idea was also to narrow the theory–practice gap and to develop evidencebased nursing students’ clinical practice in longterm care. A further clear need was to increase clinical mentors’ knowledge of PBL and EBN, and to develop their interaction, team-work and information-searching skills. In this article, the purpose is to describe tutorial working in clinical practice from the point of view of student nurses’ learning.
Background In Finland during the last decade there has been a tendency to develop health care and nursing education according to the principles of problem-based learning. The essential characteristics of problem based learning are a curriculum organized around problems that rise of clinical practise and are relevant to desired learning outcomes, and conditions that facilitate small-group working, self-directed learning, independent study, functional knowledge, critical thinking, lifelong learning, and self-evalua-
421
tion (Rideout and Carpio, 2001). According to Rideout et al. (2002) and Tompkins (2001), PBL is suitable for nursing education. PBL develops essential professional competencies such as critical thinking, interaction, relationship and self-evaluation skills. It requires information search and management skills and verbal and nonverbal communication skills. One of the benefits is that PBL develops self-evaluation and peer evaluation skills which are valuable in employment. It is also important that a tutor can promote an atmosphere where it is safe to learn to give and get honest peer review of team work (Chaves et al., 2006) PBL nursing education increases empowerment in nursing students (Siu et al., 2005) and problem-solving skills (Uys et al., 2004). PBL gives an opportunity for the nurse students to develop their information retrieval and management skills, and work on self- and individual professional development. It also gives an opportunity for them to collaborate with other students and define learning goals (Siu et al., 2005). Because of these several good reasons in nursing education in Turku University of Applied Sciences, the pedagogy strategy in nursing curricula is problem based learning. According to Rideout et al. (2002) in a study comparing PBL-based educated nursing students and traditionally educated nursing students it was shown that PBL-students were more satisfied with their education than traditionally educated nursing students. They were more satisfied with the tutors, independence, evaluation strategies and education. There were no statistically significant differences in preparedness for nursing and clinical work. The interaction and self-orientation skills of the PBL students were more proficient than those of the traditionally educated nursing students. When comparing lecture based and PBLbased education it was shown that there were no statistically significant differences between students’ learning. This supports the idea that PBL is as effective as a learning strategy than traditional methods. It is worth of considering other benefits when deciding the learning method. (Beers, 2005). These other benefits are, e.g. self-evaluation and peer-evaluation skills, critical thinking and relationship skills. However, it was notable that one year after the end of the course when tested, the PBL group gained statistically significantly better scores than the traditionally taught group (Beers and Bowden, 2005). The course was Adult Health I Nursing course and the content was diabetes. The test was basic knowledge test of diabetes (Beers, 2005; Beers and Bowden, 2005). In medical education PBL tutorials are carried out during the clinical practice period. According
¨a R.-L. A ¨ri et al.
422 to O’Neill et al. (2000) both students (especially women) and tutors have been very satisfied with tutorial work during the clinical practice period. The students have been satisfied with team working, thinking of learning aims and integrating clinical practice experience with other knowledge. One challenge in medical education has been the development of students’ self-evaluation and taking the tutorial scenario from real patient cases. Further challenges are also practical issues such as time schedules (Morrison et al., 2003). According to Ax and Kincade (2001) nursing students can experience the use of nursing research in clinical practice as useless, and they can also feel that clinical mentors do not support them enough in using research information in nursing. Nursing tends to be based on practical experience, rituals and not on utilization of research in developing nursing. This is a challenge in which both nursing education and hospital organisations have to respond to. A great deal of research has been carried out into PBL in nursing education in the five last years, but only very little research has been done on PBL tutorial work in student nurses’ clinical practice.
Methods Research questions The following research questions were addressed: 1. What kind of PBL-learning skills did the nursing students assess they had before and after PBL clinical practice?
2. What kind of interaction skills with old patients did the nursing students assess they had before and after PBL clinical practice? 3. What kind of attitudes did the nursing students have towards information retrieval and longterm care before and after PBL clinical practice?
Method The study was a questionnaire study which was based on student nurses’ self-reporting pre- and post-test. An intervention was carried out during 2006 on six long-term care wards in the Turku Health Care Department. The students access randomly the study. It depends on that who practiced on the project wards. The student nurses did not know the project wards in advance.
Intervention The intervention was carried out as one PBL-cycle with two tutorial sessions during the four weeks’ clinical practice period. The tutorial small group working sessions were hold in clinical practice near by the project wards. Otherwise the nursing students’ clinical practice in the long-term care was carried out as before (the discussions between nursing student, teacher and mentor) (see Figs. 1 and 2). Both tutorials I and II took 2 h and the number of participating nursing students ranged from eight to nine. The mentors and teacher worked together as tutors in the tutorials. The project included mentor’s training (20 h) where the basic principles of PBL and EBN were taught. The scenario and the teacher were the same during the
Problem-solving continues…
Knowledge is applied to the problem
The knowledge acquired is discussed and debated critically
Evaluation: Learning process Problem-solving Team work process
Scenario
TUTORIAL 1
Brain-storming
Analysis of output
TUTORIAL 2 Independent study and information gathering
Identification of learning issue
Identification of problem areas
References : Konsell et al. 2006; According Poikela 2001
Figure 1 PBL-cycle model (Konsell et al., 2006, published here with permission of Sairaanhoitaja – journal). (Poikela, 2001 See above-mentioned references for further information.)
Problem-based learning in clinical practice: Employment and education as development partners Middle evaluation discussion
Learning objective discussion
2. week
1. week
423
Final evaluation discussion
4. week Time
3. week
TUTORIAL I Only on project wards
TUTORIAL II Only on project wards Copyright © Turku University of Applied Sciences Reference: Konsell et al. 2006
Figure 2 The timing of discussions and tutorials in clinical practice. (Konsell et al., 2006, published here with permission of Sairaanhoitaja – journal).
intervention. The idea was that at least one to a maximum of three mentors in the project wards participated in the tutorial work. The student nurses worked in tutorials as in the school environment (see Fig. 1). One of the nursing students kept a diary of the tutorials. Altogether four PBL-cycles were carried out during 2006.
Questionnaire The students were tested before and after the clinical practice period with a self-reporting questionnaire. The students complete the questionnaire before the clinical practice in the university in an information session. During the last week in clinical practice the students were given the post-test and they were meant to complete it after clinical practice in their spare time. Completing the questionnaire took time approximately 10 min. The questionnaire was based on students’ self-ratings (Likert 1–7, 1 = very poor and 7 = excellent, or 1 = never and 7 = always, or 1 = fully disagree and 7 = fully agree, or 1 = dissatisfied and 7 = satisfied). The questionnaire included items of ‘‘PBL-learning Table 1
skills’’, ‘‘interaction with old patients’’, ‘‘attitudes towards information retrieval’’ and one item of ‘‘interested to work in long-term care’’ and one item of ‘‘own attitude towards long-term care’’. In the post-test the student nurses were also asked questions related to items such as ‘‘action when extra-information was needed’’ and ‘‘satisfaction with clinical practice’’. The items were all positive predetermined. The questionnaire and setting was pilot tested in autumn 2005 (reported Konsell et al., 2006). No changes were made after that. After that the student nurses’ identification was carried out. The questionnaire without the section ‘‘interaction with old patients’’ was also previously used in a clinical practice development project in the Central University Hospital of Turku in 2004. The cronbach’s a of every sum variable are reported in Table 1.
Ethical considerations Research permission for the study was received from the relevant organisation of Turku University of Applied Sciences. The students were informed
Sum variables of the questionnaire
Sum variable
N of items
Cronbach’s a
1 2 3 4 5 6 7 8
8 9 7 8 9 6 7 3
0.76 0.87 0.73 0.86 0.90 0.43 0.69 0.70
PBL-learning skills – before the clinical practice Interaction with old patients – before the clinical practice Attitudes towards information retrieval – before the clinical practice PBL-learning skills – after the clinical practice Interaction with old patients – after the clinical practice Action when extra-information was needed Attitudes towards information retrieval – after the clinical practice Satisfaction with clinical practice
¨a R.-L. A ¨ri et al.
424 of the aim of the project before the clinical practice in an information session. An information sheet about the project was also distributed stressing anonymity and the voluntary nature of the study. The participants’ informed consent was not asked separately, because it was assumed that participation in the study was a sign of willingly being involved in this study. Permission from the Ethical Committee was not needed because the focus of the study was on the health care professional – nursing students. The autonomy of the students was protected. Identification in pre- and post-test and data saving and analysis were done by a different person. The students were able to leave study at any time, e.g. completing the pre-and post-test questionnaires. Instead of that the tutorial small group work was obligatory because it was the students work during the first clinical practice and formed thereby a part of their clinical practice performance.
Results The data were analysed by SPSS 13.0 for Windows. The purpose was to compare the data before and after the PBL clinical practice. Sum variables were formed (see Table 1). Cronbach’s a of sum variables ranged from 0.43 to 0.90. Non parametric tests were used (Wilcoxon-test) and the significance level was set at 0.05. The mean age of students was 21 years and most were female (95%). Previous health care education had only a minority of students (13%). This means previous health care school level education such as enrolled nurse or practical nurse. The data were collected during 2006 and the average response rate was 97% (see Table 2).
Table 2
Demographics of nurse students
Age: N Mean Min Max
40 21 19 34
Gender: N Female Male
40 38 2
Previous health care exam: N Yes No
39 5 34
The means of the sum variables were compared between pre- and post-test (see Fig. 3). The variables ‘‘difference between PBL learning skills before and after’’, ‘‘difference between interaction with old patients before and after’’, and ‘‘attitudes towards information retrieval before and after’’ (see Fig. 3) were also formed. There were clear differences in PBL learning skills (Wilcoxontest, p < 0.001) and in interaction with old patients before and after (Wilcoxon-test, p < 0.001). There was no statistically significant difference in attitudes towards information retrieval before and after clinical practice (Wilcoxon-test, p = 0.793). The mean in the sum variable ‘‘Action when the extra-information was needed’’ was 4.9 (1 = never, 7 = always). Cronbach’s a in this variable was low 0.43, because it constituted items of a quite different kind. Therefore, the means of the items were also examined separately. The means of the items (6) were ‘‘I repeated the information of the lectures 4.7’’, ‘‘I searched information of books 4.4’’, ‘‘I asked the thing nurse 6.1’’, ‘‘I discussed with my college student 5.7’’, ‘‘I asked the teacher 4.2’’ or ‘‘I searched information of Internet 4.2’’. The student nurses were satisfied with their clinical practice. The mean was 5.6. The atmosphere of the ward was examined separately. The students were satisfied with it, with a mean of 5.2. ‘‘Interested in work in long-term care’’ and ‘‘own attitude towards longterm care’’ had increased between pre- and posttest. The mean increased from 3.63 to 4.08 in ‘‘interested in work in long-term care’’. Additionally, the mean increased from 5.48 to 6.13 in ‘‘own attitude against long-term care’’.
Discussion This project showed that tutorial small group working worked well as a learning method in clinical practice. Students’ PBL learning skills and interaction with old patients improved during the project. In this study the clinical practice period was the students’ first clinical practice. They had no previous experience of other kinds of clinical practice. Theory education in classrooms was based on PBL and tutorials. It can be that this also had a transfer effect to the students’ clinical practice. PBL experience in clinical practice can increase nursing students’ interest in work and can also positively change attitudes towards long-term care. The atmosphere of a ward has an effect on students’ learning experience (Saarikoski, 2002). In this study, students’ evaluation of the atmosphere of the ward was good. This encourages the idea that
Problem-based learning in clinical practice: Employment and education as development partners
425
Mean
7 6 5 4 3 2 1
At
t.
lo
ng
-t 2
-t 1
t2
ng
In At
t.
lo
t1 In
t
os tm
f
At
Sa
in E-
td
td At
In
Ld
t2
PB
At
t1
t2
At
In
t1 In
L PB
PB
L
1
2
0
PBL 1= PBL-learning skills – before the clinical practice PBL 2 = PBL-learning skills – after the clinical practice Int 1 = Interaction with old patients - before the clinical practice Int 2 = Interaction with old patients - after the clinical practice Att 1 = Attitudes towards information retrieval - before the clinical practice Att 2 = Attitudes towards information retrieval - after the clinical practice PBL – d = difference between PBL 1 and 2 Int – d = difference between interaction 1 and 2 Att –d = difference between attitudes 1 and 2 E – inf = Action when extra-information was needed Sat = Satisfaction to clinical practice Attmos. = atmosphere of ward Int 1 = interested in work in long-term care before the clinical practice Int 2 = interested in work in long-term care after the clinical practice Att. long-t 1 = own attitude towards long-term care before the clinical practice Att. long-t 2 = own attitude towards long-term care after the clinical practice
Figure 3
Means between pre- and post-test.
PBL clinical practice can have a positive effect on atmosphere and nursing students’ learning experiences. Students’ action when extra information was needed shares similarities with previous research (Elomaa, 2006). Students ask nurses – mentors more often than they search evidence, e.g. from the Internet. The importance of carrying out tutorial working in clinical practice can also be based on a fruitful dialogue of evidence based nursing between mentors, students and their teachers. PBL is usually integrated into class room theory education. Many evaluation strategies show that PBL has a positive effect on nursing students and that the PBL method has exceeded the expectations of nurse educators (e.g. Alexander et al., 2002). The development of collaboration between teacher and clinical mentor is important because it creates the best possible clinical practice environment for nurse students. Good collaboration is necessary to ensure clinical practice which can offer learning experiences for the students learning needs (Papp et al., 2003). Small group working – tutorials – together with nursing students, clinical mentors and teachers can be a forum, where it is possible to discuss relevant clinical learning issues from the point of view of nursing students and in general nursing. The process of PBL is designed to facilitate students’ independence, self-direction, critical thinking, decision-making, interaction and group skills (e.g. Crookes et al., 2001). Additionally, PBL is fun, interesting, exciting and a motivation learning
strategy, which is worth developing in nursing students’ clinical practice (Cooke and Moyle, 2001). The results of this study also support this idea. PBL students can network and release emotions easily during clinical practice, e.g. in small group sessions. In these sessions the purpose is to discuss and solve practical nursing problems, and use research and clinical experience as help. This supports the idea of evidence based nursing practice. Nurse students and medical students working together in clinical practice in a small group could improve multi-professional collaboration, professional roles and the understanding of other professionals. The gap between theory and practice can be narrowed only when the nurses share the ideology of evidence based nursing. Critical and creative thinking are preconditions to this. It is valuable that nurses and educators consider together how to introduce practical nursing research which can be useful in clinical nursing practice (Seymour et al., 2003). PBL is a student centred learning method which is based on good information management. A precondition to evidence based clinical nursing is that nurses are experts in handling information and that they have positive attitudes towards life-long learning. Today students have to be capable of effectively evaluating nursing and professional growth. Nursing students have to be encouraged to develop their information search and management, critical appraisal, and implementation skills as all these skills are needed in nursing (e.g.
¨a R.-L. A ¨ri et al.
426 Barnard et al., 2005). In tutorials during clinical practice these skills can be practised together with students, mentors and teachers. This is especially useful for mentors updating the knowledge, skills and attitudes of EBN and for teachers updating their knowledge of clinical practice. The students can utilize the expert skills of both professionals, nurses and teachers in their learning. The kind of intervention study presented here involves some inherent problems. Firstly, the sample size was quite small. Secondly, any intervention study in nursing education research is always a great challenge because of many intermediating variables (in our case, e.g. mentors and wards). Thirdly, studies which are based on students’ self-ratings are often problematic. Nursing students in this study evaluated their skills and abilities already in the pre-test to the upper half of the Likert scale 1–7. Therefore, the results of this study are preliminary but encourage the development of PBL in the clinical learning environment. There are some practical issues worth of considering when organizing small group working in nurse students’ clinical practice: (1) The scenario has to be fruitful and current both for nurse students’ learning goals and for clinical practice, (2) there has to be good space for small group work and information search (e.g. from Internet) near by ward, (3) The teacher has to arrange the timetable of small group work together with mentors in good time, (4) The nurse administration of clinical practice has to be involved of this kind of nurse students supervision and they have to see this as professional development for mentors as well, (5) The teacher and mentors of clinical practice have to be capable for collaboration. Both teacher and mentors are valuable to support nurse students learning in small group work in clinical practice. And (6) the diary of the tutorials and especially the outcome of the small group work is important to inform to the other nursing staff of ward.
Conclusions Tutorial small group working worked well as a learning method in clinical practice. Tutorial working during clinical practice is fruitful from the point of view of collaboration in students’ supervision with mentors and teachers. In tutorial sessions one benefit is the dialogue of evidence based nursing in clinical practice. We can also stress the importance of developing the problem-solving, interaction and teamwork skills of nursing students, mentors and teachers.
There are also some practical issues worth of considering when organizing tutorial small group working sessions in nurse students’ clinical practice: (1) The scenario has to be fruitful and current both for nurse students’ learning goals and for clinical practice. (2) There has to be good space for small group work and information search (e.g. from Internet) near by the ward. (3) The teacher has to arrange the timetable of tutorial small group work together with mentors in good time. (4) The nurse administration of clinical practice has to be involved of this kind of nurse students supervision and they have to see this as professional development for mentors as well. (5) The teacher and mentors of clinical practice have to be capable for collaboration. Both teacher and mentors are valuable to support nurse students’ learning in small group work in clinical practice. And (6) the diary of the tutorials and especially the outcome of the tutorial small group work are important to inform to the other nursing staff of the ward.
References Alexander, J., McDaniel, G., Baldwin, M., Money, B., 2002. Promoting, applying, and evaluating problem-based learning in the undergraduate nursing curriculum. Nursing Education Perspectives 23 (5), 248–253. Ax, S., Kincade, E., 2001. Nursing students’ perceptions of research: usefulness, implementation and training. Journal of Advanced Nursing 35 (2), 161–170. Barnard, A., Nash, R., O’Brien, M., 2005. Information literacy: developing lifelong skills through nursing education. Journal of Nursing Education 44 (11), 505–510. Beers, G., 2005. The effect of teaching method on objective test scores: problem-based learning versus lecture. Journal of Nursing Education 44 (7), 305–309. Beers, G., Bowden, S., 2005. The effect of teaching method on long-term knowledge retention. Journal of Nursing Education 44 (11), 511–514. Chaves, J.F., Baker, C., Chaves, J.A., Fisher, M., 2006. Self, peer and tutor assessments of MSN competencies using the PBL-evaluator. Journal of Nursing Education 45 (1), 25–31. Cooke, M., Moyle, K., 2001. Students’ evaluation of problembased learning. Nurse Education Today 22 (4), 330–339. Crookes, D., Lunyk-Child, O., Patterson, C., LeGris, J., 2001. Facilitating self-direct learning. In: Rideout, E. (Ed.), Transforming Nursing Education Through Problem-Based Learning. Jones and Bartlett Publishers, Sudbury, pp. 51–74. Elomaa, L., 2006. Research evidence implementation and its requirements in nursing education. Annales Universitatis Turkuensis, ser D, tom 532, Doctoral thesis. ¨a Konsell, O., A ¨ri, R-L., Elomaa, L., Ylo ¨nen, M., 2006. Problembased learning in clinical practice – employment and education as development partners. In Finnish. English abstract. Sairaanhoitaja 79 (4), 22–24. Morrison, S., Boohan, M., Jenkins, J., Moutray, M., 2003. Facilitating undergraduate interprofessional learning in healthcare: comparing classroom and clinical learning for nursing and medical students. Learning in Health and Social Care 2, 92–104.
Problem-based learning in clinical practice: Employment and education as development partners O’Neill, P.A., Morris, J., Baxter, C-M., 2000. Evaluation of an integrated curriculum using problem-based learning in a clinical environment: the Manchester experience. Medical Education 34, 222–230. Papp, I., Markkanen, M., Bonsdorff, M., 2003. Clinical environment as a learning environment: student nurses’ perceptions concerning clinical learning experiences. Nurse Education Today 23 (4), 262–268. Poikela, E., 2001. Ongelmaperustainen oppiminen yliopistossa. ¨ ystila In: Poikela, E., O ¨, S. (Eds.), Tutkiminen on Oppimista – ja Oppiminen on Tutkimista. Tampere university Press, Tampere, pp. 101–117. Rideout, E., Carpio, B., 2001. The problem-based learning model of nursing education. In: Rideout, E. (Ed.), Transforming Nursing Education Trough Problem-Based Learning. Jones and Bartlett Publisher, Sudbury, pp. 21–49. Rideout, E., England-Oxford, V., Brown, B., Fothergill-Bourbonnais, F., Ingram, C., Benson, G., Ross, M., Coates, A., 2002. A Comparison of problem-based and conventional curricula in nursing education. Advances in Health Sciences Education 7, 3–17.
427
Saarikoski, M., 2002. Clinical learning environment and supervision. Development and validation of the CLES evaluation scale. Annales Universitatis Turkuensis, ser D, tom 525. Doctoral thesis. Seymour, B., Kinn, S., Sutherland, N., 2003. Valuing both critical and creative thinking in clinical practice: narrowing the research-practice gap? Journal of Advanced Nursing 42 (3), 288–296. Siu, H., Laschinger, H., Vingilis, E., 2005. The effect of problem-based learning on nursing students’ perceptions of empowerment. Journal of Nursing Education 44 (10), 459– 469. Tompkins, C., 2001. Nursing education for the twenty-first century. In: Rideout, E. (Ed.), Transforming Nursing Education Through Problem-Based Learning. Jones and Bartlett Publishers, Sudbury, pp. 1–19. Uys, L.R., Van Rhyn, L.L., Nomthandaso, S.G., McInerney, P., Tanga, T., 2004. Problem-solving competency of nursing graduates. Journal of Advanced Nursing 48 (5), 500–509.
Available online at www.sciencedirect.com
Nurse Education in Practice www.elsevier.com/nepr
Problem-based learning in clinical practice: Employment and education as development partners ¨¨ ¨nen b, Riitta-Liisa A ari a,*, Leena Elomaa a, Minna Ylo Mikko Saarikoski a a b
Turku University of Applied Sciences, Ruiskatu 8, 20720 Turku, Finland Turku Health Care Department, Luolavuorentie 2, 20700 Turku, Finland
Accepted 17 October 2007
KEYWORDS
Summary In this project tutorial small group working sessions of problem-based learning were carried out in the long-term care practice period of nursing students. The purpose was to describe nursing students’ learning when tutorial work was carried out during their first clinical practice period. The students had one PBL cycle during their four weeks’ clinical practice consisting of two tutorial sessions where clinical mentors and the teacher worked together as tutors. The students were pre-tested before the clinical practice and post-tested afterwards using a questionnaire. The questionnaire had been used in pilot projects during 2004–2005. Altogether, 40 nursing students participated in the study. There were statistically significant differences between pre- and post-test in ‘‘PBL-learning skills’’ (p < 0.001) and ‘‘interaction with old patients’’ (p < 0.001). The means of items ‘‘Interested in work in long-term care’’ and ‘‘Own attitude towards long-term care’’ had increased between pre- and post-test. In conclusion it can be suggested that carrying out tutorials in nursing students’ clinical practice is beneficial. c 2007 Elsevier Ltd. All rights reserved.
Clinical practice; Problem-based learning; Evidence based nursing; Long-term care
Introduction
* Corresponding author. Tel.: +358 50 365 2885; fax: +358 10 55 35 451. E-mail addresses: [email protected], leena.elomaa @turkuamk.fi, [email protected], [email protected]
This article describes a collaboration project between Turku University of Applied Sciences and Turku Health Care Department. Nursing education has been carried out based on a problem-based learning (PBL) curriculum since 2004. Therefore, there was also an evident need to develop the clinical learning
1471-5953/$ - see front matter c 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.nepr.2007.10.005
Problem-based learning in clinical practice: Employment and education as development partners environment and inform clinical mentor nurses of PBL. The aim of the project was to test tutorial small group work in clinical practice during the student nurses’ first clinical practice period (four weeks) in long-term care nursing. Long-term care is a part of health care services for the elderly in the Turku health care department. In long-term care patients need considerable help in their daily living activities. Long-term care consists of taking care of patients’ basic needs and nursing needs. The goal is on good patient oriented nursing which ensures quality of life and a safe environment during the patients’ stay in the ward. In this project ‘‘old patients’’ concept means older people who are patients in long-term care. The term ‘‘mentor’’ is used for a staff nurse working in long-term care (registered nurse or enrolled nurse). The students are nursing students in their first clinical practice period in long-term care and the term ‘‘teacher’’ is used to illustrate a teacher who is responsible both for theoretical and clinical teaching. Initially the clinical mentors of the nurse students participated in short clinical mentor’s training (20 h) where the basic principles of PBL and evidence based nursing (EBN) were dealt with. In long-term care nursing, the student nurses’ clinical mentors are registered nurses (RN) and enrolled nurses (EN). The clinical mentors who took part this project were also both RNs and ENs. The idea was that the clinical mentors and teacher worked together in small group tutorials to promote students’ learning. The idea was also to narrow the theory–practice gap and to develop evidencebased nursing students’ clinical practice in longterm care. A further clear need was to increase clinical mentors’ knowledge of PBL and EBN, and to develop their interaction, team-work and information-searching skills. In this article, the purpose is to describe tutorial working in clinical practice from the point of view of student nurses’ learning.
Background In Finland during the last decade there has been a tendency to develop health care and nursing education according to the principles of problem-based learning. The essential characteristics of problem based learning are a curriculum organized around problems that rise of clinical practise and are relevant to desired learning outcomes, and conditions that facilitate small-group working, self-directed learning, independent study, functional knowledge, critical thinking, lifelong learning, and self-evalua-
421
tion (Rideout and Carpio, 2001). According to Rideout et al. (2002) and Tompkins (2001), PBL is suitable for nursing education. PBL develops essential professional competencies such as critical thinking, interaction, relationship and self-evaluation skills. It requires information search and management skills and verbal and nonverbal communication skills. One of the benefits is that PBL develops self-evaluation and peer evaluation skills which are valuable in employment. It is also important that a tutor can promote an atmosphere where it is safe to learn to give and get honest peer review of team work (Chaves et al., 2006) PBL nursing education increases empowerment in nursing students (Siu et al., 2005) and problem-solving skills (Uys et al., 2004). PBL gives an opportunity for the nurse students to develop their information retrieval and management skills, and work on self- and individual professional development. It also gives an opportunity for them to collaborate with other students and define learning goals (Siu et al., 2005). Because of these several good reasons in nursing education in Turku University of Applied Sciences, the pedagogy strategy in nursing curricula is problem based learning. According to Rideout et al. (2002) in a study comparing PBL-based educated nursing students and traditionally educated nursing students it was shown that PBL-students were more satisfied with their education than traditionally educated nursing students. They were more satisfied with the tutors, independence, evaluation strategies and education. There were no statistically significant differences in preparedness for nursing and clinical work. The interaction and self-orientation skills of the PBL students were more proficient than those of the traditionally educated nursing students. When comparing lecture based and PBLbased education it was shown that there were no statistically significant differences between students’ learning. This supports the idea that PBL is as effective as a learning strategy than traditional methods. It is worth of considering other benefits when deciding the learning method. (Beers, 2005). These other benefits are, e.g. self-evaluation and peer-evaluation skills, critical thinking and relationship skills. However, it was notable that one year after the end of the course when tested, the PBL group gained statistically significantly better scores than the traditionally taught group (Beers and Bowden, 2005). The course was Adult Health I Nursing course and the content was diabetes. The test was basic knowledge test of diabetes (Beers, 2005; Beers and Bowden, 2005). In medical education PBL tutorials are carried out during the clinical practice period. According
¨a R.-L. A ¨ri et al.
422 to O’Neill et al. (2000) both students (especially women) and tutors have been very satisfied with tutorial work during the clinical practice period. The students have been satisfied with team working, thinking of learning aims and integrating clinical practice experience with other knowledge. One challenge in medical education has been the development of students’ self-evaluation and taking the tutorial scenario from real patient cases. Further challenges are also practical issues such as time schedules (Morrison et al., 2003). According to Ax and Kincade (2001) nursing students can experience the use of nursing research in clinical practice as useless, and they can also feel that clinical mentors do not support them enough in using research information in nursing. Nursing tends to be based on practical experience, rituals and not on utilization of research in developing nursing. This is a challenge in which both nursing education and hospital organisations have to respond to. A great deal of research has been carried out into PBL in nursing education in the five last years, but only very little research has been done on PBL tutorial work in student nurses’ clinical practice.
Methods Research questions The following research questions were addressed: 1. What kind of PBL-learning skills did the nursing students assess they had before and after PBL clinical practice?
2. What kind of interaction skills with old patients did the nursing students assess they had before and after PBL clinical practice? 3. What kind of attitudes did the nursing students have towards information retrieval and longterm care before and after PBL clinical practice?
Method The study was a questionnaire study which was based on student nurses’ self-reporting pre- and post-test. An intervention was carried out during 2006 on six long-term care wards in the Turku Health Care Department. The students access randomly the study. It depends on that who practiced on the project wards. The student nurses did not know the project wards in advance.
Intervention The intervention was carried out as one PBL-cycle with two tutorial sessions during the four weeks’ clinical practice period. The tutorial small group working sessions were hold in clinical practice near by the project wards. Otherwise the nursing students’ clinical practice in the long-term care was carried out as before (the discussions between nursing student, teacher and mentor) (see Figs. 1 and 2). Both tutorials I and II took 2 h and the number of participating nursing students ranged from eight to nine. The mentors and teacher worked together as tutors in the tutorials. The project included mentor’s training (20 h) where the basic principles of PBL and EBN were taught. The scenario and the teacher were the same during the
Problem-solving continues…
Knowledge is applied to the problem
The knowledge acquired is discussed and debated critically
Evaluation: Learning process Problem-solving Team work process
Scenario
TUTORIAL 1
Brain-storming
Analysis of output
TUTORIAL 2 Independent study and information gathering
Identification of learning issue
Identification of problem areas
References : Konsell et al. 2006; According Poikela 2001
Figure 1 PBL-cycle model (Konsell et al., 2006, published here with permission of Sairaanhoitaja – journal). (Poikela, 2001 See above-mentioned references for further information.)
Problem-based learning in clinical practice: Employment and education as development partners Middle evaluation discussion
Learning objective discussion
2. week
1. week
423
Final evaluation discussion
4. week Time
3. week
TUTORIAL I Only on project wards
TUTORIAL II Only on project wards Copyright © Turku University of Applied Sciences Reference: Konsell et al. 2006
Figure 2 The timing of discussions and tutorials in clinical practice. (Konsell et al., 2006, published here with permission of Sairaanhoitaja – journal).
intervention. The idea was that at least one to a maximum of three mentors in the project wards participated in the tutorial work. The student nurses worked in tutorials as in the school environment (see Fig. 1). One of the nursing students kept a diary of the tutorials. Altogether four PBL-cycles were carried out during 2006.
Questionnaire The students were tested before and after the clinical practice period with a self-reporting questionnaire. The students complete the questionnaire before the clinical practice in the university in an information session. During the last week in clinical practice the students were given the post-test and they were meant to complete it after clinical practice in their spare time. Completing the questionnaire took time approximately 10 min. The questionnaire was based on students’ self-ratings (Likert 1–7, 1 = very poor and 7 = excellent, or 1 = never and 7 = always, or 1 = fully disagree and 7 = fully agree, or 1 = dissatisfied and 7 = satisfied). The questionnaire included items of ‘‘PBL-learning Table 1
skills’’, ‘‘interaction with old patients’’, ‘‘attitudes towards information retrieval’’ and one item of ‘‘interested to work in long-term care’’ and one item of ‘‘own attitude towards long-term care’’. In the post-test the student nurses were also asked questions related to items such as ‘‘action when extra-information was needed’’ and ‘‘satisfaction with clinical practice’’. The items were all positive predetermined. The questionnaire and setting was pilot tested in autumn 2005 (reported Konsell et al., 2006). No changes were made after that. After that the student nurses’ identification was carried out. The questionnaire without the section ‘‘interaction with old patients’’ was also previously used in a clinical practice development project in the Central University Hospital of Turku in 2004. The cronbach’s a of every sum variable are reported in Table 1.
Ethical considerations Research permission for the study was received from the relevant organisation of Turku University of Applied Sciences. The students were informed
Sum variables of the questionnaire
Sum variable
N of items
Cronbach’s a
1 2 3 4 5 6 7 8
8 9 7 8 9 6 7 3
0.76 0.87 0.73 0.86 0.90 0.43 0.69 0.70
PBL-learning skills – before the clinical practice Interaction with old patients – before the clinical practice Attitudes towards information retrieval – before the clinical practice PBL-learning skills – after the clinical practice Interaction with old patients – after the clinical practice Action when extra-information was needed Attitudes towards information retrieval – after the clinical practice Satisfaction with clinical practice
¨a R.-L. A ¨ri et al.
424 of the aim of the project before the clinical practice in an information session. An information sheet about the project was also distributed stressing anonymity and the voluntary nature of the study. The participants’ informed consent was not asked separately, because it was assumed that participation in the study was a sign of willingly being involved in this study. Permission from the Ethical Committee was not needed because the focus of the study was on the health care professional – nursing students. The autonomy of the students was protected. Identification in pre- and post-test and data saving and analysis were done by a different person. The students were able to leave study at any time, e.g. completing the pre-and post-test questionnaires. Instead of that the tutorial small group work was obligatory because it was the students work during the first clinical practice and formed thereby a part of their clinical practice performance.
Results The data were analysed by SPSS 13.0 for Windows. The purpose was to compare the data before and after the PBL clinical practice. Sum variables were formed (see Table 1). Cronbach’s a of sum variables ranged from 0.43 to 0.90. Non parametric tests were used (Wilcoxon-test) and the significance level was set at 0.05. The mean age of students was 21 years and most were female (95%). Previous health care education had only a minority of students (13%). This means previous health care school level education such as enrolled nurse or practical nurse. The data were collected during 2006 and the average response rate was 97% (see Table 2).
Table 2
Demographics of nurse students
Age: N Mean Min Max
40 21 19 34
Gender: N Female Male
40 38 2
Previous health care exam: N Yes No
39 5 34
The means of the sum variables were compared between pre- and post-test (see Fig. 3). The variables ‘‘difference between PBL learning skills before and after’’, ‘‘difference between interaction with old patients before and after’’, and ‘‘attitudes towards information retrieval before and after’’ (see Fig. 3) were also formed. There were clear differences in PBL learning skills (Wilcoxontest, p < 0.001) and in interaction with old patients before and after (Wilcoxon-test, p < 0.001). There was no statistically significant difference in attitudes towards information retrieval before and after clinical practice (Wilcoxon-test, p = 0.793). The mean in the sum variable ‘‘Action when the extra-information was needed’’ was 4.9 (1 = never, 7 = always). Cronbach’s a in this variable was low 0.43, because it constituted items of a quite different kind. Therefore, the means of the items were also examined separately. The means of the items (6) were ‘‘I repeated the information of the lectures 4.7’’, ‘‘I searched information of books 4.4’’, ‘‘I asked the thing nurse 6.1’’, ‘‘I discussed with my college student 5.7’’, ‘‘I asked the teacher 4.2’’ or ‘‘I searched information of Internet 4.2’’. The student nurses were satisfied with their clinical practice. The mean was 5.6. The atmosphere of the ward was examined separately. The students were satisfied with it, with a mean of 5.2. ‘‘Interested in work in long-term care’’ and ‘‘own attitude towards longterm care’’ had increased between pre- and posttest. The mean increased from 3.63 to 4.08 in ‘‘interested in work in long-term care’’. Additionally, the mean increased from 5.48 to 6.13 in ‘‘own attitude against long-term care’’.
Discussion This project showed that tutorial small group working worked well as a learning method in clinical practice. Students’ PBL learning skills and interaction with old patients improved during the project. In this study the clinical practice period was the students’ first clinical practice. They had no previous experience of other kinds of clinical practice. Theory education in classrooms was based on PBL and tutorials. It can be that this also had a transfer effect to the students’ clinical practice. PBL experience in clinical practice can increase nursing students’ interest in work and can also positively change attitudes towards long-term care. The atmosphere of a ward has an effect on students’ learning experience (Saarikoski, 2002). In this study, students’ evaluation of the atmosphere of the ward was good. This encourages the idea that
Problem-based learning in clinical practice: Employment and education as development partners
425
Mean
7 6 5 4 3 2 1
At
t.
lo
ng
-t 2
-t 1
t2
ng
In At
t.
lo
t1 In
t
os tm
f
At
Sa
in E-
td
td At
In
Ld
t2
PB
At
t1
t2
At
In
t1 In
L PB
PB
L
1
2
0
PBL 1= PBL-learning skills – before the clinical practice PBL 2 = PBL-learning skills – after the clinical practice Int 1 = Interaction with old patients - before the clinical practice Int 2 = Interaction with old patients - after the clinical practice Att 1 = Attitudes towards information retrieval - before the clinical practice Att 2 = Attitudes towards information retrieval - after the clinical practice PBL – d = difference between PBL 1 and 2 Int – d = difference between interaction 1 and 2 Att –d = difference between attitudes 1 and 2 E – inf = Action when extra-information was needed Sat = Satisfaction to clinical practice Attmos. = atmosphere of ward Int 1 = interested in work in long-term care before the clinical practice Int 2 = interested in work in long-term care after the clinical practice Att. long-t 1 = own attitude towards long-term care before the clinical practice Att. long-t 2 = own attitude towards long-term care after the clinical practice
Figure 3
Means between pre- and post-test.
PBL clinical practice can have a positive effect on atmosphere and nursing students’ learning experiences. Students’ action when extra information was needed shares similarities with previous research (Elomaa, 2006). Students ask nurses – mentors more often than they search evidence, e.g. from the Internet. The importance of carrying out tutorial working in clinical practice can also be based on a fruitful dialogue of evidence based nursing between mentors, students and their teachers. PBL is usually integrated into class room theory education. Many evaluation strategies show that PBL has a positive effect on nursing students and that the PBL method has exceeded the expectations of nurse educators (e.g. Alexander et al., 2002). The development of collaboration between teacher and clinical mentor is important because it creates the best possible clinical practice environment for nurse students. Good collaboration is necessary to ensure clinical practice which can offer learning experiences for the students learning needs (Papp et al., 2003). Small group working – tutorials – together with nursing students, clinical mentors and teachers can be a forum, where it is possible to discuss relevant clinical learning issues from the point of view of nursing students and in general nursing. The process of PBL is designed to facilitate students’ independence, self-direction, critical thinking, decision-making, interaction and group skills (e.g. Crookes et al., 2001). Additionally, PBL is fun, interesting, exciting and a motivation learning
strategy, which is worth developing in nursing students’ clinical practice (Cooke and Moyle, 2001). The results of this study also support this idea. PBL students can network and release emotions easily during clinical practice, e.g. in small group sessions. In these sessions the purpose is to discuss and solve practical nursing problems, and use research and clinical experience as help. This supports the idea of evidence based nursing practice. Nurse students and medical students working together in clinical practice in a small group could improve multi-professional collaboration, professional roles and the understanding of other professionals. The gap between theory and practice can be narrowed only when the nurses share the ideology of evidence based nursing. Critical and creative thinking are preconditions to this. It is valuable that nurses and educators consider together how to introduce practical nursing research which can be useful in clinical nursing practice (Seymour et al., 2003). PBL is a student centred learning method which is based on good information management. A precondition to evidence based clinical nursing is that nurses are experts in handling information and that they have positive attitudes towards life-long learning. Today students have to be capable of effectively evaluating nursing and professional growth. Nursing students have to be encouraged to develop their information search and management, critical appraisal, and implementation skills as all these skills are needed in nursing (e.g.
¨a R.-L. A ¨ri et al.
426 Barnard et al., 2005). In tutorials during clinical practice these skills can be practised together with students, mentors and teachers. This is especially useful for mentors updating the knowledge, skills and attitudes of EBN and for teachers updating their knowledge of clinical practice. The students can utilize the expert skills of both professionals, nurses and teachers in their learning. The kind of intervention study presented here involves some inherent problems. Firstly, the sample size was quite small. Secondly, any intervention study in nursing education research is always a great challenge because of many intermediating variables (in our case, e.g. mentors and wards). Thirdly, studies which are based on students’ self-ratings are often problematic. Nursing students in this study evaluated their skills and abilities already in the pre-test to the upper half of the Likert scale 1–7. Therefore, the results of this study are preliminary but encourage the development of PBL in the clinical learning environment. There are some practical issues worth of considering when organizing small group working in nurse students’ clinical practice: (1) The scenario has to be fruitful and current both for nurse students’ learning goals and for clinical practice, (2) there has to be good space for small group work and information search (e.g. from Internet) near by ward, (3) The teacher has to arrange the timetable of small group work together with mentors in good time, (4) The nurse administration of clinical practice has to be involved of this kind of nurse students supervision and they have to see this as professional development for mentors as well, (5) The teacher and mentors of clinical practice have to be capable for collaboration. Both teacher and mentors are valuable to support nurse students learning in small group work in clinical practice. And (6) the diary of the tutorials and especially the outcome of the small group work is important to inform to the other nursing staff of ward.
Conclusions Tutorial small group working worked well as a learning method in clinical practice. Tutorial working during clinical practice is fruitful from the point of view of collaboration in students’ supervision with mentors and teachers. In tutorial sessions one benefit is the dialogue of evidence based nursing in clinical practice. We can also stress the importance of developing the problem-solving, interaction and teamwork skills of nursing students, mentors and teachers.
There are also some practical issues worth of considering when organizing tutorial small group working sessions in nurse students’ clinical practice: (1) The scenario has to be fruitful and current both for nurse students’ learning goals and for clinical practice. (2) There has to be good space for small group work and information search (e.g. from Internet) near by the ward. (3) The teacher has to arrange the timetable of tutorial small group work together with mentors in good time. (4) The nurse administration of clinical practice has to be involved of this kind of nurse students supervision and they have to see this as professional development for mentors as well. (5) The teacher and mentors of clinical practice have to be capable for collaboration. Both teacher and mentors are valuable to support nurse students’ learning in small group work in clinical practice. And (6) the diary of the tutorials and especially the outcome of the tutorial small group work are important to inform to the other nursing staff of the ward.
References Alexander, J., McDaniel, G., Baldwin, M., Money, B., 2002. Promoting, applying, and evaluating problem-based learning in the undergraduate nursing curriculum. Nursing Education Perspectives 23 (5), 248–253. Ax, S., Kincade, E., 2001. Nursing students’ perceptions of research: usefulness, implementation and training. Journal of Advanced Nursing 35 (2), 161–170. Barnard, A., Nash, R., O’Brien, M., 2005. Information literacy: developing lifelong skills through nursing education. Journal of Nursing Education 44 (11), 505–510. Beers, G., 2005. The effect of teaching method on objective test scores: problem-based learning versus lecture. Journal of Nursing Education 44 (7), 305–309. Beers, G., Bowden, S., 2005. The effect of teaching method on long-term knowledge retention. Journal of Nursing Education 44 (11), 511–514. Chaves, J.F., Baker, C., Chaves, J.A., Fisher, M., 2006. Self, peer and tutor assessments of MSN competencies using the PBL-evaluator. Journal of Nursing Education 45 (1), 25–31. Cooke, M., Moyle, K., 2001. Students’ evaluation of problembased learning. Nurse Education Today 22 (4), 330–339. Crookes, D., Lunyk-Child, O., Patterson, C., LeGris, J., 2001. Facilitating self-direct learning. In: Rideout, E. (Ed.), Transforming Nursing Education Through Problem-Based Learning. Jones and Bartlett Publishers, Sudbury, pp. 51–74. Elomaa, L., 2006. Research evidence implementation and its requirements in nursing education. Annales Universitatis Turkuensis, ser D, tom 532, Doctoral thesis. ¨a Konsell, O., A ¨ri, R-L., Elomaa, L., Ylo ¨nen, M., 2006. Problembased learning in clinical practice – employment and education as development partners. In Finnish. English abstract. Sairaanhoitaja 79 (4), 22–24. Morrison, S., Boohan, M., Jenkins, J., Moutray, M., 2003. Facilitating undergraduate interprofessional learning in healthcare: comparing classroom and clinical learning for nursing and medical students. Learning in Health and Social Care 2, 92–104.
Problem-based learning in clinical practice: Employment and education as development partners O’Neill, P.A., Morris, J., Baxter, C-M., 2000. Evaluation of an integrated curriculum using problem-based learning in a clinical environment: the Manchester experience. Medical Education 34, 222–230. Papp, I., Markkanen, M., Bonsdorff, M., 2003. Clinical environment as a learning environment: student nurses’ perceptions concerning clinical learning experiences. Nurse Education Today 23 (4), 262–268. Poikela, E., 2001. Ongelmaperustainen oppiminen yliopistossa. ¨ ystila In: Poikela, E., O ¨, S. (Eds.), Tutkiminen on Oppimista – ja Oppiminen on Tutkimista. Tampere university Press, Tampere, pp. 101–117. Rideout, E., Carpio, B., 2001. The problem-based learning model of nursing education. In: Rideout, E. (Ed.), Transforming Nursing Education Trough Problem-Based Learning. Jones and Bartlett Publisher, Sudbury, pp. 21–49. Rideout, E., England-Oxford, V., Brown, B., Fothergill-Bourbonnais, F., Ingram, C., Benson, G., Ross, M., Coates, A., 2002. A Comparison of problem-based and conventional curricula in nursing education. Advances in Health Sciences Education 7, 3–17.
427
Saarikoski, M., 2002. Clinical learning environment and supervision. Development and validation of the CLES evaluation scale. Annales Universitatis Turkuensis, ser D, tom 525. Doctoral thesis. Seymour, B., Kinn, S., Sutherland, N., 2003. Valuing both critical and creative thinking in clinical practice: narrowing the research-practice gap? Journal of Advanced Nursing 42 (3), 288–296. Siu, H., Laschinger, H., Vingilis, E., 2005. The effect of problem-based learning on nursing students’ perceptions of empowerment. Journal of Nursing Education 44 (10), 459– 469. Tompkins, C., 2001. Nursing education for the twenty-first century. In: Rideout, E. (Ed.), Transforming Nursing Education Through Problem-Based Learning. Jones and Bartlett Publishers, Sudbury, pp. 1–19. Uys, L.R., Van Rhyn, L.L., Nomthandaso, S.G., McInerney, P., Tanga, T., 2004. Problem-solving competency of nursing graduates. Journal of Advanced Nursing 48 (5), 500–509.
Available online at www.sciencedirect.com