- Email: [email protected]
Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts
Validation of the tool Assessment of Clinical Education (AssCE): A study using Delphi method and clinical experts Anna L¨ofmark, Gunilla M˚artensson PII: DOI: Reference:
S0260-6917(16)30305-7 doi:10.1016/j.nedt.2016.12.009 YNEDT 3455
To appear in:
Nurse Education Today
Received date: Revised date: Accepted date:
25 July 2016 11 November 2016 12 December 2016
Please cite this article as: L¨ ofmark, Anna, M˚ artensson, Gunilla, Validation of the tool Assessment of Clinical Education (AssCE): A study using Delphi method and clinical experts, Nurse Education Today (2016), doi:10.1016/j.nedt.2016.12.009
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Validation of the tool Assessment of Clinical Education (AssCE): A study using Delphi
1,2
, Gunilla Mårtensson RN, PhD, Associate
NU
Anna Löfmark RN, PhD, Associate Professor Professor 1,3
SC R
IP
T
method and clinical experts
Faculty of Health and Occupational Studies, University of Gävle, Sweden
2
Stord/Haugesund University College, Department of Health Sciences, Haugesund, Norway
3
Department of Public Health and Caring Sciences, Uppsala University, Sweden
CE P
TE
D
MA
1
Correspondence: Gunilla Mårtensson Faculty of Health and Occupational Studies, Department of Health and Caring Sciences
AC
University of Gävle, Kungsbäcksvägen 47, 801 76 Gävle, SWEDEN Phone: +46 26 648294; email: [email protected]
Word count: 3884
ACCEPTED MANUSCRIPT
T
INTRODUCTION
IP
In health care professions, such as nursing, it is important to ensure that students are prepared
SC R
to take well-considered decisions and actions in their everyday practice as graduates and to engage in life-long learning (Levett-Jones et al., 2011). The quality of clinical education therefore plays a crucial role in ensuring that nursing students meet the required demands. Use
NU
of assessment tools for systematic and continuous evaluation of clinical education is a
MA
common method of following students´ individual development. Various assessment tools are in use, but no standard exists (Licen and Plazar, 2015; Wu et al., 2015). Given that there are many students, preceptors and clinical lecturers involved in the assessment process and that
TE
D
clinical education environments are dynamic and complex, it is important that the tools in use be subjected to regular audit and critical reviews. The present study describes how the
CE P
assessment tool Assessment of Clinical Education (AssCE) was updated in a validation
AC
process, with the assistance of two groups of experts (clinical lecturers).
ACCEPTED MANUSCRIPT BACKGROUND The frameworks for assessment tools are generally developed with reference to standards
T
established by national boards of nursing or higher education requirements. Internationally,
IP
universities generally have autonomy in designing their curricula, and therefore syllabi and
SC R
assessment tools may differ. Assessment tools typically consist of broad domains such as professional practice, holistic approaches to care and the integration of knowledge,
NU
interpersonal relationships, management of care, and personal and professional development (Butler et al., 2011). In their systematic review, Wu et al. (2015) found that the current trend
MA
is toward using more holistic models of clinical assessment tools, entailing a focus on a combination of knowledge and understanding, clinical skills, interpersonal skills, problem-
D
solving skills, clinical judgment and management skills. They concluded that there is a need
CE P
TE
both to develop such tools and to ensure a reasonable level of validity and reliability.
The AssCE tool, which takes a holistic approach, was developed in Sweden and has been used since 1999. The tool was originally developed using overall guidelines for national and
AC
international requirements as benchmarks, e.g., the Swedish Higher Education Act (SFS, 1992:1434), Swedish Higher Education Ordinance (SFS, 1993:100), and international guidelines for nursing education (ICN, 1997; Salvage and Heijnen, 1997). In 2000, the first version of the tool, with eighteen factors, was evaluated and found to be highly valued by both students and preceptors (Löfmark and Thorell-Ekstrand, 2000). The second evaluation study was carried out six years later. Respondents were nurse researchers who were involved in nursing education. Their views resulted in increasing the number of factors (21 factors in total) (Löfmark and Thorell-Ekstrand 2004). The third evaluation of the AssCE tool was carried out after a revision in line with the Bologna process, a reform in European higher education focusing on learning outcomes (Löfmark and Thorell-Ekstrand, 2014). Clearly
ACCEPTED MANUSCRIPT stated learning outcomes should articulate measurable student achievement, this is, what the students should know, understand and be able to perform after graduation (EHEA, 2005;
T
EQF, 2008). The areas covered in AssCE are: 1) Communication and teaching; 2) The
IP
nursing process; 3) Examinations and treatments; 4) Management and co-operation; and 5)
SC R
Professional approach. Each factor is accompanied by explanatory text corresponding to two levels of goal achievement: „Good achievement of goals‟ and „Very good achievement of goals.‟ The level „Inadequate achievements of goals‟ is also included, but not described as
NU
criteria using explanatory text. Each factor includes a graded scale, which covers the three
MA
levels and is used to take a position on the level of goals achieved during a clinical education period.
TE
D
Assessment of nursing students in clinical education is a complex process in which requirements must be met, such as knowing that students have acquired a unique fusion of
CE P
knowledge, skills and abilities (Cowin et al., 2008). The purpose of assessment is to prepare students for professional work as safe, ethical and accountable nurses (Bourbonnais et al.,
AC
2008). Studies have stressed that quality clinical assessment requires assessors who are trained and have a strong educational and clinical background (Levett-Jones et al., 2011). In their recent review, Helminen et al. (2016) stated that no common assessment practices exist today. They concluded that one key problem with the assessment process is that it is inconsistent and uncertain and that the personal characteristics of students, preceptors and clinical lecturers significantly affect the process. Assessment of students´ performance concerns achievement in relation to learning outcomes. Formative assessment is focused on each individual student´s learning needs, identifying both strengths and areas in need of development as well as providing feedback to students in order to support learning (Black and William, 1998; Ramsden, 2003). Feedback is the central component of effective formative
ACCEPTED MANUSCRIPT assessment (Rushton, 2005). The process of assessment leads to a summative assessment, meaning a summary of the student‟s achievements and a judgment as to whether he/she has
T
achieved the required learning outcomes (Taras, 2005). Assessment practice is closely linked
IP
to the overall teaching context, where preceptors´ and clinical lecturers´ support and feedback
SC R
are important (Butler et al., 2011; Cassidy et al., 2012). In summary, studies have indicated that assessment in clinical education is a complex and collaborative exercise among students,
NU
preceptors and clinical lecturers.
MA
The AssCE tool is used by students, preceptors and lecturers for systematic and continuous assessment to support a dialogue about each student‟s individual development and for both formative and summative assessment. This type of assessment forms the basis of and is a
TE
D
complement to course examinations, which assign students‟ grades. Each period of clinical education has its own, often multidimensional and broad, learning outcomes. The AssCE tool
CE P
provides guidance on how the learning outcomes can be performed in a concrete manner in an authentic, professional nursing care setting. During clinical education, students and preceptors
AC
have their own AssCE tools. Assessment discussions at the halfway point and at the end of the clinical education are prepared separately by students and preceptors, based on the criteria for each factor. The practical information that is added to the AssCE tool stresses that prior to the assessment discussions students should perform self-assessment and preceptors should assess students´ achievement in their own assessment tool. The AssCE tool has been used frequently in nursing programs at the Bachelor‟s level in Sweden, and in nursing programs in Norway and Finland. The importance of using valid assessment tools to determine standards for students‟ knowledge, skills and abilities has been highlighted in the current international literature (Wu et al., 2015; Ossenberg et al., 2016). In order for any assessment tool to remain
ACCEPTED MANUSCRIPT up-to-date, accessible and user-friendly, it must be subjected to recurrent reviews and regular audit.
T
The present paper describes the process of validation of the AssCE tool through a structured
IP
process in two stages. The aims were to a) evaluate and elaborate the AssCE tool using the
SC R
Delphi technique and b) in the second stage, to investigate to what extent there was
MA
NU
congruence between the content of the revised AssCE tool and learning outcomes.
THE VALIDATION PROCESS
TE
D
Study design
The first stage employed the Delphi technique, which is a widely accepted method of
CE P
consensus building among experts concerning a certain topic (Keeney et al., 2006; Hsu and Sandford, 2007). The Delphi technique represents a significant methodological tool for
AC
problem-solving, planning and forecasting (Polit and Beck, 2012). It is a multi-stage process, where each stage builds on the results of the previous one and where a series of repeated rounds is used. A universally agreed-upon proportion for defining the level of consensus does not exist. A commonly accepted strategy for determining consensus is to assign a percentage value to the level of agreement, which Keeney et al. (2006) suggested should be 75% or, as Polit and Beck (2012) recommended, a range from a liberal 51% to a more cautious 70%. In the present study, consensus was considered to have been achieved on statements for which more than 70% of participants gave the same response. In the second stage, audit was used as a technique to capture to what extent congruence was evident between the factors in the
ACCEPTED MANUSCRIPT revised AssCE tool and learning outcomes described in the clinical courses from the
T
participating universities.
IP
Procedure and findings
SC R
The validation process began with a workshop at a university in Sweden in June 2013. Invited to participate in the workshop were clinical nurse lecturers working at universities, especially those with experience of using the AssCE tool in clinical education. The invitation described
NU
the aim of the workshop, which was to discuss use of the tool and suggestions for its further
MA
development. A total of 40 clinical nurse lecturers participated, representing 14 out of the 25 universities offering nursing education in Sweden. During the workshop, the participants worked in smaller groups with questions such as ´Do you have suggestions for changes in or
TE
D
clarifications of the AssCE tool? and ´Do you see trends in the nursing profession role that need to be elaborated in the AssCE tool?´ Written comments from the discussions were
CE P
handed in to be analyzed and compiled by the research group. The workshop participants received oral and written information about the validation study, the aim of which was to
AC
achieve consensus around further development of the AssCE tool based on their comments from the workshop discussions. They were asked about 1) permission to use their comments and 2) permission to participate in the study. All participants gave permission to use their comments, and 35 out of 40 agreed to participate in the further development. Please see Table 1 for the participants’ characteristics.
A statement generator, originating from the AssCE tool, was designed using the written comments from the discussions. The statements generator included fourteen statements concerning the AssCE tool and seven statements concerning the instructions for using and the layout of the tool. In addition, demographical data were collected. The Delphi technique in
ACCEPTED MANUSCRIPT three rounds (from September 2013 to May 2014) was used to reach consensus. The participants received the statement generator by mail together with a stamped reply envelope
T
and were asked to consider the statements on a 5-point scale from 1= totally agree to 5=
IP
totally disagree (in the analysis, alternative 1 and 2 were considered agreement). The response
SC R
rate was 90%, 88% and 90% in the three rounds, respectively. After the first round, we reached consensus on 16 statements, six of the statements lacked consensus and we received two new alternatives that were elaborated in the next round. After the second round, only 3
MA
was achieved for the remaining statements.
NU
statements lacked consensus, and after the third round, the stipulated level of 70% consensus
TE
D
Please insert table 1 about here
In the second stage of the validation process, another workshop was arranged in June 2014 in
CE P
which 46 nurse lecturers took part, and where the new elaborated and revised version of AssCE was tested. The participants had been asked to bring the syllabus (with learning
AC
outcomes) from the last clinical course in the nursing program at their respective universities to this workshop. The participants represented eleven Swedish universities and one Norwegian university (2-7 participants from each university). Eighteen of these nurse lecturers had participated in the first workshop and in the Delphi process. Participants from each university worked in groups and had to consider whether the learning outcomes in the syllabus could be operationalized in the 21 factors included in the revised AssCE tool. An evaluation questionnaire was developed in which the nurse lecturers were asked to identify their level of agreement using a 5-point scale (from 1=“not at all” to 5= “very well”) concerning whether the factors in the AssCE tool corresponded with the learning outcomes. In other words, they were to consider whether the 21 factors in the AssCE tool measured the
ACCEPTED MANUSCRIPT content that had to be examined in the course. The results (Table 2) showed that the learning outcomes could be operationalized in the AssCE factors, with median values of 5 for all
IP
T
factors.
SC R
Please insert table 2 about here
Ethical considerations
NU
Informed consent was obtained from both groups of participants: for the first group, oral
MA
consent in connection with the workshop and written consent before the first round; for the second group, oral consent in connection with the workshop. In accordance with the informed consent, participants were able to withdraw from the study at any time and without
DISCUSSION
AC
CE P
TE
D
consequences.
The present validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, the Delphi technique was used, and obtaining overall consensus was not difficult. The participants were conversant in assessment in clinical education and were experts in this situation. Their knowledge probably influenced their interest in taking part in the study to further develop the tool. The response rate was high in all three rounds, around 90%; in other studies using the Delphi technique, this has been reported to be a problem (Lloyd and Stirling, 2015). The positive results in all three rounds can be assumed to be a result of the detailed information about the Delphi procedure provided in advance and during the rounds. In all, the suggested alterations to the AssCE tool were moderate. In the
ACCEPTED MANUSCRIPT second stage of the validation process, validation of the content of the revised assessment tool was performed. The participants, experts in the field, found high congruence between the
T
factors in the AssCE and learning outcomes in the syllabus. These syllabuses from clinical
IP
courses represented 11 out of 25 universities in Sweden, which is nearly the half amount of
SC R
universities in Sweden and also one university in Norway. In summary, the revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing
NU
education.
MA
Learning in clinical practice is a vital component, and students are exposed to the everchanging clinical environment. Universities have the overall responsibility for the quality of clinical education, and they must establish assessment processes and tools based on standards
TE
D
stated by national boards of nursing or higher education requirements. Recurring updates and audit of assessment tools are needed to demonstrate that students have reached the required
CE P
level of knowledge, skills and abilities to enter the professional nursing area (Helminen et al., 2014) Assessment tools need to be subjected to regular reviews to ensure educational and
AC
quality. Assessment is a matter of making judgments about the quality of students´ performance (Knight, 2006). Use of a validated instrument supports assessment of students in authentic practice settings.
However, assessment of nursing students‟ performance in clinical education is also a question of reliability. It is known that assessment relies on the judgment of the assessor, and assessor variability and bias continue to be a concern (Yanhua and Watson, 2011). Differences in assessors‟ interpretation of the assessment tool impact the reliability of the assessment (Wu et al., 2015). Professional judgment and interpretation are required when assessors give their standpoints. During clinical education, the students´ preceptors and nurse lecturers are
ACCEPTED MANUSCRIPT involved in learning and assessment in the clinical environment, and both groups are dependent on having an assessment tool that is easy to understand and user-friendly. To assist
T
students, preceptors and nurse lecturers, to clarify demands concerning students‟ level of
IP
attainment and to give students feedback, each factor in the AssCE tool has been amended
SC R
such that the goals are described at two different levels using behavioral terms and everyday language. (Please see Figure 1 for a short description). Clearly articulated behavior statements based on and clearly connectable to the course learning outcomes are therefore of great
NU
importance and may promote consistency of interpretation (Ossenberg and Hendersson,
MA
2015).
To ensure a situation in which assessment of students‟ performance is both valid and reliable,
TE
D
it is necessary to have: 1) a user-friendly, validated tool, 2) clear instructions on how to use the tool and 3) knowledgeable users. Studies have stressed that quality clinical assessment
CE P
requires assessors who are trained and have a strong educational and clinical background (Levett-Jones et al., 2011). In their recent review, Helminen et al. (2016) stated that no
AC
common assessment practices exist today. They concluded that one key problem with the assessment process is that it is inconsistent and uncertain and that the personal characteristics of students, preceptors and clinical lecturers significantly affect the process.
Methodological considerations At first, the Delphi technique is characterized by the anonymity of the experts, the goal being to avoid group dominance. However, true anonymity is often difficult to achieve, because the participants need to be known to the researchers and because of the need for follow-up and feedback (Hasson et al., 2000). The findings represent an expert opinion, rather than fact. Interpretation of the findings rests on the justification of detailed decision-making and rigor in
ACCEPTED MANUSCRIPT the execution of the study (Powell, 2003). The existence of consensus found using the Delphi technique in the first stage and audit of the AssCE tool and learning outcomes in the second
T
stage do not mean that there is no uncertainty involved in interpreting the study. Instead the
IP
methods and findings can be used to enhance the validity of the findings (Hasson et al., 2000).
SC R
The results from this validation process are dependent on the expertise and honesty of the groups of clinical nurse lecturers. They can be seen as relevant experts in the field. Anonymity (in relation to other participants) in the part where the Delphi technique was used
NU
and in the discussions in the group of clinical nurse lecturers when filling in the questionnaire
MA
should have encouraged frank responses. However, that 18 clinical nurse lecturers participated in both stages could be seen as a limitations. Their participation in the second stage of the validation process may have biased the findings.
TE
D
CONCLUSION
The revised AssCE tool seems to meet its objective, which is to be a valid assessment tool for
CE P
use in clinical nursing education. To ensure valid assessment of nursing students‟ performance, the user-friendly AssCE tool, with its clear instructions, is recommended.
AC
However, the reliability of the assessment situation depends on the user, and further evaluation studies among clinical lecturers, preceptors and students are needed in the Nordic countries. In an international perspective nursing education has much in common; the results from the present validation process may entail an interest from universities and nursing colleagues outside the Nordic Countries of testing and using the AssCE tool.
ACCEPTED MANUSCRIPT REFERENCES Black, P. William, D., 1998. Assessment and classroom learning. Assessment in Education 5 (1), 7-74.
IP
T
Bourbonnais, FF., Langford, S., Giannantonio, L., 2008. Development of a clinical evaluation tool for baccalaureate nursing students. Nurse Education in Practice 8(1), 62–71.
SC R
Butler, MP., Cassidy, I., Quillinan, B,. Fahy, A., Bradshaw, C., Tuohy, D., O'Connor, M., Mc Namara, MC., Egan, G., Tierney, C., 2011. Nurse Education in Practice 11(5), 298-303.
NU
Cassidy, I., Butler, MP., Quillinan, B., Egan, G., McNamara, MC., Tuohy, D., Bradshaw, C. , Fahy, A.,Connor, MO., Tierney, C., 2012. Preceptors view of assessing nursing students using a competency based approach. Nurse Education in Practice 12(6), 346-351.
MA
Cowin, LS., Hengstberger-Sims, C., Eagar, SC., Gregory, L., Andrews, S., Rolley, J., 2008. Competence measurements: testing convergent validity for two measures. Journal of Advanced Nursing 64(3), 272-277.
D
EHEA, 2005. The Framework of qualifications for the European Higher Education. Bologna Working Group on Qualifications Frameworks, Ministry of Science, Technology and Innovation. Copenhagen: Ministry of Science, Technology and Innovation.
CE P
TE
EQF. 2008. Office for Official Publications of the European Communities. European Qualification Frameworks for Lifelong Learning. Luxembourg: Office for Official Publications of the European Communities. Hasson, F., Keeney, S., McKenna, H. 2000. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 32(4), 1008-1015.
AC
Helminen, K., Tossavainen, K., Turunen, H., 2014. Assessing clinical practice of student nurses: Views of teachers, mentors and students. nurse Education Today 34(8), 1161-1166. Helminen, K., Coco, K., Johnson, M., Turunen, H., 2016. Summative assessment of clinical practice of student nurses: a review of the literature. International Journal of Nursing Studies 53, 308-319. Hsu, C., Sandford, B., 2007. The Delphi technique making sense of consensus. Practical Assessment Research and Evaluation 12, 1-8. ICN. 1997. Nursing education: past to present. Vol. I: Current and future trends. No 27. Geneve. Keeney, S., Hasson, F., McKenna, H., 2006. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing 53, 205-212. Knight, P., 2006. The local practices of assessment. Assessment & Evaluation in Higher Education 31(4), 435-452.
ACCEPTED MANUSCRIPT Levett-Jones, T., Gersbach, J., Arthur, C., Roche, J., 2011. Implementing a clinical competency assessment model that promotes critical reflection and ensures´ graduates readiness for professional practice. Nurse Education in Practice 11(1), 64-69.
IP
T
Licen, S., Plazar, N., 2015. Identification of nursing competency assessment tools as possibility of their use in nursing education in Slovenia – A systematic literature review. Nurse Education Today 35(4), 602-8.
SC R
Lloyd, B., Stirling, C., 2015. A tool to support meaningful person-centered activity for clients with dementia - a Delphi study. BMC Nursing 14:10. doi: 10.1186/s12912-015-0060-3.
NU
Löfmark, A., Thorell-Ekstrand, I., 2000. Evaluation by nurses and students of a new assessment form for clinical nursing education. Scandinavian Journal of Caring Sciences, 14 (2), 89-96.
MA
Löfmark, A., Thorell-Ekstrand, I., 2004. An assessment form for clinical nursing education: a Delphi study. Journal of Advanced Nursing, 48 (3), 291-298. Löfmark, A., Thorell-Ekstrand, I., 2014. Nursing students' and preceptors' perceptions of using a revised assessment form in clinical nursing education. Nurse Education in Practice 14(3), 275-280.
TE
D
Ossenberg, C., Henderson, A., 2015. Determining attainment of nursing standards: The use of behavioural cues to enhance clarity and transparency in student clinical assessment. Nurse Education Today 35, 12-15.
CE P
Ossenberg, C., Dalton, M., Henderson, A., 2016. Validation of the Australian Nursing Standards Assessment Tool (ANSAT): A pilot study. Nurse Education Today 36, 23-30.
AC
Polit, D., Beck, C., 2012. Nursing Research. Principles and Methods. 9th Edition. Philadelphia: Lippincott Williams & Wilkins. Powell. C., 2003. The Delphi technique: myths and realities. Journal of Advanced Nursing 41 (4), 376-382. Ramsden, P., 2003. Learning to teach in Higher Education, second ed. Routledge: London. Rushton, A., 2005. Formative assessment: a key to deep learning? Medical Teacher 27(6) 509-513. Salvage, J., Heijnen, S., 1997. Nursing in Europe: a resource for better health. Copenhagen: WHO Regional Publications, European Series, No. 74. Swedish Higher Education Ordinance (SFS 1993:100) Swedish Council for Higher Education. Stockholm Swedish Higher Education Act (SFS 1992:1434) Swedish Council for Higher Education. Stockholm Taras, M., 2005. Assessment – summative and formative – some theoretical reflections. British Journal of Educational Studies 53(4), 466-478.
ACCEPTED MANUSCRIPT Wu, X., Enskär, K., Lee, C., Wang, W., 2015. A systematic review of clinical assessment for undergraduate nursing students. Nurse Education Today 35(2), 347-359.
AC
CE P
TE
D
MA
NU
SC R
IP
T
Yanhua, C., Watson, R., 2011. A review of clinical competence in nursing. Nurse Education Today 31(8), 832-836.
ACCEPTED MANUSCRIPT Instructions for use of the AssCE tool (a summary)
IP
T
The purpose of the AssCE tool is to serve as an aid and support in assessment of students‟ professional development during clinical practice. The factors in the AssCE tool exemplify the goals in the syllabus for the various clinical courses. Every factor in the AssCE tool includes written descriptions formulated as evaluation criteria related to the levels ‟Good achievement of goals‟ and ‟Very good achievement of goals‟. To help in rating the student‟s development in relation to each factor, a scale is draw as a line with marked intervals. If the decision is ‟Inadequate achievement of goals‟ the reason for such an assessment shall be given in the space provided for ‟Comments‟.
NU
SC R
Planning discussion A discussion of the student‟s knowledge, previous experience and expectations should result in an overall plan for how the student is to achieve the goals of the training, where the factors in the AssCE tool can be used as a basis for this discussion. requent and regular feed a on students performance during the training period is a prerequisite for continued development and an opportunity for students who need more training in certain areas to be made aware of this.
MA
Mid-course discussion Student and the preceptor prepare the assessment discussion by bringing examples of patient care situations for the discussion and making ratings in the scale for each factor. The student shall be given and take upon him/herself a highly active role in the discussions. The clinical lecturer contributes to the discussion and influences its content such that the assessment also becomes a learning opportunity. The student shall receive clear information about his/her strengths and weaknesses in relation to the assessment criteria. The discussion is summarized by the preceptor or lecturer on the last page of the AssCE tool.
CE P
Example of the AssCE tool
TE
D
Final assessment Should be prepared and carried out in the same way as the mid-course discussion. The discussion is summarized in an assessment that is documented on the assessment form, and the participants sign the tool. The lecturer is responsible for determining the student‟s marks using the pre eptor‟s viewpoints as supporting information.
1. Communication and interaction with patients
AC
Inadequate achievement of goals
Good achievement of goals
Very good achievement of goals
Communicates with patients in an engaged manner, Listens. Shows respectand empathy.
Adapts communication to the patient’s needs, e.g., in cases of communication difficulties. Gives the patientadequate room in the dialogue.
Comments:………………………………………………………………………………………………………………………...
Figure1. Summary of instructions for use of the AssCE tool and example of one of the 21 factors “Communication and interaction with patients”
AC
CE P
TE
D
MA
NU
SC R
IP
T
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT Table 1. Characteristics of the clinical nursing lecturers who participated in the elaboration using Delphi technique
T
52 (8.1) 9 (7.6)
IP
f (%) 1 (3) 31 (88) 3 (9)
AC
CE P
TE
D
MA
NU
Age Experience as nursing lecturer (years) Education level* Ba helor‟s degree Master‟s degree Doctoral degree *n=34, one internal missing data
Mean (SD)
SC R
Nursing teachers (n=35)
ACCEPTED MANUSCRIPT
median (Range)a
1.Communication and interaction with patients
5 (2)
2. Communication and encounter with family and friends
T
Factors in the AssCE tool validated against the learning outcomes in clinical courses (n=12)
IP
Table 2. Factors in the revised AssCE tool validated against learning outcomes in clinical courses from twelve universities.
SC R
5 (2)
3. Co-operation with various authorities within nursing and health care
5 (2)
4. Informs and teaches patients and family/friends
5 (1) 5 (2)
NU
5. Informs and teaches colleagues and students 6. Des ri es patients‟ nursing are needs
MA
7. Plans and prioritizes nursing care measures 8. Carries out nursing care measures
5 (1) 5 (1) 5 (0) 5 (0)
10. Reports, documents and keeps records *
5 (1)
11. Participates in and carries out examinations and treatments
5 (3)
12. Administers medications
5 (4)
TE
D
9. Follows up needs/problems and nursing care measures
5 (2)
14. Co-operates
5 (1)
CE P
13. Plans, organizes and allocates work assignments
5 (3)
16. Safety awareness
5 (1)
AC
15. Readiness to act *
17. Scientific awareness *
5 (1)
18. Ethical awareness „
5 (0)
19. Self-knowledge *
5 (4)
20. Thoroughness, reliability and judgment **
5 (1)
21. Independence *
5 (4)
a=min-max = 1-5; *n=11 and one internal missing data; **n=10 and two internal missing data
ACCEPTED MANUSCRIPT Research Highligths NET Reasonable level of validity and reliability are needed in tools assessing nursing students performance in clinical education
•
The AssCE tool is widely used in Sweden but also in other Nordic countries
•
The method used for establish validity is Delphi technique and clinical experts
•
The revised AssCE tool meets its objective and could be of interest for universities and nursing colleges also outside the Nordic countries.
AC
CE P
TE
D
MA
NU
SC R
IP
T
•
S0260-6917(16)30305-7 doi:10.1016/j.nedt.2016.12.009 YNEDT 3455
To appear in:
Nurse Education Today
Received date: Revised date: Accepted date:
25 July 2016 11 November 2016 12 December 2016
Please cite this article as: L¨ ofmark, Anna, M˚ artensson, Gunilla, Validation of the tool Assessment of Clinical Education (AssCE): A study using Delphi method and clinical experts, Nurse Education Today (2016), doi:10.1016/j.nedt.2016.12.009
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Validation of the tool Assessment of Clinical Education (AssCE): A study using Delphi
1,2
, Gunilla Mårtensson RN, PhD, Associate
NU
Anna Löfmark RN, PhD, Associate Professor Professor 1,3
SC R
IP
T
method and clinical experts
Faculty of Health and Occupational Studies, University of Gävle, Sweden
2
Stord/Haugesund University College, Department of Health Sciences, Haugesund, Norway
3
Department of Public Health and Caring Sciences, Uppsala University, Sweden
CE P
TE
D
MA
1
Correspondence: Gunilla Mårtensson Faculty of Health and Occupational Studies, Department of Health and Caring Sciences
AC
University of Gävle, Kungsbäcksvägen 47, 801 76 Gävle, SWEDEN Phone: +46 26 648294; email: [email protected]
Word count: 3884
ACCEPTED MANUSCRIPT
T
INTRODUCTION
IP
In health care professions, such as nursing, it is important to ensure that students are prepared
SC R
to take well-considered decisions and actions in their everyday practice as graduates and to engage in life-long learning (Levett-Jones et al., 2011). The quality of clinical education therefore plays a crucial role in ensuring that nursing students meet the required demands. Use
NU
of assessment tools for systematic and continuous evaluation of clinical education is a
MA
common method of following students´ individual development. Various assessment tools are in use, but no standard exists (Licen and Plazar, 2015; Wu et al., 2015). Given that there are many students, preceptors and clinical lecturers involved in the assessment process and that
TE
D
clinical education environments are dynamic and complex, it is important that the tools in use be subjected to regular audit and critical reviews. The present study describes how the
CE P
assessment tool Assessment of Clinical Education (AssCE) was updated in a validation
AC
process, with the assistance of two groups of experts (clinical lecturers).
ACCEPTED MANUSCRIPT BACKGROUND The frameworks for assessment tools are generally developed with reference to standards
T
established by national boards of nursing or higher education requirements. Internationally,
IP
universities generally have autonomy in designing their curricula, and therefore syllabi and
SC R
assessment tools may differ. Assessment tools typically consist of broad domains such as professional practice, holistic approaches to care and the integration of knowledge,
NU
interpersonal relationships, management of care, and personal and professional development (Butler et al., 2011). In their systematic review, Wu et al. (2015) found that the current trend
MA
is toward using more holistic models of clinical assessment tools, entailing a focus on a combination of knowledge and understanding, clinical skills, interpersonal skills, problem-
D
solving skills, clinical judgment and management skills. They concluded that there is a need
CE P
TE
both to develop such tools and to ensure a reasonable level of validity and reliability.
The AssCE tool, which takes a holistic approach, was developed in Sweden and has been used since 1999. The tool was originally developed using overall guidelines for national and
AC
international requirements as benchmarks, e.g., the Swedish Higher Education Act (SFS, 1992:1434), Swedish Higher Education Ordinance (SFS, 1993:100), and international guidelines for nursing education (ICN, 1997; Salvage and Heijnen, 1997). In 2000, the first version of the tool, with eighteen factors, was evaluated and found to be highly valued by both students and preceptors (Löfmark and Thorell-Ekstrand, 2000). The second evaluation study was carried out six years later. Respondents were nurse researchers who were involved in nursing education. Their views resulted in increasing the number of factors (21 factors in total) (Löfmark and Thorell-Ekstrand 2004). The third evaluation of the AssCE tool was carried out after a revision in line with the Bologna process, a reform in European higher education focusing on learning outcomes (Löfmark and Thorell-Ekstrand, 2014). Clearly
ACCEPTED MANUSCRIPT stated learning outcomes should articulate measurable student achievement, this is, what the students should know, understand and be able to perform after graduation (EHEA, 2005;
T
EQF, 2008). The areas covered in AssCE are: 1) Communication and teaching; 2) The
IP
nursing process; 3) Examinations and treatments; 4) Management and co-operation; and 5)
SC R
Professional approach. Each factor is accompanied by explanatory text corresponding to two levels of goal achievement: „Good achievement of goals‟ and „Very good achievement of goals.‟ The level „Inadequate achievements of goals‟ is also included, but not described as
NU
criteria using explanatory text. Each factor includes a graded scale, which covers the three
MA
levels and is used to take a position on the level of goals achieved during a clinical education period.
TE
D
Assessment of nursing students in clinical education is a complex process in which requirements must be met, such as knowing that students have acquired a unique fusion of
CE P
knowledge, skills and abilities (Cowin et al., 2008). The purpose of assessment is to prepare students for professional work as safe, ethical and accountable nurses (Bourbonnais et al.,
AC
2008). Studies have stressed that quality clinical assessment requires assessors who are trained and have a strong educational and clinical background (Levett-Jones et al., 2011). In their recent review, Helminen et al. (2016) stated that no common assessment practices exist today. They concluded that one key problem with the assessment process is that it is inconsistent and uncertain and that the personal characteristics of students, preceptors and clinical lecturers significantly affect the process. Assessment of students´ performance concerns achievement in relation to learning outcomes. Formative assessment is focused on each individual student´s learning needs, identifying both strengths and areas in need of development as well as providing feedback to students in order to support learning (Black and William, 1998; Ramsden, 2003). Feedback is the central component of effective formative
ACCEPTED MANUSCRIPT assessment (Rushton, 2005). The process of assessment leads to a summative assessment, meaning a summary of the student‟s achievements and a judgment as to whether he/she has
T
achieved the required learning outcomes (Taras, 2005). Assessment practice is closely linked
IP
to the overall teaching context, where preceptors´ and clinical lecturers´ support and feedback
SC R
are important (Butler et al., 2011; Cassidy et al., 2012). In summary, studies have indicated that assessment in clinical education is a complex and collaborative exercise among students,
NU
preceptors and clinical lecturers.
MA
The AssCE tool is used by students, preceptors and lecturers for systematic and continuous assessment to support a dialogue about each student‟s individual development and for both formative and summative assessment. This type of assessment forms the basis of and is a
TE
D
complement to course examinations, which assign students‟ grades. Each period of clinical education has its own, often multidimensional and broad, learning outcomes. The AssCE tool
CE P
provides guidance on how the learning outcomes can be performed in a concrete manner in an authentic, professional nursing care setting. During clinical education, students and preceptors
AC
have their own AssCE tools. Assessment discussions at the halfway point and at the end of the clinical education are prepared separately by students and preceptors, based on the criteria for each factor. The practical information that is added to the AssCE tool stresses that prior to the assessment discussions students should perform self-assessment and preceptors should assess students´ achievement in their own assessment tool. The AssCE tool has been used frequently in nursing programs at the Bachelor‟s level in Sweden, and in nursing programs in Norway and Finland. The importance of using valid assessment tools to determine standards for students‟ knowledge, skills and abilities has been highlighted in the current international literature (Wu et al., 2015; Ossenberg et al., 2016). In order for any assessment tool to remain
ACCEPTED MANUSCRIPT up-to-date, accessible and user-friendly, it must be subjected to recurrent reviews and regular audit.
T
The present paper describes the process of validation of the AssCE tool through a structured
IP
process in two stages. The aims were to a) evaluate and elaborate the AssCE tool using the
SC R
Delphi technique and b) in the second stage, to investigate to what extent there was
MA
NU
congruence between the content of the revised AssCE tool and learning outcomes.
THE VALIDATION PROCESS
TE
D
Study design
The first stage employed the Delphi technique, which is a widely accepted method of
CE P
consensus building among experts concerning a certain topic (Keeney et al., 2006; Hsu and Sandford, 2007). The Delphi technique represents a significant methodological tool for
AC
problem-solving, planning and forecasting (Polit and Beck, 2012). It is a multi-stage process, where each stage builds on the results of the previous one and where a series of repeated rounds is used. A universally agreed-upon proportion for defining the level of consensus does not exist. A commonly accepted strategy for determining consensus is to assign a percentage value to the level of agreement, which Keeney et al. (2006) suggested should be 75% or, as Polit and Beck (2012) recommended, a range from a liberal 51% to a more cautious 70%. In the present study, consensus was considered to have been achieved on statements for which more than 70% of participants gave the same response. In the second stage, audit was used as a technique to capture to what extent congruence was evident between the factors in the
ACCEPTED MANUSCRIPT revised AssCE tool and learning outcomes described in the clinical courses from the
T
participating universities.
IP
Procedure and findings
SC R
The validation process began with a workshop at a university in Sweden in June 2013. Invited to participate in the workshop were clinical nurse lecturers working at universities, especially those with experience of using the AssCE tool in clinical education. The invitation described
NU
the aim of the workshop, which was to discuss use of the tool and suggestions for its further
MA
development. A total of 40 clinical nurse lecturers participated, representing 14 out of the 25 universities offering nursing education in Sweden. During the workshop, the participants worked in smaller groups with questions such as ´Do you have suggestions for changes in or
TE
D
clarifications of the AssCE tool? and ´Do you see trends in the nursing profession role that need to be elaborated in the AssCE tool?´ Written comments from the discussions were
CE P
handed in to be analyzed and compiled by the research group. The workshop participants received oral and written information about the validation study, the aim of which was to
AC
achieve consensus around further development of the AssCE tool based on their comments from the workshop discussions. They were asked about 1) permission to use their comments and 2) permission to participate in the study. All participants gave permission to use their comments, and 35 out of 40 agreed to participate in the further development. Please see Table 1 for the participants’ characteristics.
A statement generator, originating from the AssCE tool, was designed using the written comments from the discussions. The statements generator included fourteen statements concerning the AssCE tool and seven statements concerning the instructions for using and the layout of the tool. In addition, demographical data were collected. The Delphi technique in
ACCEPTED MANUSCRIPT three rounds (from September 2013 to May 2014) was used to reach consensus. The participants received the statement generator by mail together with a stamped reply envelope
T
and were asked to consider the statements on a 5-point scale from 1= totally agree to 5=
IP
totally disagree (in the analysis, alternative 1 and 2 were considered agreement). The response
SC R
rate was 90%, 88% and 90% in the three rounds, respectively. After the first round, we reached consensus on 16 statements, six of the statements lacked consensus and we received two new alternatives that were elaborated in the next round. After the second round, only 3
MA
was achieved for the remaining statements.
NU
statements lacked consensus, and after the third round, the stipulated level of 70% consensus
TE
D
Please insert table 1 about here
In the second stage of the validation process, another workshop was arranged in June 2014 in
CE P
which 46 nurse lecturers took part, and where the new elaborated and revised version of AssCE was tested. The participants had been asked to bring the syllabus (with learning
AC
outcomes) from the last clinical course in the nursing program at their respective universities to this workshop. The participants represented eleven Swedish universities and one Norwegian university (2-7 participants from each university). Eighteen of these nurse lecturers had participated in the first workshop and in the Delphi process. Participants from each university worked in groups and had to consider whether the learning outcomes in the syllabus could be operationalized in the 21 factors included in the revised AssCE tool. An evaluation questionnaire was developed in which the nurse lecturers were asked to identify their level of agreement using a 5-point scale (from 1=“not at all” to 5= “very well”) concerning whether the factors in the AssCE tool corresponded with the learning outcomes. In other words, they were to consider whether the 21 factors in the AssCE tool measured the
ACCEPTED MANUSCRIPT content that had to be examined in the course. The results (Table 2) showed that the learning outcomes could be operationalized in the AssCE factors, with median values of 5 for all
IP
T
factors.
SC R
Please insert table 2 about here
Ethical considerations
NU
Informed consent was obtained from both groups of participants: for the first group, oral
MA
consent in connection with the workshop and written consent before the first round; for the second group, oral consent in connection with the workshop. In accordance with the informed consent, participants were able to withdraw from the study at any time and without
DISCUSSION
AC
CE P
TE
D
consequences.
The present validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, the Delphi technique was used, and obtaining overall consensus was not difficult. The participants were conversant in assessment in clinical education and were experts in this situation. Their knowledge probably influenced their interest in taking part in the study to further develop the tool. The response rate was high in all three rounds, around 90%; in other studies using the Delphi technique, this has been reported to be a problem (Lloyd and Stirling, 2015). The positive results in all three rounds can be assumed to be a result of the detailed information about the Delphi procedure provided in advance and during the rounds. In all, the suggested alterations to the AssCE tool were moderate. In the
ACCEPTED MANUSCRIPT second stage of the validation process, validation of the content of the revised assessment tool was performed. The participants, experts in the field, found high congruence between the
T
factors in the AssCE and learning outcomes in the syllabus. These syllabuses from clinical
IP
courses represented 11 out of 25 universities in Sweden, which is nearly the half amount of
SC R
universities in Sweden and also one university in Norway. In summary, the revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing
NU
education.
MA
Learning in clinical practice is a vital component, and students are exposed to the everchanging clinical environment. Universities have the overall responsibility for the quality of clinical education, and they must establish assessment processes and tools based on standards
TE
D
stated by national boards of nursing or higher education requirements. Recurring updates and audit of assessment tools are needed to demonstrate that students have reached the required
CE P
level of knowledge, skills and abilities to enter the professional nursing area (Helminen et al., 2014) Assessment tools need to be subjected to regular reviews to ensure educational and
AC
quality. Assessment is a matter of making judgments about the quality of students´ performance (Knight, 2006). Use of a validated instrument supports assessment of students in authentic practice settings.
However, assessment of nursing students‟ performance in clinical education is also a question of reliability. It is known that assessment relies on the judgment of the assessor, and assessor variability and bias continue to be a concern (Yanhua and Watson, 2011). Differences in assessors‟ interpretation of the assessment tool impact the reliability of the assessment (Wu et al., 2015). Professional judgment and interpretation are required when assessors give their standpoints. During clinical education, the students´ preceptors and nurse lecturers are
ACCEPTED MANUSCRIPT involved in learning and assessment in the clinical environment, and both groups are dependent on having an assessment tool that is easy to understand and user-friendly. To assist
T
students, preceptors and nurse lecturers, to clarify demands concerning students‟ level of
IP
attainment and to give students feedback, each factor in the AssCE tool has been amended
SC R
such that the goals are described at two different levels using behavioral terms and everyday language. (Please see Figure 1 for a short description). Clearly articulated behavior statements based on and clearly connectable to the course learning outcomes are therefore of great
NU
importance and may promote consistency of interpretation (Ossenberg and Hendersson,
MA
2015).
To ensure a situation in which assessment of students‟ performance is both valid and reliable,
TE
D
it is necessary to have: 1) a user-friendly, validated tool, 2) clear instructions on how to use the tool and 3) knowledgeable users. Studies have stressed that quality clinical assessment
CE P
requires assessors who are trained and have a strong educational and clinical background (Levett-Jones et al., 2011). In their recent review, Helminen et al. (2016) stated that no
AC
common assessment practices exist today. They concluded that one key problem with the assessment process is that it is inconsistent and uncertain and that the personal characteristics of students, preceptors and clinical lecturers significantly affect the process.
Methodological considerations At first, the Delphi technique is characterized by the anonymity of the experts, the goal being to avoid group dominance. However, true anonymity is often difficult to achieve, because the participants need to be known to the researchers and because of the need for follow-up and feedback (Hasson et al., 2000). The findings represent an expert opinion, rather than fact. Interpretation of the findings rests on the justification of detailed decision-making and rigor in
ACCEPTED MANUSCRIPT the execution of the study (Powell, 2003). The existence of consensus found using the Delphi technique in the first stage and audit of the AssCE tool and learning outcomes in the second
T
stage do not mean that there is no uncertainty involved in interpreting the study. Instead the
IP
methods and findings can be used to enhance the validity of the findings (Hasson et al., 2000).
SC R
The results from this validation process are dependent on the expertise and honesty of the groups of clinical nurse lecturers. They can be seen as relevant experts in the field. Anonymity (in relation to other participants) in the part where the Delphi technique was used
NU
and in the discussions in the group of clinical nurse lecturers when filling in the questionnaire
MA
should have encouraged frank responses. However, that 18 clinical nurse lecturers participated in both stages could be seen as a limitations. Their participation in the second stage of the validation process may have biased the findings.
TE
D
CONCLUSION
The revised AssCE tool seems to meet its objective, which is to be a valid assessment tool for
CE P
use in clinical nursing education. To ensure valid assessment of nursing students‟ performance, the user-friendly AssCE tool, with its clear instructions, is recommended.
AC
However, the reliability of the assessment situation depends on the user, and further evaluation studies among clinical lecturers, preceptors and students are needed in the Nordic countries. In an international perspective nursing education has much in common; the results from the present validation process may entail an interest from universities and nursing colleagues outside the Nordic Countries of testing and using the AssCE tool.
ACCEPTED MANUSCRIPT REFERENCES Black, P. William, D., 1998. Assessment and classroom learning. Assessment in Education 5 (1), 7-74.
IP
T
Bourbonnais, FF., Langford, S., Giannantonio, L., 2008. Development of a clinical evaluation tool for baccalaureate nursing students. Nurse Education in Practice 8(1), 62–71.
SC R
Butler, MP., Cassidy, I., Quillinan, B,. Fahy, A., Bradshaw, C., Tuohy, D., O'Connor, M., Mc Namara, MC., Egan, G., Tierney, C., 2011. Nurse Education in Practice 11(5), 298-303.
NU
Cassidy, I., Butler, MP., Quillinan, B., Egan, G., McNamara, MC., Tuohy, D., Bradshaw, C. , Fahy, A.,Connor, MO., Tierney, C., 2012. Preceptors view of assessing nursing students using a competency based approach. Nurse Education in Practice 12(6), 346-351.
MA
Cowin, LS., Hengstberger-Sims, C., Eagar, SC., Gregory, L., Andrews, S., Rolley, J., 2008. Competence measurements: testing convergent validity for two measures. Journal of Advanced Nursing 64(3), 272-277.
D
EHEA, 2005. The Framework of qualifications for the European Higher Education. Bologna Working Group on Qualifications Frameworks, Ministry of Science, Technology and Innovation. Copenhagen: Ministry of Science, Technology and Innovation.
CE P
TE
EQF. 2008. Office for Official Publications of the European Communities. European Qualification Frameworks for Lifelong Learning. Luxembourg: Office for Official Publications of the European Communities. Hasson, F., Keeney, S., McKenna, H. 2000. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 32(4), 1008-1015.
AC
Helminen, K., Tossavainen, K., Turunen, H., 2014. Assessing clinical practice of student nurses: Views of teachers, mentors and students. nurse Education Today 34(8), 1161-1166. Helminen, K., Coco, K., Johnson, M., Turunen, H., 2016. Summative assessment of clinical practice of student nurses: a review of the literature. International Journal of Nursing Studies 53, 308-319. Hsu, C., Sandford, B., 2007. The Delphi technique making sense of consensus. Practical Assessment Research and Evaluation 12, 1-8. ICN. 1997. Nursing education: past to present. Vol. I: Current and future trends. No 27. Geneve. Keeney, S., Hasson, F., McKenna, H., 2006. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing 53, 205-212. Knight, P., 2006. The local practices of assessment. Assessment & Evaluation in Higher Education 31(4), 435-452.
ACCEPTED MANUSCRIPT Levett-Jones, T., Gersbach, J., Arthur, C., Roche, J., 2011. Implementing a clinical competency assessment model that promotes critical reflection and ensures´ graduates readiness for professional practice. Nurse Education in Practice 11(1), 64-69.
IP
T
Licen, S., Plazar, N., 2015. Identification of nursing competency assessment tools as possibility of their use in nursing education in Slovenia – A systematic literature review. Nurse Education Today 35(4), 602-8.
SC R
Lloyd, B., Stirling, C., 2015. A tool to support meaningful person-centered activity for clients with dementia - a Delphi study. BMC Nursing 14:10. doi: 10.1186/s12912-015-0060-3.
NU
Löfmark, A., Thorell-Ekstrand, I., 2000. Evaluation by nurses and students of a new assessment form for clinical nursing education. Scandinavian Journal of Caring Sciences, 14 (2), 89-96.
MA
Löfmark, A., Thorell-Ekstrand, I., 2004. An assessment form for clinical nursing education: a Delphi study. Journal of Advanced Nursing, 48 (3), 291-298. Löfmark, A., Thorell-Ekstrand, I., 2014. Nursing students' and preceptors' perceptions of using a revised assessment form in clinical nursing education. Nurse Education in Practice 14(3), 275-280.
TE
D
Ossenberg, C., Henderson, A., 2015. Determining attainment of nursing standards: The use of behavioural cues to enhance clarity and transparency in student clinical assessment. Nurse Education Today 35, 12-15.
CE P
Ossenberg, C., Dalton, M., Henderson, A., 2016. Validation of the Australian Nursing Standards Assessment Tool (ANSAT): A pilot study. Nurse Education Today 36, 23-30.
AC
Polit, D., Beck, C., 2012. Nursing Research. Principles and Methods. 9th Edition. Philadelphia: Lippincott Williams & Wilkins. Powell. C., 2003. The Delphi technique: myths and realities. Journal of Advanced Nursing 41 (4), 376-382. Ramsden, P., 2003. Learning to teach in Higher Education, second ed. Routledge: London. Rushton, A., 2005. Formative assessment: a key to deep learning? Medical Teacher 27(6) 509-513. Salvage, J., Heijnen, S., 1997. Nursing in Europe: a resource for better health. Copenhagen: WHO Regional Publications, European Series, No. 74. Swedish Higher Education Ordinance (SFS 1993:100) Swedish Council for Higher Education. Stockholm Swedish Higher Education Act (SFS 1992:1434) Swedish Council for Higher Education. Stockholm Taras, M., 2005. Assessment – summative and formative – some theoretical reflections. British Journal of Educational Studies 53(4), 466-478.
ACCEPTED MANUSCRIPT Wu, X., Enskär, K., Lee, C., Wang, W., 2015. A systematic review of clinical assessment for undergraduate nursing students. Nurse Education Today 35(2), 347-359.
AC
CE P
TE
D
MA
NU
SC R
IP
T
Yanhua, C., Watson, R., 2011. A review of clinical competence in nursing. Nurse Education Today 31(8), 832-836.
ACCEPTED MANUSCRIPT Instructions for use of the AssCE tool (a summary)
IP
T
The purpose of the AssCE tool is to serve as an aid and support in assessment of students‟ professional development during clinical practice. The factors in the AssCE tool exemplify the goals in the syllabus for the various clinical courses. Every factor in the AssCE tool includes written descriptions formulated as evaluation criteria related to the levels ‟Good achievement of goals‟ and ‟Very good achievement of goals‟. To help in rating the student‟s development in relation to each factor, a scale is draw as a line with marked intervals. If the decision is ‟Inadequate achievement of goals‟ the reason for such an assessment shall be given in the space provided for ‟Comments‟.
NU
SC R
Planning discussion A discussion of the student‟s knowledge, previous experience and expectations should result in an overall plan for how the student is to achieve the goals of the training, where the factors in the AssCE tool can be used as a basis for this discussion. requent and regular feed a on students performance during the training period is a prerequisite for continued development and an opportunity for students who need more training in certain areas to be made aware of this.
MA
Mid-course discussion Student and the preceptor prepare the assessment discussion by bringing examples of patient care situations for the discussion and making ratings in the scale for each factor. The student shall be given and take upon him/herself a highly active role in the discussions. The clinical lecturer contributes to the discussion and influences its content such that the assessment also becomes a learning opportunity. The student shall receive clear information about his/her strengths and weaknesses in relation to the assessment criteria. The discussion is summarized by the preceptor or lecturer on the last page of the AssCE tool.
CE P
Example of the AssCE tool
TE
D
Final assessment Should be prepared and carried out in the same way as the mid-course discussion. The discussion is summarized in an assessment that is documented on the assessment form, and the participants sign the tool. The lecturer is responsible for determining the student‟s marks using the pre eptor‟s viewpoints as supporting information.
1. Communication and interaction with patients
AC
Inadequate achievement of goals
Good achievement of goals
Very good achievement of goals
Communicates with patients in an engaged manner, Listens. Shows respectand empathy.
Adapts communication to the patient’s needs, e.g., in cases of communication difficulties. Gives the patientadequate room in the dialogue.
Comments:………………………………………………………………………………………………………………………...
Figure1. Summary of instructions for use of the AssCE tool and example of one of the 21 factors “Communication and interaction with patients”
AC
CE P
TE
D
MA
NU
SC R
IP
T
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT Table 1. Characteristics of the clinical nursing lecturers who participated in the elaboration using Delphi technique
T
52 (8.1) 9 (7.6)
IP
f (%) 1 (3) 31 (88) 3 (9)
AC
CE P
TE
D
MA
NU
Age Experience as nursing lecturer (years) Education level* Ba helor‟s degree Master‟s degree Doctoral degree *n=34, one internal missing data
Mean (SD)
SC R
Nursing teachers (n=35)
ACCEPTED MANUSCRIPT
median (Range)a
1.Communication and interaction with patients
5 (2)
2. Communication and encounter with family and friends
T
Factors in the AssCE tool validated against the learning outcomes in clinical courses (n=12)
IP
Table 2. Factors in the revised AssCE tool validated against learning outcomes in clinical courses from twelve universities.
SC R
5 (2)
3. Co-operation with various authorities within nursing and health care
5 (2)
4. Informs and teaches patients and family/friends
5 (1) 5 (2)
NU
5. Informs and teaches colleagues and students 6. Des ri es patients‟ nursing are needs
MA
7. Plans and prioritizes nursing care measures 8. Carries out nursing care measures
5 (1) 5 (1) 5 (0) 5 (0)
10. Reports, documents and keeps records *
5 (1)
11. Participates in and carries out examinations and treatments
5 (3)
12. Administers medications
5 (4)
TE
D
9. Follows up needs/problems and nursing care measures
5 (2)
14. Co-operates
5 (1)
CE P
13. Plans, organizes and allocates work assignments
5 (3)
16. Safety awareness
5 (1)
AC
15. Readiness to act *
17. Scientific awareness *
5 (1)
18. Ethical awareness „
5 (0)
19. Self-knowledge *
5 (4)
20. Thoroughness, reliability and judgment **
5 (1)
21. Independence *
5 (4)
a=min-max = 1-5; *n=11 and one internal missing data; **n=10 and two internal missing data
ACCEPTED MANUSCRIPT Research Highligths NET Reasonable level of validity and reliability are needed in tools assessing nursing students performance in clinical education
•
The AssCE tool is widely used in Sweden but also in other Nordic countries
•
The method used for establish validity is Delphi technique and clinical experts
•
The revised AssCE tool meets its objective and could be of interest for universities and nursing colleges also outside the Nordic countries.
AC
CE P
TE
D
MA
NU
SC R
IP
T
•