- Email: [email protected]
Structuring Doctor of Nursing Practice project courses to facilitate success and ensure rigor
Journal of Professional Nursing xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
Journal of Professional Nursing journal homepage: www.elsevier.com/locate/jpnu
Structuring Doctor of Nursing Practice project courses to facilitate success and ensure rigor ⁎
Kathleen M. Buckley , Shannon Idzik, Debra Bingham, Brenda Windemuth, Susan L. Bindon University of Maryland School of Nursing, 655 West Lombard Street, Baltimore, MD 21201, United States of America
A R T I C LE I N FO
A B S T R A C T
Keywords: Graduate nursing education Doctoral programs Quality improvement Doctoral students Doctor of Nursing Practice Doctor of Nursing Practice Project
There is currently a lack of consensus on the best format for Doctor of Nursing Practice project deliverables. In this article the project course history, current format, and evaluation methods are described for a Doctor of Nursing Practice program during the transition from a sole post-master's option to one that also admitted postbaccalaureate students. The project course format shifted focus from one in which students independently implemented multiple types of projects under the direction of a chairperson and committee to one in which students carried out projects utilizing quality improvement methods and tools under the mentorship of a project faculty advisor and clinical site representative. The integration of quality improvement models is exemplified through course objectives and assignments. Lessons learned through this transition are provided in the hope that the work may benefit other nursing schools with similar programs.
Introduction A final Doctor of Nursing Practice (DNP) project was noted to be a core requirement of the DNP degree in the Essentials for Doctoral Education for Advanced Nursing Practice (American Association of Colleges in Nursing [AACN], 2006). Examples of what these projects might entail included a variety of formats, such as practice portfolios, evidence based practice change initiatives, integrated literature reviews and quality improvement projects. Schools were allowed to define these projects in a form that best integrated the specialty requirements. As DNP programs evolved, there appeared to be a growing lack of consensus in the appropriate format, requirements, goals, evaluation and dissemination of DNP projects (Brown & Crabtree, 2013). AACN responded to the call for more clarity by establishing a task force to address some of these issues. The report that was generated stated that the requirements for the DNP project should be the same for post-baccalaureate and post-master’s students. The report further stated that all projects should include planning, implementation, and evaluation components; focus on a change that impacts healthcare outcomes either through direct or indirect care; include a plan for sustainability and an evaluation of processes and/or outcomes; and provide a foundation for future practice scholarship (AACN, 2015). However, there is currently some disagreement on the best format for project courses and deliverables, especially as schools admit larger cohorts of students. The purpose of this paper is to describe a nursing
⁎
school's process in developing and modifying DNP project courses for a large cohort of students with the expectation that the structure and processes described may serve as models for other faculty who are directing DNP project courses and working to expand the ability of faculty and students to more effectively translate evidence based practice changes. History of DNP project course changes at UMSON In 2006, the University of Maryland School of Nursing (UMSON) opened a post-master's DNP program. The school enrolled approximately 20–35 students each year in the program and accepted students from all areas of advanced nursing practice including advanced practice nurses, nurse informaticians, public health nurses and nurse leaders. The program included four one-credit pass/fail DNP project courses (see Table 1). Students progressed through the first two courses in a face-to-face seminar format and completed the last two courses independently under the supervision of a chairperson and at least two committee members. The four courses were scheduled to occur in the final four semesters of the students' program. The first project course focused heavily on the history of and movement to the DNP degree. Much time was spent discussing the Essentials for Doctoral Education for Advanced Nursing Practice (AACN, 2006), and understanding roles for which nurses were prepared with a doctoral degree in nursing practice. During this course, students also identified a practice problem
Corresponding author. E-mail address: [email protected] (K.M. Buckley).
https://doi.org/10.1016/j.profnurs.2019.12.001 Received 1 July 2019; Received in revised form 23 November 2019; Accepted 5 December 2019 8755-7223/ © 2019 Elsevier Inc. All rights reserved.
Please cite this article as: Kathleen M. Buckley, et al., Journal of Professional Nursing, https://doi.org/10.1016/j.profnurs.2019.12.001
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Table 1 DNP project course changes from 2006 to 2017. 2006 NDNP NDNP NDNP NDNP
810 811 812 813
Capstone Capstone Capstone Capstone
1: 2: 3: 4:
Professional Development Project Identification Project Implementation Project Evaluation and Dissemination
2014
2017
Deleted course NDNP 811 Scholarly Project Development NDNP 812 Scholarly Project Implementation NDNP 813 Scholarly Project Evaluation and Dissemination
NDNP 810 DNP Project Identification NDNP 811 DNP Project Development NDNP 812 DNP Project Implementation NDNP 813 DNP Project Evaluation and Dissemination
worksite. Their previous relationship with the CSRs helped to encourage buy-in to supporting the student in implementing a QI project. It was preferable, but not essential, that the CSR be doctorally prepared (e.g., DNP, Ph.D., MD). The responsibilities of the CSR were to:
of interest and began a literature search for evidence-based solutions. The second course required students to develop a theory-based proposal for their DNP project. They worked with course faculty to select an evidence-based intervention, complete a full Institutional Review Board (IRB) submission and prepare a “proposal defense” in front of their chairperson and committee members. In the third course, students worked independently with their chairperson and committee members to implement their project. In the fourth and final course, the student completed the project evaluation with the guidance of the committee and finalized a manuscript. A publication-quality manuscript became the final deliverable along with a public “project defense”. As the DNP program evolved, faculty leaders made modifications to the project courses. They felt that a reduction in credits would help with the overall course schedule and that the course/curriculum was too heavily focusing on the rationale for the DNP, arguments for and against the DNP, and roles that DNP prepared graduates would assume upon graduation. In 2014, the first project course was deleted from the course sequence and the term “capstone project” changed to scholarly project in the course titles (see Table 1). A gap between the first and second courses was created in the summer to allow the post-masters' DNP students additional time to complete their IRB review, and to decrease burden on faculty and practice sites during the summer. Unfortunately, the deletion of the first project course in 2014 meant that there were only three DNP project courses. After the first project course was deleted, faculty noted that the students struggled to identify a problem, solution and plan for action in one semester. Faculty and students communicated that there was a mismatch between the amount of work associated with the first project course for the one credit earned. Work began to be shifted to a “pre-class” requirement which required faculty and students to do work outside of the course structure. Practice sites grew weary when students stalled their projects over the summer and occasionally implemented the projects without the students. Students implementing and evaluating the projects independently under the direction of chairpersons also created a perception of poor quality among practice partners due to variation in expectations. In response to the challenges and in preparation for a large cohort of students in the project course sequence due to the start of the postbaccalaureate DNP program in 2014, the DNP project course structure was revised in 2017. The first course in the sequence returned with a stronger focus on proposal development (see Table 1). In the second course students completed the development of their proposal, presented it to stakeholders in the clinical site for approval, and submitted an application to the appropriate IRBs. The term “scholarly project” was replaced by “DNP project” in the course titles. Two cohorts of students have been through the series of DNP project courses. In order to manage the larger cohorts of students (85–130 students) with the limited number of faculty to serve on project committees, all four project courses met as classes rather than as independent student-chairperson dyads supported by a committee. There were 16–23 sections of each project course. A section typically consisted of six students under the direction of one faculty project advisor. The student and faculty project advisor worked with each student's clinical site representative (CSR) to plan, implement and evaluate the project. Students took an active part in the process of finding potential CSRs for projects by seeking out previous preceptors or managers at their
• meet with the DNP student at least monthly to develop a proposal for a QI project in NDNP 810 • listen to a formal presentation of the DNP student's proposal, pro• •
vide written approval and/or recommendations for change, and facilitate the IRB process at the practice site in NDNP 811 guide and support the DNP student in assembling an implementation team, whom the student will lead in implementing practice changes in structures, processes and outcomes in NDNP 812 support the dissemination of the DNP project findings within the organization in NDNP 813
Students were also responsible for getting verbal approval from a sponsor for their project before beginning the first project course. A sponsor was identified as an administrator with the power and leadership skill within the organization to secure resources or remove barriers that would affect the DNP project's implementation effectiveness. The sponsor was also the individual identified by the organization with the authority to approve or close down the project. The classes for each project course were held during weeks 1, 5, 9, 12 and 14 over a 14 week semester via web conferencing, and were 2 h in length. The sections were overseen by one faculty course coordinator, and class announcements occurred through one online forum to maintain consistency among the sections. This significant change necessitated a critical look at course content as the course structure became the driver of project content and quality. The following section describes the current UMSON course structure and content. DNP project course structure and content The project courses currently run over four consecutive semesters or 17 months—beginning in January and ending in May of the following year—with the same faculty project advisors per group for all four courses (see Table 1). This sequence applies to all DNP students. If they miss taking the first project course in the Spring, they have to wait until the following year to begin. In the first course (January–May), the students identify a significant problem at a practice site where there was a gap between the research evidence and the current practice. They then review the evidence to support a best practice that could be implemented through a quality improvement (QI) project to address the identified practice problem. During the second course (June–July), the students finalize their implementation plan and present it to stakeholders at the project site for approval. They also apply for any necessary stakeholder or IRB determination that their project is indeed quality improvement. In the third course (August–December), they operationalize the implementation plan and collect data to evaluate the effectiveness of the implementation effort. In the last course (January–May), they analyze their data, write up their findings and evaluation of their project in a manuscript format, and disseminate their results and recommendations for next steps with stakeholders and in a public forum through a formal poster or podium presentation. Course objectives and relevant assignments to meet those objectives 2
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Table 2 DNP project course objectives and assignments. Course and semester Pre-requisites for NDNP 810 (Fall semester prior to taking NDNP 810)
NDNP 810 DNP Project Identification (Spring semester)
NDNP 811 DNP Project Development (Summer semester)
NDNP 812 DNP Project Implementation (Fall semester)
NDNP 813 DNP Project Evaluation and Dissemination (Spring semester)
Objectives
Assignments
a practice problem that can be formally addressed in a • Identify DNP project appropriate stakeholders at the practice site necessary to • Identify carry out the project
of a problem at a clinical or community site in • Identification need of a quality improvement solution of a clinical site representative and sponsor • Identification (Ishikawa or cause and effect) diagram to categorize the • Fishbone causes and sub-causes of a QI problem within a specific context in
the current literature related to the practice problem • Analyze external evidence to support and confirm the significance • ofCollect the practice problem an appropriate theoretical framework based on • Identify implementation science to guide the DNP project. quality improvement process tools to illustrate the proposed • Use DNP project
Model to identify the resources, activities, outputs, • Logic outcomes, and potential impact of a project Project Proposal in manuscript format: background, • DNP theoretical framework, literature review, and evidence review
stakeholders and partners within the clinical setting • Engage necessary to carry out the project. an appropriate implementation plan for proposed DNP • Develop project a succinct DNP project proposal presentation to faculty • Deliver and clinical partner for approval IRB submission and obtain other necessary approvals • Complete an evidence-based DNP project based on problem • Implement identification, conceptualization of the problem in the context of current practice, evidence and theory, and development of methods for addressing the problem
Analyze and display the findings from the completed DNP project • and make recommendations for sustainability and spread of the
order to identify the root causes of the problem
table
portion of MAP-IT Project Charter Worksheet reviewed by • MAP second reader and including:
• • • • • • • • •
practice changes
changes in practice made, results of the data analysis • Describe and outcomes, and facilitators and barriers encountered project results including a comparison of results with • Discuss findings from other publications, as well as strengths and limitations of project.
Disseminate the DNP project's findings and lessons learned • through a written manuscript, following SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) guidelines, and a formal poster or oral presentation (Ogrinc, Davies, Goodman, Batalden, & Davidoff, 2016)
•
a. Updated fishbone diagram b. Goals c. Process model d. Structure, Process and Outcome Measures e. Run chart format DNP Project Proposal: plan for implementation and abstract added to previous parts of proposal in manuscript format Data collection tools IRB submission for determination as quality improvement with prior review by compliance coordinator (no exempt, expedited or full IRB review) DNP Project Presentation to stakeholders at practice site Implementation of an evidence-based intervention over 14 weeks Updated MAP-IT with IT portion added Force field analysis based on the Consolidated Framework of Implementation Research or a framework to identify the barriers and facilitators of the practice change being implemented (Damschroder et al., 2009) Abstracts submitted to external local, regional and national conferences held in following Spring semester Dissemination requirements a. Present back to stakeholders and at the UMSON DNP Poster Day plus b. One of 2 options i. Local, regional or national conference regional poster or oral presentation ii. Formal presentation within organization (e.g., nursing grand rounds) DNP project proposal converted to final report following SQUIRE Guidelines in manuscript format with results, discussion, conclusion and abstract added
the entire group of students benefit from the feedback. The students peer review assignments in all four project courses. The assignments include their DNP Project Logic Model, evidence review table, project proposal, MAP-IT Project Charter Worksheet (which includes multiple components, such as their Fishbone diagram, process map, measurement and analysis plan), proposal presentation, poster presentation, and the final manuscript. Individual written feedback is provided by the faculty and peer reviewer immediately after the class. While most of the projects are led by individual students, there have been a small number of group projects within each cohort. Based on our previous experience of working with DNP students in group projects, it was decided that the maximum size for a group would be three student members. The group projects consist of smaller separate distinct projects that are parts of a larger initiative or implementation of a similar practice change across different settings. For example, three students worked together in a preoperative center preparing patients for surgery. One of the three students implemented a clinical practice guideline for using chlorhexidine cloths preoperatively. Another student implemented electronic prescribing of preoperative medications, and the third student implemented the screening of patients preoperatively for methicillin-resistant Staphylococcus aureus colonization. Although some
have been developed for each course (see Table 2). Assignments are iterative in that students have an opportunity to get feedback and submit revisions that go through additional reviews. The project proposal is also scaffolded in that it is broken down into smaller parts (i.e., overview and statement of the problem, evidence review, theoretical framework and implementation plan) and then sequenced in an order that builds toward a larger written assignment. Students submit parts of their proposal, receive faculty and peer feedback, make the suggested revisions, and then add the next section for the following assignment. This allows the students to constantly improve their work through faculty and peer feedback. The students also use the proposal as the basis for their final report written in manuscript format. Since the proposal and final manuscript are writing intensive, peer review is required for several assignments. The peer review process is a well-established practice for improving students' academic writing (Baker, 2016). Doctoral students also often have the background and experience that is especially helpful to their peers in developing, implementing and evaluating a project. The students submit assignments a week before each web conferencing session allowing the faculty and student peer reviewer to go over the assignment and come to class prepared to provide verbal feedback in a structured format. In that way, 3
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Fig. 1. Process map for NDNP 811 DNP project development.
practice change, and Tracking the progress over time. The decisions to move toward QI for all DNP projects and to use MAP-IT as a QI process model was made by a DNP project workgroup. The DNP project workgroup was one of four workgroups formed under the guidance of a grant funded Advancing Implementation Science Education (AdvISE) Steering Committee. The other three workgroups formed were: a) Leadership, b) Theory, and c) Measurement. It is beyond the scope of this paper to describe the efforts of the all four workgroups. This paper focuses on the DNP Project workgroup. The DNP project workgroup included the DNP Course Coordinator, the DNP Academic Dean, a DNP specialty track director, a faculty member with QI expertise, and a business analyst process improvement specialist. With this move toward QI, it seemed a natural flow for the DNP project workgroup to use some of the same QI tools to guide their work evaluating the DNP Project courses curriculum that the students were asked to use as a part of their course assignments, such as process maps, to meet the workgroup goals. The goals of the workgroup were to identify opportunities for improvement in the DNP course curricula in order to ensure students demonstrate an adequate level of knowledge and skills needed to effectively translate research into practice by developing, leading, and implementing data-guided QI initiatives. Process mapping, a key tool to facilitate student success and ensure rigor, was one tool used by the workgroup. Process mapping was used to clarify and improve how DNP projects were identified, planned, and approved in each project course because a DNP project with a strong foundation has the greatest chance of success. The process map method helped to identify inconsistencies and lack of clarity about the course structures (curriculum objectives and assignments) and processes (e.g., sequencing of assignments and faculty reviewers) that could cause or had caused confusion and frustration among the faculty, the students, clinical site representatives, and site implementation teams. The
DNP projects are carried out in small groups, each student is responsible for submitting separate and distinctly different course deliverables. Course content and QI process models Early in the program, the DNP projects varied considerably in quality and scope. A key decision made by the faculty to facilitate student success and ensure rigor was to move toward the requirement that all DNP projects would utilize QI methods and tools. The definition of QI that was accepted are “systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of health care in particular settings” (Baily, Bottrell, Lynn, & Jennings, 2006, p. S5). The DNP faculty team decided that three types of QI projects were acceptable as DNP projects, including: a) implementation of an evidence-based practice change, b) health policy projects, and c) program development, implementation and evaluation. While health policy projects are not classically thought of as meeting QI project criteria, UMSON uses a QI framework to allow students to identify policy problems and address them using QI tools. Since the UMSON DNP program does not have a standalone quality improvement course, the QI content required is threaded through core courses and project courses. The integration of QI throughout the DNP project courses is guided by the MAP-IT (Mobilize-Assess-Plan-Implement-Track) QI process model (Center for Community Health and Development, 2019). QI process models such as MAP-IT and the Model for Improvement that includes PDSA (Plan-Do-Study-Act) are used to guide quality improvement efforts (Nilsen, 2015). MAP-IT consists of Mobilizing individuals and the organization to work on a QI project, Assessing the areas of greatest needs as well as resources and strengths of the organization that can be used to implement the practice change, Planning strategies and tactics to target the practice problem, Implementing a 4
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
workgroup developed process maps for DNP project selection prior to entering the first project course, and for each of the four DNP project courses. The process maps helped identify the need to adjust deadlines, assignments, sequencing of assignments, and who would review and approve assignments. The process maps have also been a useful communication tool among faculty and students. When an opportunity for improvement in the course curriculum was identified and a plan was approved by the workgroup, the course faculty would implement the plan as soon as possible after the curriculum committee approved the desired changes. An example of a process map for the second DNP project course is provided in Fig. 1. Since the projects were no longer overseen by a committee and some of the clinical site representatives did not have doctoral degrees, it became clear some other level of project review was needed to enhance the projects' quality and rigor. The decision was made to seek other doctorally prepared second readers for some of the course assignments, including the MAP-IT Project Charter worksheet, IRB submission and final DNP project manuscript. Initially, faculty teaching the project courses served as second readers to students in other sections of the course. By the time of the second cohort, it was clear that the additional workload for course faculty was unsustainable and other doctorally prepared faculty members external to the course were recruited as volunteer second readers. The second readers were offered the opportunity to learn more about DNP quality improvement projects, while earning workload service units. Rubrics specific to the required content and format of each of these assignments had already been developed in the course for the faculty project advisors to use in providing feedback and a numerical grade to the students. The same rubrics were used by the second readers to provide feedback to the students, but in place of numerical grade, they rated the items as being unsatisfactory, satisfactory or excellent. If two or more items were rated as unsatisfactory, the student was expected to make revisions and resubmit the work for a second review. If the work continued to be unsatisfactory, the review then went out to a third reader to work with the student.
Table 3 End of course evaluation of DNP project courses 2017–2018 Mean ± S.D. NDNP NDNP NDNP NDNP
810 811 812 813
4.09 4.37 4.44 4.34
± ± ± ±
0.70 0.57 0.73 0.61
2018–2019 Mean ± S.D. (n (n (n (n
= = = =
79) 52) 54) 41)
3.72 4.33 4.31 4.43
± ± ± ±
0.89 0.65 0.87 0.67
(n (n (n (n
= = = =
71) 76) 80) 68)
In terms of the comments provided with the course evaluations, the primary concern of the students was the students felt there was “too much work” for the first project course, which was not consistent with a one-credit course. In response to this concern, faculty introduced the MAP-IT Project Charter worksheet to replace more writing intensive assignments. The second main concern was the students felt the second reader of their manuscript should have been included earlier in the review process. In response, the second readers were also assigned to review the MAP-IT Project Charter worksheet in the first project course to determine if the project was of sufficient clarity, rigor and quality for the student to move to the second project course. The most common positive comment from students was they appreciated peer reviewer feedback since it gave them a different perspective. Lessons learned The DNP Project is the culmination of a student's doctoral learning experience and reflects not only the DNP Essentials but also the student's abilities as a QI scholar and practice change leader. As the students learn and grow during the DNP Project process, so too does the faculty. The project team has learned several lessons from their work that may benefit others considering similar changes in their courses and DNP project focus. The lessons learned in transitioning DNP project courses to larger cohorts can be grouped into three general categories: a) project course readiness, b) faculty development, and c) faculty and student resources. Regarding the readiness for the project courses, the team put several resources in place based on student and faculty feedback. Key DNP faculty worked with the school's learning technology staff to produce two informational videos, which were distributed in the semester prior to the start of the first project course (NDNP 810). They welcomed students who planned on taking the first project course and provided recommendations on how to identify a practice gap and select a clinical site representative (CSR) for support in planning and implementing the project. Students were encouraged to reach out to their employers and clinical preceptors to identify organizational needs for a quality improvement project. The students were told, “Don't find a solution and look for a problem; find a problem and look for evidence-based solutions.” Faculty also met with QI directors of organizations to identify problems that students and a site implementation team might target with a QI project. Once an organizational problem and a CSR were identified, students were asked to develop a fishbone diagram on the most likely contributing factors to the problem and share it with the CSR to determine what intervention would best target the contributors to the problem. They were instructed to create and share an evidence review table with their CSR and get feedback on which evidence-based strategies would work best for the organization's culture. The need for multiple sections of each DNP Project course created demand for nearly two dozen project faculty advisors. Since many faculty were new to QI concepts, tools, and processes, there was a need for faculty development. Faculty created an online repository of several Quality and Safety Briefs. The QI Briefs are short (around 10 min) video and slide presentations with attachments. They include QI topics such as the differences between research and QI, QI process models, defining the problem, measurement in QI projects (run charts), developing logic models, and the SQUIRE 2.0 guidelines (Ogrinc et al., 2016). There are
Summative evaluation outcomes The DNP project courses for the 2017–2018 and 2018–2019 student cohorts were evaluated by students through a summative end-of-semester Course Evaluation Questionnaire (CEQ) consisting of the following statements:
• Course content flowed logically from course objectives. • Readings were consistent with course content. • Learning activities were consistent with course content. • Graded assessments (e.g., paper, exams) were consistent with course content. • Blackboard navigation was easily executed. • The amount of work for credit earned was about right. • The assigned textbooks and other course materials were valuable to my learning. • I learned relevant information. The statements were scored on a 5-point Likert-type scale ranging from Strongly Disagree to Strongly Agree with a score of five indicating the most positive attitude. Questionnaires were distributed online and students' responses were anonymously collected to eliminate bias. The response rates for the two cohorts ranged between 29 and 100%. The response rate was higher with the first two project courses and decreased for the final two project courses. The mean CEQ scores for seven of the eight project courses were above 4.09 out of 5.00 indicating a positive attitude toward the courses. The only significant difference in scores was with the most recent introductory (NDNP 810) project course in which the mean CEQ decreased from 4.09 to 3.72 (p < .05; see Table 3). 5
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
This article described the transition of the DNP project courses through the experiences of one school implementing quality improvement projects. Key strategies that were provided included a focus on quality improvement projects, the application of process mapping to evaluate DNP project courses, and the use of second readers for selected DNP project course assignments. The success and rigor of the DNP project courses were measured through student satisfaction and their ability to demonstrate the knowledge and skills to effectively translate research into practice by developing, leading, and implementing dataguided QI initiatives. It is anticipated that the lessons learned will be of value to other schools that are undergoing similar challenges.
also QI Briefs on facilitating meetings and elements of an effective poster presentation. In addition to being used to educate faculty on QI topics, the briefs were also incorporated into the online modules for the student project courses. The approach has been to weave QI methods and tools throughout the curriculum rather than have a standalone quality course in the program. The content in the QI Briefs was instrumental to supplement coursework and ensure students received consistent information on how to plan, implement and evaluate a quality improvement project. The QI Briefs continue in development and production as new topics are identified or requested. Other resources were developed to support faculty and students through the IRB process. The faculty enlisted the help of the School's Research QI Manager, who had extensive experience with project design and IRB processes. The manager developed a video, resource guide and module on how to create a CICERO (the school's research evaluation portal) account and submit an application for a quality improvement determination. The manager also worked with the University IRB and local healthcare organizations to coordinate the review process. Arrangements were also made for the manager's review of the applications before submitting to the IRBs. The manager's expertise has helped to streamline and de-mystify the IRB process, saved valuable time and improved DNP project rigor and consistency. The QI Brief that was developed to outline the differences between research and QI included the Ogrinc, Nelson, Adams, and O'Hara (2013) checklist to ensure that students were implementing QI projects and not research. This approach of using the Ogrinc et al. (2013) checklist should be considered by schools who do not have a QI manager to determine if the students' proposed project activities fall within the realm of research or QI. Another resource included providing student group support sessions led by a biostatistician to guide students' plans for data analysis. These sessions were held face-to-face and through web conferencing outside of regular class meetings. Students were discouraged from meeting with the biostatistician outside of these group sessions because of workload concerns for biostatistician, whose primary responsibility was research. However, faculty project advisors consulted with the biostatistician as needed regarding their students' projects. It was anticipated that these sessions would also serve a dual purpose of educating faculty while supporting them in mentoring students regarding statistical issues. This practice of only allowing faculty to meet with the biostatistician has created unforeseen problems. A faculty member would take the information back to the student, and then realize there were more questions to be asked of the biostatistician, and thus more meetings to be scheduled. In response, faculty are now encouraged to bring the students with them to the consultation to encourage a common understanding of issues and mitigate the confusion that results from separate conversations.
Acknowledgements The authors wish to acknowledge the contributions and support of other members of the Advancing Implementation Science Education (AdvISE) Steering Committee members Margaret Hammersla, PhD, CRNP-A, Kathryn Montgomery, PhD, RN, NEA-BC, Linda Costa, PhD, RN, NEA-BC, Patricia Franklin, PhD, RN, Jeffrey Martin, MBA, and Lucy-Rose Davidoff, RN. Part of this work was funded by the Nurse Support Program II, which in turn is funded by the Maryland Health Services Cost Review Commission and administered by the Maryland Higher Education Commission, grant number NSP II 19-125. References American Association of Colleges in Nursing (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from https://www.aacnnursing.org/ Portals/42/Publications/DNPEssentials.pdf. American Association of Colleges in Nursing (2015). The Doctor of Nursing Practice: Current issues and clarifying recommendations: Report from the task force on implementation of the DNP. Retrieved from https://www.pncb.org/sites/default/files/ 2017-02/AACN_DNP_Recommendations.pdf. Baily, M. A., Bottrell, M., Lynn, J., & Jennings (2006). Special report: The ethics of using QI methods to improve healthcare quality and safety. Garrison, New York: The Hastings Center. Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179–192. https://doi.org/10.1177/ 1469787416654794. Brown, M. A., & Crabtree, K. (2013). The development of practice scholarship in DNP programs: A paradigm shift. Journal of Professional Nursing, 9(6), 330–337. https:// doi.org/10.1016/j.profnurs.2013.08.003. Center for Community Health and Development (2019). Chapter 2, section 14: MAP-IT: A model for implementing Healthy People 2020. Retrieved from http://ctb.ku.edu/en/ table-of-contents/overview/models-for-community-health-and-development/map-it/ main. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(50), https://doi.org/10.1186/1748-5908-4-50. Retrieved from https:// implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50. Nilsen, P. (2015). Making sense of implementation theories, frameworks, and models. Implementation Science, 10(53), 1–13. https://doi.org/10.1186/s13012-015-0242. Retrieved from https://implementationscience.biomedcentral.com/articles/10. 1186/s13012-015-0242-0. Ogrinc, G., Davies, L., Goodman, D., Batalden, P., & Davidoff, F. (2016). SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) revised publication guidelines from a detailed consensus process. Journal of Nursing Care Quality, 31(1), 1–8. https://doi.org/10.7812/TPP/15-141. Retrieved from https://journals.lww. com/jncqjournal/Fulltext/2016/01000/SQUIRE_2_0__Standards_for_QUality_ Improvement.1.aspx. Ogrinc, G., Nelson, W. A., Adams, S. M., & O'Hara, A. E. (2013). An instrument to differentiate between clinical research and quality improvement. IRB: Ethics & Human Research, 35(5), 1–8.
Conclusion As DNP programs are expanding and evolving across the country, faculty are struggling to determine the best format for project courses. The challenges are even greater as the number of students in these programs increases. Whereas many programs may have followed the dissertation process with chairpersons and committees to oversee the project, that process requires significant faculty resources, and was found to be unfeasible with larger cohorts of students.
6
Contents lists available at ScienceDirect
Journal of Professional Nursing journal homepage: www.elsevier.com/locate/jpnu
Structuring Doctor of Nursing Practice project courses to facilitate success and ensure rigor ⁎
Kathleen M. Buckley , Shannon Idzik, Debra Bingham, Brenda Windemuth, Susan L. Bindon University of Maryland School of Nursing, 655 West Lombard Street, Baltimore, MD 21201, United States of America
A R T I C LE I N FO
A B S T R A C T
Keywords: Graduate nursing education Doctoral programs Quality improvement Doctoral students Doctor of Nursing Practice Doctor of Nursing Practice Project
There is currently a lack of consensus on the best format for Doctor of Nursing Practice project deliverables. In this article the project course history, current format, and evaluation methods are described for a Doctor of Nursing Practice program during the transition from a sole post-master's option to one that also admitted postbaccalaureate students. The project course format shifted focus from one in which students independently implemented multiple types of projects under the direction of a chairperson and committee to one in which students carried out projects utilizing quality improvement methods and tools under the mentorship of a project faculty advisor and clinical site representative. The integration of quality improvement models is exemplified through course objectives and assignments. Lessons learned through this transition are provided in the hope that the work may benefit other nursing schools with similar programs.
Introduction A final Doctor of Nursing Practice (DNP) project was noted to be a core requirement of the DNP degree in the Essentials for Doctoral Education for Advanced Nursing Practice (American Association of Colleges in Nursing [AACN], 2006). Examples of what these projects might entail included a variety of formats, such as practice portfolios, evidence based practice change initiatives, integrated literature reviews and quality improvement projects. Schools were allowed to define these projects in a form that best integrated the specialty requirements. As DNP programs evolved, there appeared to be a growing lack of consensus in the appropriate format, requirements, goals, evaluation and dissemination of DNP projects (Brown & Crabtree, 2013). AACN responded to the call for more clarity by establishing a task force to address some of these issues. The report that was generated stated that the requirements for the DNP project should be the same for post-baccalaureate and post-master’s students. The report further stated that all projects should include planning, implementation, and evaluation components; focus on a change that impacts healthcare outcomes either through direct or indirect care; include a plan for sustainability and an evaluation of processes and/or outcomes; and provide a foundation for future practice scholarship (AACN, 2015). However, there is currently some disagreement on the best format for project courses and deliverables, especially as schools admit larger cohorts of students. The purpose of this paper is to describe a nursing
⁎
school's process in developing and modifying DNP project courses for a large cohort of students with the expectation that the structure and processes described may serve as models for other faculty who are directing DNP project courses and working to expand the ability of faculty and students to more effectively translate evidence based practice changes. History of DNP project course changes at UMSON In 2006, the University of Maryland School of Nursing (UMSON) opened a post-master's DNP program. The school enrolled approximately 20–35 students each year in the program and accepted students from all areas of advanced nursing practice including advanced practice nurses, nurse informaticians, public health nurses and nurse leaders. The program included four one-credit pass/fail DNP project courses (see Table 1). Students progressed through the first two courses in a face-to-face seminar format and completed the last two courses independently under the supervision of a chairperson and at least two committee members. The four courses were scheduled to occur in the final four semesters of the students' program. The first project course focused heavily on the history of and movement to the DNP degree. Much time was spent discussing the Essentials for Doctoral Education for Advanced Nursing Practice (AACN, 2006), and understanding roles for which nurses were prepared with a doctoral degree in nursing practice. During this course, students also identified a practice problem
Corresponding author. E-mail address: [email protected] (K.M. Buckley).
https://doi.org/10.1016/j.profnurs.2019.12.001 Received 1 July 2019; Received in revised form 23 November 2019; Accepted 5 December 2019 8755-7223/ © 2019 Elsevier Inc. All rights reserved.
Please cite this article as: Kathleen M. Buckley, et al., Journal of Professional Nursing, https://doi.org/10.1016/j.profnurs.2019.12.001
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Table 1 DNP project course changes from 2006 to 2017. 2006 NDNP NDNP NDNP NDNP
810 811 812 813
Capstone Capstone Capstone Capstone
1: 2: 3: 4:
Professional Development Project Identification Project Implementation Project Evaluation and Dissemination
2014
2017
Deleted course NDNP 811 Scholarly Project Development NDNP 812 Scholarly Project Implementation NDNP 813 Scholarly Project Evaluation and Dissemination
NDNP 810 DNP Project Identification NDNP 811 DNP Project Development NDNP 812 DNP Project Implementation NDNP 813 DNP Project Evaluation and Dissemination
worksite. Their previous relationship with the CSRs helped to encourage buy-in to supporting the student in implementing a QI project. It was preferable, but not essential, that the CSR be doctorally prepared (e.g., DNP, Ph.D., MD). The responsibilities of the CSR were to:
of interest and began a literature search for evidence-based solutions. The second course required students to develop a theory-based proposal for their DNP project. They worked with course faculty to select an evidence-based intervention, complete a full Institutional Review Board (IRB) submission and prepare a “proposal defense” in front of their chairperson and committee members. In the third course, students worked independently with their chairperson and committee members to implement their project. In the fourth and final course, the student completed the project evaluation with the guidance of the committee and finalized a manuscript. A publication-quality manuscript became the final deliverable along with a public “project defense”. As the DNP program evolved, faculty leaders made modifications to the project courses. They felt that a reduction in credits would help with the overall course schedule and that the course/curriculum was too heavily focusing on the rationale for the DNP, arguments for and against the DNP, and roles that DNP prepared graduates would assume upon graduation. In 2014, the first project course was deleted from the course sequence and the term “capstone project” changed to scholarly project in the course titles (see Table 1). A gap between the first and second courses was created in the summer to allow the post-masters' DNP students additional time to complete their IRB review, and to decrease burden on faculty and practice sites during the summer. Unfortunately, the deletion of the first project course in 2014 meant that there were only three DNP project courses. After the first project course was deleted, faculty noted that the students struggled to identify a problem, solution and plan for action in one semester. Faculty and students communicated that there was a mismatch between the amount of work associated with the first project course for the one credit earned. Work began to be shifted to a “pre-class” requirement which required faculty and students to do work outside of the course structure. Practice sites grew weary when students stalled their projects over the summer and occasionally implemented the projects without the students. Students implementing and evaluating the projects independently under the direction of chairpersons also created a perception of poor quality among practice partners due to variation in expectations. In response to the challenges and in preparation for a large cohort of students in the project course sequence due to the start of the postbaccalaureate DNP program in 2014, the DNP project course structure was revised in 2017. The first course in the sequence returned with a stronger focus on proposal development (see Table 1). In the second course students completed the development of their proposal, presented it to stakeholders in the clinical site for approval, and submitted an application to the appropriate IRBs. The term “scholarly project” was replaced by “DNP project” in the course titles. Two cohorts of students have been through the series of DNP project courses. In order to manage the larger cohorts of students (85–130 students) with the limited number of faculty to serve on project committees, all four project courses met as classes rather than as independent student-chairperson dyads supported by a committee. There were 16–23 sections of each project course. A section typically consisted of six students under the direction of one faculty project advisor. The student and faculty project advisor worked with each student's clinical site representative (CSR) to plan, implement and evaluate the project. Students took an active part in the process of finding potential CSRs for projects by seeking out previous preceptors or managers at their
• meet with the DNP student at least monthly to develop a proposal for a QI project in NDNP 810 • listen to a formal presentation of the DNP student's proposal, pro• •
vide written approval and/or recommendations for change, and facilitate the IRB process at the practice site in NDNP 811 guide and support the DNP student in assembling an implementation team, whom the student will lead in implementing practice changes in structures, processes and outcomes in NDNP 812 support the dissemination of the DNP project findings within the organization in NDNP 813
Students were also responsible for getting verbal approval from a sponsor for their project before beginning the first project course. A sponsor was identified as an administrator with the power and leadership skill within the organization to secure resources or remove barriers that would affect the DNP project's implementation effectiveness. The sponsor was also the individual identified by the organization with the authority to approve or close down the project. The classes for each project course were held during weeks 1, 5, 9, 12 and 14 over a 14 week semester via web conferencing, and were 2 h in length. The sections were overseen by one faculty course coordinator, and class announcements occurred through one online forum to maintain consistency among the sections. This significant change necessitated a critical look at course content as the course structure became the driver of project content and quality. The following section describes the current UMSON course structure and content. DNP project course structure and content The project courses currently run over four consecutive semesters or 17 months—beginning in January and ending in May of the following year—with the same faculty project advisors per group for all four courses (see Table 1). This sequence applies to all DNP students. If they miss taking the first project course in the Spring, they have to wait until the following year to begin. In the first course (January–May), the students identify a significant problem at a practice site where there was a gap between the research evidence and the current practice. They then review the evidence to support a best practice that could be implemented through a quality improvement (QI) project to address the identified practice problem. During the second course (June–July), the students finalize their implementation plan and present it to stakeholders at the project site for approval. They also apply for any necessary stakeholder or IRB determination that their project is indeed quality improvement. In the third course (August–December), they operationalize the implementation plan and collect data to evaluate the effectiveness of the implementation effort. In the last course (January–May), they analyze their data, write up their findings and evaluation of their project in a manuscript format, and disseminate their results and recommendations for next steps with stakeholders and in a public forum through a formal poster or podium presentation. Course objectives and relevant assignments to meet those objectives 2
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Table 2 DNP project course objectives and assignments. Course and semester Pre-requisites for NDNP 810 (Fall semester prior to taking NDNP 810)
NDNP 810 DNP Project Identification (Spring semester)
NDNP 811 DNP Project Development (Summer semester)
NDNP 812 DNP Project Implementation (Fall semester)
NDNP 813 DNP Project Evaluation and Dissemination (Spring semester)
Objectives
Assignments
a practice problem that can be formally addressed in a • Identify DNP project appropriate stakeholders at the practice site necessary to • Identify carry out the project
of a problem at a clinical or community site in • Identification need of a quality improvement solution of a clinical site representative and sponsor • Identification (Ishikawa or cause and effect) diagram to categorize the • Fishbone causes and sub-causes of a QI problem within a specific context in
the current literature related to the practice problem • Analyze external evidence to support and confirm the significance • ofCollect the practice problem an appropriate theoretical framework based on • Identify implementation science to guide the DNP project. quality improvement process tools to illustrate the proposed • Use DNP project
Model to identify the resources, activities, outputs, • Logic outcomes, and potential impact of a project Project Proposal in manuscript format: background, • DNP theoretical framework, literature review, and evidence review
stakeholders and partners within the clinical setting • Engage necessary to carry out the project. an appropriate implementation plan for proposed DNP • Develop project a succinct DNP project proposal presentation to faculty • Deliver and clinical partner for approval IRB submission and obtain other necessary approvals • Complete an evidence-based DNP project based on problem • Implement identification, conceptualization of the problem in the context of current practice, evidence and theory, and development of methods for addressing the problem
Analyze and display the findings from the completed DNP project • and make recommendations for sustainability and spread of the
order to identify the root causes of the problem
table
portion of MAP-IT Project Charter Worksheet reviewed by • MAP second reader and including:
• • • • • • • • •
practice changes
changes in practice made, results of the data analysis • Describe and outcomes, and facilitators and barriers encountered project results including a comparison of results with • Discuss findings from other publications, as well as strengths and limitations of project.
Disseminate the DNP project's findings and lessons learned • through a written manuscript, following SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) guidelines, and a formal poster or oral presentation (Ogrinc, Davies, Goodman, Batalden, & Davidoff, 2016)
•
a. Updated fishbone diagram b. Goals c. Process model d. Structure, Process and Outcome Measures e. Run chart format DNP Project Proposal: plan for implementation and abstract added to previous parts of proposal in manuscript format Data collection tools IRB submission for determination as quality improvement with prior review by compliance coordinator (no exempt, expedited or full IRB review) DNP Project Presentation to stakeholders at practice site Implementation of an evidence-based intervention over 14 weeks Updated MAP-IT with IT portion added Force field analysis based on the Consolidated Framework of Implementation Research or a framework to identify the barriers and facilitators of the practice change being implemented (Damschroder et al., 2009) Abstracts submitted to external local, regional and national conferences held in following Spring semester Dissemination requirements a. Present back to stakeholders and at the UMSON DNP Poster Day plus b. One of 2 options i. Local, regional or national conference regional poster or oral presentation ii. Formal presentation within organization (e.g., nursing grand rounds) DNP project proposal converted to final report following SQUIRE Guidelines in manuscript format with results, discussion, conclusion and abstract added
the entire group of students benefit from the feedback. The students peer review assignments in all four project courses. The assignments include their DNP Project Logic Model, evidence review table, project proposal, MAP-IT Project Charter Worksheet (which includes multiple components, such as their Fishbone diagram, process map, measurement and analysis plan), proposal presentation, poster presentation, and the final manuscript. Individual written feedback is provided by the faculty and peer reviewer immediately after the class. While most of the projects are led by individual students, there have been a small number of group projects within each cohort. Based on our previous experience of working with DNP students in group projects, it was decided that the maximum size for a group would be three student members. The group projects consist of smaller separate distinct projects that are parts of a larger initiative or implementation of a similar practice change across different settings. For example, three students worked together in a preoperative center preparing patients for surgery. One of the three students implemented a clinical practice guideline for using chlorhexidine cloths preoperatively. Another student implemented electronic prescribing of preoperative medications, and the third student implemented the screening of patients preoperatively for methicillin-resistant Staphylococcus aureus colonization. Although some
have been developed for each course (see Table 2). Assignments are iterative in that students have an opportunity to get feedback and submit revisions that go through additional reviews. The project proposal is also scaffolded in that it is broken down into smaller parts (i.e., overview and statement of the problem, evidence review, theoretical framework and implementation plan) and then sequenced in an order that builds toward a larger written assignment. Students submit parts of their proposal, receive faculty and peer feedback, make the suggested revisions, and then add the next section for the following assignment. This allows the students to constantly improve their work through faculty and peer feedback. The students also use the proposal as the basis for their final report written in manuscript format. Since the proposal and final manuscript are writing intensive, peer review is required for several assignments. The peer review process is a well-established practice for improving students' academic writing (Baker, 2016). Doctoral students also often have the background and experience that is especially helpful to their peers in developing, implementing and evaluating a project. The students submit assignments a week before each web conferencing session allowing the faculty and student peer reviewer to go over the assignment and come to class prepared to provide verbal feedback in a structured format. In that way, 3
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
Fig. 1. Process map for NDNP 811 DNP project development.
practice change, and Tracking the progress over time. The decisions to move toward QI for all DNP projects and to use MAP-IT as a QI process model was made by a DNP project workgroup. The DNP project workgroup was one of four workgroups formed under the guidance of a grant funded Advancing Implementation Science Education (AdvISE) Steering Committee. The other three workgroups formed were: a) Leadership, b) Theory, and c) Measurement. It is beyond the scope of this paper to describe the efforts of the all four workgroups. This paper focuses on the DNP Project workgroup. The DNP project workgroup included the DNP Course Coordinator, the DNP Academic Dean, a DNP specialty track director, a faculty member with QI expertise, and a business analyst process improvement specialist. With this move toward QI, it seemed a natural flow for the DNP project workgroup to use some of the same QI tools to guide their work evaluating the DNP Project courses curriculum that the students were asked to use as a part of their course assignments, such as process maps, to meet the workgroup goals. The goals of the workgroup were to identify opportunities for improvement in the DNP course curricula in order to ensure students demonstrate an adequate level of knowledge and skills needed to effectively translate research into practice by developing, leading, and implementing data-guided QI initiatives. Process mapping, a key tool to facilitate student success and ensure rigor, was one tool used by the workgroup. Process mapping was used to clarify and improve how DNP projects were identified, planned, and approved in each project course because a DNP project with a strong foundation has the greatest chance of success. The process map method helped to identify inconsistencies and lack of clarity about the course structures (curriculum objectives and assignments) and processes (e.g., sequencing of assignments and faculty reviewers) that could cause or had caused confusion and frustration among the faculty, the students, clinical site representatives, and site implementation teams. The
DNP projects are carried out in small groups, each student is responsible for submitting separate and distinctly different course deliverables. Course content and QI process models Early in the program, the DNP projects varied considerably in quality and scope. A key decision made by the faculty to facilitate student success and ensure rigor was to move toward the requirement that all DNP projects would utilize QI methods and tools. The definition of QI that was accepted are “systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of health care in particular settings” (Baily, Bottrell, Lynn, & Jennings, 2006, p. S5). The DNP faculty team decided that three types of QI projects were acceptable as DNP projects, including: a) implementation of an evidence-based practice change, b) health policy projects, and c) program development, implementation and evaluation. While health policy projects are not classically thought of as meeting QI project criteria, UMSON uses a QI framework to allow students to identify policy problems and address them using QI tools. Since the UMSON DNP program does not have a standalone quality improvement course, the QI content required is threaded through core courses and project courses. The integration of QI throughout the DNP project courses is guided by the MAP-IT (Mobilize-Assess-Plan-Implement-Track) QI process model (Center for Community Health and Development, 2019). QI process models such as MAP-IT and the Model for Improvement that includes PDSA (Plan-Do-Study-Act) are used to guide quality improvement efforts (Nilsen, 2015). MAP-IT consists of Mobilizing individuals and the organization to work on a QI project, Assessing the areas of greatest needs as well as resources and strengths of the organization that can be used to implement the practice change, Planning strategies and tactics to target the practice problem, Implementing a 4
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
workgroup developed process maps for DNP project selection prior to entering the first project course, and for each of the four DNP project courses. The process maps helped identify the need to adjust deadlines, assignments, sequencing of assignments, and who would review and approve assignments. The process maps have also been a useful communication tool among faculty and students. When an opportunity for improvement in the course curriculum was identified and a plan was approved by the workgroup, the course faculty would implement the plan as soon as possible after the curriculum committee approved the desired changes. An example of a process map for the second DNP project course is provided in Fig. 1. Since the projects were no longer overseen by a committee and some of the clinical site representatives did not have doctoral degrees, it became clear some other level of project review was needed to enhance the projects' quality and rigor. The decision was made to seek other doctorally prepared second readers for some of the course assignments, including the MAP-IT Project Charter worksheet, IRB submission and final DNP project manuscript. Initially, faculty teaching the project courses served as second readers to students in other sections of the course. By the time of the second cohort, it was clear that the additional workload for course faculty was unsustainable and other doctorally prepared faculty members external to the course were recruited as volunteer second readers. The second readers were offered the opportunity to learn more about DNP quality improvement projects, while earning workload service units. Rubrics specific to the required content and format of each of these assignments had already been developed in the course for the faculty project advisors to use in providing feedback and a numerical grade to the students. The same rubrics were used by the second readers to provide feedback to the students, but in place of numerical grade, they rated the items as being unsatisfactory, satisfactory or excellent. If two or more items were rated as unsatisfactory, the student was expected to make revisions and resubmit the work for a second review. If the work continued to be unsatisfactory, the review then went out to a third reader to work with the student.
Table 3 End of course evaluation of DNP project courses 2017–2018 Mean ± S.D. NDNP NDNP NDNP NDNP
810 811 812 813
4.09 4.37 4.44 4.34
± ± ± ±
0.70 0.57 0.73 0.61
2018–2019 Mean ± S.D. (n (n (n (n
= = = =
79) 52) 54) 41)
3.72 4.33 4.31 4.43
± ± ± ±
0.89 0.65 0.87 0.67
(n (n (n (n
= = = =
71) 76) 80) 68)
In terms of the comments provided with the course evaluations, the primary concern of the students was the students felt there was “too much work” for the first project course, which was not consistent with a one-credit course. In response to this concern, faculty introduced the MAP-IT Project Charter worksheet to replace more writing intensive assignments. The second main concern was the students felt the second reader of their manuscript should have been included earlier in the review process. In response, the second readers were also assigned to review the MAP-IT Project Charter worksheet in the first project course to determine if the project was of sufficient clarity, rigor and quality for the student to move to the second project course. The most common positive comment from students was they appreciated peer reviewer feedback since it gave them a different perspective. Lessons learned The DNP Project is the culmination of a student's doctoral learning experience and reflects not only the DNP Essentials but also the student's abilities as a QI scholar and practice change leader. As the students learn and grow during the DNP Project process, so too does the faculty. The project team has learned several lessons from their work that may benefit others considering similar changes in their courses and DNP project focus. The lessons learned in transitioning DNP project courses to larger cohorts can be grouped into three general categories: a) project course readiness, b) faculty development, and c) faculty and student resources. Regarding the readiness for the project courses, the team put several resources in place based on student and faculty feedback. Key DNP faculty worked with the school's learning technology staff to produce two informational videos, which were distributed in the semester prior to the start of the first project course (NDNP 810). They welcomed students who planned on taking the first project course and provided recommendations on how to identify a practice gap and select a clinical site representative (CSR) for support in planning and implementing the project. Students were encouraged to reach out to their employers and clinical preceptors to identify organizational needs for a quality improvement project. The students were told, “Don't find a solution and look for a problem; find a problem and look for evidence-based solutions.” Faculty also met with QI directors of organizations to identify problems that students and a site implementation team might target with a QI project. Once an organizational problem and a CSR were identified, students were asked to develop a fishbone diagram on the most likely contributing factors to the problem and share it with the CSR to determine what intervention would best target the contributors to the problem. They were instructed to create and share an evidence review table with their CSR and get feedback on which evidence-based strategies would work best for the organization's culture. The need for multiple sections of each DNP Project course created demand for nearly two dozen project faculty advisors. Since many faculty were new to QI concepts, tools, and processes, there was a need for faculty development. Faculty created an online repository of several Quality and Safety Briefs. The QI Briefs are short (around 10 min) video and slide presentations with attachments. They include QI topics such as the differences between research and QI, QI process models, defining the problem, measurement in QI projects (run charts), developing logic models, and the SQUIRE 2.0 guidelines (Ogrinc et al., 2016). There are
Summative evaluation outcomes The DNP project courses for the 2017–2018 and 2018–2019 student cohorts were evaluated by students through a summative end-of-semester Course Evaluation Questionnaire (CEQ) consisting of the following statements:
• Course content flowed logically from course objectives. • Readings were consistent with course content. • Learning activities were consistent with course content. • Graded assessments (e.g., paper, exams) were consistent with course content. • Blackboard navigation was easily executed. • The amount of work for credit earned was about right. • The assigned textbooks and other course materials were valuable to my learning. • I learned relevant information. The statements were scored on a 5-point Likert-type scale ranging from Strongly Disagree to Strongly Agree with a score of five indicating the most positive attitude. Questionnaires were distributed online and students' responses were anonymously collected to eliminate bias. The response rates for the two cohorts ranged between 29 and 100%. The response rate was higher with the first two project courses and decreased for the final two project courses. The mean CEQ scores for seven of the eight project courses were above 4.09 out of 5.00 indicating a positive attitude toward the courses. The only significant difference in scores was with the most recent introductory (NDNP 810) project course in which the mean CEQ decreased from 4.09 to 3.72 (p < .05; see Table 3). 5
Journal of Professional Nursing xxx (xxxx) xxx–xxx
K.M. Buckley, et al.
This article described the transition of the DNP project courses through the experiences of one school implementing quality improvement projects. Key strategies that were provided included a focus on quality improvement projects, the application of process mapping to evaluate DNP project courses, and the use of second readers for selected DNP project course assignments. The success and rigor of the DNP project courses were measured through student satisfaction and their ability to demonstrate the knowledge and skills to effectively translate research into practice by developing, leading, and implementing dataguided QI initiatives. It is anticipated that the lessons learned will be of value to other schools that are undergoing similar challenges.
also QI Briefs on facilitating meetings and elements of an effective poster presentation. In addition to being used to educate faculty on QI topics, the briefs were also incorporated into the online modules for the student project courses. The approach has been to weave QI methods and tools throughout the curriculum rather than have a standalone quality course in the program. The content in the QI Briefs was instrumental to supplement coursework and ensure students received consistent information on how to plan, implement and evaluate a quality improvement project. The QI Briefs continue in development and production as new topics are identified or requested. Other resources were developed to support faculty and students through the IRB process. The faculty enlisted the help of the School's Research QI Manager, who had extensive experience with project design and IRB processes. The manager developed a video, resource guide and module on how to create a CICERO (the school's research evaluation portal) account and submit an application for a quality improvement determination. The manager also worked with the University IRB and local healthcare organizations to coordinate the review process. Arrangements were also made for the manager's review of the applications before submitting to the IRBs. The manager's expertise has helped to streamline and de-mystify the IRB process, saved valuable time and improved DNP project rigor and consistency. The QI Brief that was developed to outline the differences between research and QI included the Ogrinc, Nelson, Adams, and O'Hara (2013) checklist to ensure that students were implementing QI projects and not research. This approach of using the Ogrinc et al. (2013) checklist should be considered by schools who do not have a QI manager to determine if the students' proposed project activities fall within the realm of research or QI. Another resource included providing student group support sessions led by a biostatistician to guide students' plans for data analysis. These sessions were held face-to-face and through web conferencing outside of regular class meetings. Students were discouraged from meeting with the biostatistician outside of these group sessions because of workload concerns for biostatistician, whose primary responsibility was research. However, faculty project advisors consulted with the biostatistician as needed regarding their students' projects. It was anticipated that these sessions would also serve a dual purpose of educating faculty while supporting them in mentoring students regarding statistical issues. This practice of only allowing faculty to meet with the biostatistician has created unforeseen problems. A faculty member would take the information back to the student, and then realize there were more questions to be asked of the biostatistician, and thus more meetings to be scheduled. In response, faculty are now encouraged to bring the students with them to the consultation to encourage a common understanding of issues and mitigate the confusion that results from separate conversations.
Acknowledgements The authors wish to acknowledge the contributions and support of other members of the Advancing Implementation Science Education (AdvISE) Steering Committee members Margaret Hammersla, PhD, CRNP-A, Kathryn Montgomery, PhD, RN, NEA-BC, Linda Costa, PhD, RN, NEA-BC, Patricia Franklin, PhD, RN, Jeffrey Martin, MBA, and Lucy-Rose Davidoff, RN. Part of this work was funded by the Nurse Support Program II, which in turn is funded by the Maryland Health Services Cost Review Commission and administered by the Maryland Higher Education Commission, grant number NSP II 19-125. References American Association of Colleges in Nursing (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from https://www.aacnnursing.org/ Portals/42/Publications/DNPEssentials.pdf. American Association of Colleges in Nursing (2015). The Doctor of Nursing Practice: Current issues and clarifying recommendations: Report from the task force on implementation of the DNP. Retrieved from https://www.pncb.org/sites/default/files/ 2017-02/AACN_DNP_Recommendations.pdf. Baily, M. A., Bottrell, M., Lynn, J., & Jennings (2006). Special report: The ethics of using QI methods to improve healthcare quality and safety. Garrison, New York: The Hastings Center. Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179–192. https://doi.org/10.1177/ 1469787416654794. Brown, M. A., & Crabtree, K. (2013). The development of practice scholarship in DNP programs: A paradigm shift. Journal of Professional Nursing, 9(6), 330–337. https:// doi.org/10.1016/j.profnurs.2013.08.003. Center for Community Health and Development (2019). Chapter 2, section 14: MAP-IT: A model for implementing Healthy People 2020. Retrieved from http://ctb.ku.edu/en/ table-of-contents/overview/models-for-community-health-and-development/map-it/ main. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(50), https://doi.org/10.1186/1748-5908-4-50. Retrieved from https:// implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50. Nilsen, P. (2015). Making sense of implementation theories, frameworks, and models. Implementation Science, 10(53), 1–13. https://doi.org/10.1186/s13012-015-0242. Retrieved from https://implementationscience.biomedcentral.com/articles/10. 1186/s13012-015-0242-0. Ogrinc, G., Davies, L., Goodman, D., Batalden, P., & Davidoff, F. (2016). SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) revised publication guidelines from a detailed consensus process. Journal of Nursing Care Quality, 31(1), 1–8. https://doi.org/10.7812/TPP/15-141. Retrieved from https://journals.lww. com/jncqjournal/Fulltext/2016/01000/SQUIRE_2_0__Standards_for_QUality_ Improvement.1.aspx. Ogrinc, G., Nelson, W. A., Adams, S. M., & O'Hara, A. E. (2013). An instrument to differentiate between clinical research and quality improvement. IRB: Ethics & Human Research, 35(5), 1–8.
Conclusion As DNP programs are expanding and evolving across the country, faculty are struggling to determine the best format for project courses. The challenges are even greater as the number of students in these programs increases. Whereas many programs may have followed the dissertation process with chairpersons and committees to oversee the project, that process requires significant faculty resources, and was found to be unfeasible with larger cohorts of students.
6