- Email: [email protected]
Veterans Affairs Interprofessional Nurse Practitioner Residency in Primary Care: A Competency-based Program
Veterans Affairs Interprofessional Nurse Practitioner Residency in Primary Care: A Competency-based Program Kathryn Wirtz Rugen, PhD, FNP-BC, Elena Speroff, DNP, NP-C, Susan A. Zapatka, MSN, ANP-BC, and Rebecca Brienza, MD, MPH ABSTRACT
The Institute of Medicine recommended the implementation of nurse practitioner transition-to-practice programs, either called residency or fellowship, for new graduates. These programs are rapidly expanding on the national level in a variety of practice areas. However, there is a lack of literature on the effectiveness of these programs. The Veterans Affairs Centers of Excellence in Primary Care Education developed a competency-based assessment tool to measure program effectiveness, document the achievement of competency, and promote standardization. This article describes the development of the tool along with curricular examples to promote nurse practitioner transition to practice. Keywords: competency assessment, nurse practitioner postgraduate training Published by Elsevier Inc.
N
ew nurse practitioner (NP) graduates face many challenges as they transition from academia to the complex health care environment, at both the patient and system levels. Despite an excellent academic preparation, the transition to practice can be difficult.1-5 The literature has shown that new NPs described their first year of practice as distressing and tumultuous, with a selfperception that they lacked legitimacy in the NP role.6 The authors concluded that it is imperative to reduce the vulnerability of new NPs.6 Another study revealed that 87% of new NP graduates reported they would be interested in a postgraduate training program if it were available to them.3 Furthermore, the Institute of Medicine recommended advanced practice nurses complete a transition-to-practice program (nurse residency program).7 NP postgraduate training programs are optional and can provide new NPs with “additional management strategies for working with patients, families, or communities where significant financial, social, and emotional co-morbidities make traditional management of health more challenging, such as in www.npjournal.org
community health centers and in the Veterans Affairs (VA) health system.”8 NP postgraduate training programs are in demand as shown by the number of NP trainees who have applied to programs.4,5 Currently, there are NP postgraduate training opportunities in acute care, primary care, psychiatry/ mental health, and specialty areas throughout the United States. These programs offer variable amounts of didactic training, specialty rotations, structured mentorship, and interprofessional collaboration.5,9,10 However, there is a paucity of literature on the effectiveness of these programs. The purpose of this article is to describe the process of the development of a competency-based tool to measure the effectiveness of the VA Center of Excellence in Primary Care Education (CoEPCE) NP residency program. VA CENTERS OF EXCELLENCE IN PRIMARY CARE EDUCATION
In 2011, the VA Office of Academic Affiliations funded the development of 5 VA CoEPCEs. These centers were charged with developing and testing The Journal for Nurse Practitioners - JNP
e267
innovative models of interprofessional education and collaborative practice in the VA primary care setting. In the first year, the West Haven CoEPCE developed a NP residency embedded within the interprofessional learning environment. This NP residency program was developed for new graduates of adult gerontology primary care or family NP programs to learn to work in and lead interprofessional, patient-centered care teams. Interprofessional learning and collaborative practice occurs with physician residents, postdoctorate pharmacy residents, and postdoctorate psychology fellows. Based on the successes of the West Haven NP residency program in recruitment, retention, trainee competency progression, faculty and resident satisfaction, and job placement, the remaining 4 CoEPCEs (Boise, Cleveland, San Francisco, and Seattle) initiated NP postgraduate training programs in subsequent years.4,5 All CoEPCE NP residency training programs are 12 months long and are supported with a trainee stipend that is approximately half of a full-time NP staff position and comparable with the stipend of a first-year physician resident. The stipend includes benefits for health care, vacation, and sick leave. Candidates must have graduated from an accredited master’s or doctor of nursing practice (DNP) program within the past year, have board certification as an adult-gerontology primary care or family NP and state advanced practice registered nurse licensure, have outstanding references from faculty, and have completed a rigorous interviewing process. All the CoEPCEs have been successful in recruiting NP students who have had clinical practicums in CoEPCE to continue in their NP residency program. Across all of the CoEPCE sites, there have been consistently more applicants than positions available. The CoEPCE NP residency curriculum focuses on the advancement of clinical and diagnostic skills as well as leadership and scholarship skills through interprofessional experiential learning opportunities and collaborative care. The NP residents are assigned a panel of patients that they share with their faculty supervisors and at some sites also share or cross cover patients with the physician residents in practice partnership models. They work in a patient-centered medical home model with a teamlet composed of a e268
The Journal for Nurse Practitioners - JNP
registered nurse case manager, licensed practical nurse/licensed vocational nurse, health care technician, and clerical staff. They are also assigned a faculty mentor(s) who could be an NP or physician. At some sites, they are supervised by MD NP dyad faculty mentors. Optional specialty care rotations in areas such as cardiology, dermatology, pulmonology, palliative care, and gastroenterology are available. Several of the CoEPCEs offer an inpatient rotation. NP residents are mentored to lead shared medical appointments and group visits, case conferences, and team huddles. Scholarly pursuits, such as presenting and publishing, are encouraged and mentored. Trainees of all professions learn panel management, performance improvement, and population health skills. Trainees work collaboratively on performance/ quality improvement projects. In the second half of the program, the NP residents participate in precepting NP students and trainees of other professions with supervised mentorship. Development of the NP Residency Competency Tool
In 2012, with the expansion of NP residency programs to all the CoEPCE sites, it became imperative to develop a standardized evaluation process. This was especially important because this was a pilot program incorporated into a high-profile interprofessional education demonstration project. The goal was to develop a tool that would document and show program effectiveness and achievement of competencies essential for interprofessional teambased patient-centered primary care practice. These competencies include the ability to assess, diagnose, treat, and manage common acute and chronic health conditions; patient-centered care; leadership skills; and performance improvement/ population health skills. These competencies are not considered remediation of knowledge and skills already acquired during graduate education but advancement of skills to support the transition to a fully competent and confident provider. The development of the competency tool was an iterative team process led by the NP consultant from the CoEPCE Coordinating Center in collaboration with the NP codirectors from each CoEPCE and the CoEPCE Coordinating Center physician consultant who has expertise in health professions education evaluation. Volume 12, Issue 6, June 2016
Several standards and competencies for NP education were reviewed to ensure the NP residency program would build on the academic curriculum and competencies achieved upon graduation from either a master’s or DNP nurse practitioner program. These included the American Association of Colleges of Nursing competencies for an adult-gerontology primary care NP and the National Organization of Nurse Practitioner Faculties (NONPF) NP core competencies.11,12 It is essential for all NPs, including adult-gerontology primary care NPs, to develop aptitude in the 9 NONPF competency domains before graduation. The 9 domains include scientific foundation, leadership, quality, practice, inquiry, technology and information literacy, policy, health delivery system, ethics, and independent practice.12 The NONPF core competencies are reinforced by the 8 American Association of Colleges of Nursing DNP essential competencies, which include scientific underpinnings for practice, organizational and systems leadership for quality improvement and systems thinking, clinical scholarship and analytical methods for evidence-based practice, information systems and patient care technology, health care policy, and interprofessional collaboration.11,13 Because the NP residency program takes place in the recently redesigned patient-centered primary care setting called patient-aligned care teams (PACT), VA’s model of the patient-centered medical home, the National Committee for Quality Assurance14 patient-centered medical home standards were also reviewed. These standards are 1) enhance access and continuity, 2) identify and manage patient populations, 3) plan and manage care, 4) provide self-care and community support, 5) track and coordinate care, and 6) measure and improve performance. The Interprofessional Education Collaborative Expert Panel core competencies were reviewed to ensure inclusion in the competency tool because the NP residency takes place in an interprofessional learning environment.15 The core competencies for interprofessional collaborative practice include values and ethics, roles and responsibilities, interprofessional communication, and teams and teamwork. The Accreditation Council for Graduate Medical Education core competencies for physician residents were also reviewed and considered when developing the www.npjournal.org
competencies.16 The 6 Accreditation Council for Graduate Medical Education core competencies are medical knowledge, patient care and procedural skills, professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice. Many of these are similar to the NONPF competencies and most are included in this VA CoEPCE NP residency competency tool. The CoEPCEs were mandated to develop and test educational models for all learners that incorporate the core domains of patient-centered team-based care, including shared decision making, sustained relationships, interprofessional collaboration, and performance improvement. Therefore, these domains were also included in the competency tool. The tool development group considered many different rating scales. The goal was to use a scale that was objective and easily measurable. It was decided to use a scale that could show progression to competent/proficient practice without supervision over the course of the training program. The group selected the levels of entrustable professional activities (EPAs) models. EPAs are key professional activities that constitute critical elements that operationally define a profession or discipline.17 EPAs are used in competency-based medical education to operationalize broad complex clinical competencies and allow the clinical supervisor to evaluate tangible, observable clinical activities of the learner over time.18 Examples of EPAs are performing a comprehensive medication review and reconciliation and performing a comprehensive physical examination. Entrustment refers to granting learners the privilege to perform the professional activity and is predicated on the belief that the desired outcome(s) will occur.17 Therefore, the level of entrustment of a learner to perform a professional activity is based on the level of competence he or she has reached. The rating scale is comprised of 5 levels of entrustment: 1, observes task only; 2, needs full supervision; 3, needs supervision periodically; 4, is able to perform without supervision; and 5, able to supervise others. The fifth level, able to supervise others, is considered an aspirational goal for those who exceed the expectations of the program. There is also a not applicable/not performed /not observed rating category. The EPA model seemed especially appropriate for this setting because the NP The Journal for Nurse Practitioners - JNP
e269
residency is embedded in an interprofessional learning environment where physician faculty also precept and evaluate the NP residents. The goal is to be able to eventually use this tool with trainees of other professions. The tool also collects qualitative data. At each time point, the NP resident responds to open-ended questions. These questions ask for comments about what the NP resident perceives he or she does well, could improve, short-term goals, and long-term goals. Additionally, the NP resident is asked how he or she will achieve goals and if there are opportunities and/or obstacles to goal achievement. The competency tool is completed by both the NP resident and the designated mentor (either an NP or physician) independently at 1 month, 6 months, and 12 months of the training program. The NP resident performs a self-evaluation, whereas the mentor evaluates the resident based on direct observation, input from other faculty, and a review of the electronic health record. At each time interval, the NP resident and mentor discuss the competency ratings, progress made, identify learning needs and strategies to achieve competence. Content validity of the competency tool was established by several methods. First, a thorough literature review was conducted, and no existing NP postgraduate competency tools were identified. Second, the competency tool was developed in an iterative process by NP and physician experts in primary care and interprofessional education using the previously described resources. Third, an NP resident completing her training program reviewed the tool and offered suggestions for improvement. Finally, the tool was sent out through the VA advanced practice registered nurse national e-mail group to solicit feedback from both new and experienced NPs working in VA primary care settings. The few who responded thought the tool was satisfactory and did not have substantive comments. The following describes each competency domain and provides examples of the curriculum to achieve competence in each domain.
and manage health conditions that are commonly seen in primary care settings. These health conditions include hypertension, obesity, diabetes mellitus, depression, ischemic heart disease, gastroesophageal reflux, benign prostatic hypertrophy, chronic obstructive pulmonary disease, anemia, chronic renal failure, heart failure, asthma, peripheral arterial disease, and osteoarthritis. Health conditions that are more prevalent in the veteran population such as posttraumatic stress disorder, traumatic brain injury, military sexual trauma, suicidality, and hepatitis C virus are also included. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) perform a comprehensive history and physical examination; 2) construct pertinent differential diagnoses; 3) order appropriate screening diagnostic tests and appropriate medications; 4) perform comprehensive medication review and reconciliation; and 5) present a clear, concise, and organized presentation of the patient. The NP resident also receives monthly data on performance measures for his or her panel of patients on hypertension control, diabetes control, and heart disease management and preventive care measures. Leadership Domain
In 2010, the IOM recommended that nurses assume leadership roles to advance health and nursing practice.7 The CoEPCE NP residency programs build on leadership skills that were learned during formal graduate education by applying them to practice to create future leaders in advanced practice nursing in areas such as patient-centered care, interprofessional education, quality improvement, and health policy. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) lead team huddles, case conferences, team meetings, quality improvement projects, and shared medical/group appointments and 2) apply leadership strategies that support collaborative practice and team effectiveness. The curriculum consists of authentic workplace experiential learning opportunities to achieve leadership competency.
CLINICAL DOMAIN
INTERPROFESSIONAL TEAM COLLABORATION DOMAIN
In the clinical domain, the NP resident is evaluated on his or her competence to assess, diagnose, treat,
Interprofessional team collaboration is guided by 3 assumptions: 1) respect and understanding of
e270
The Journal for Nurse Practitioners - JNP
Volume 12, Issue 6, June 2016
each others roles/scope of practice, 2) improved efficiency (less duplication and fragmentation of effort), and 3) improved quality and cost-effective care.19 Many of the competencies in this domain come from the Interprofessional Education Collaborative Expert Panel.15 The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) develop own professional identity and ability to explain one’s role to patients, families, and other professions; 2) use respectful language and understand and appreciate contributions of other team members; 3) function as a resource to other professions and maintain open communication with team members; 4) safely transition patients among team members; 5) seek feedback from team members; 6) constructively manage disagreements with team; and 7) engage in continuous professional and interprofessional development to enhance team performance. The curriculum consists of authentic workplace experiential learning opportunities and didactic session such as TeamSTEPPS and University of Toronto Centre for Interprofessional Education curricula to achieve interprofessional team collaboration competence.20,21 PATIENT-CENTERED CARE DOMAIN
One of the goals of the CoEPCE was to create an interprofessional workforce that will be capable of working and leading within the PACT model of care to provide quality patient-centered care. Patient-centered care focuses on collaborating with patients to individualize care to the patient’s preferences. The NP resident performs a self-evaluation and is rated by the mentor on the ability to perform the following: 1) Communicate with patients between office visits by telephone, secure messaging, and telehealth monitoring; 2) Elicit patient’s values, preferences, and cultural beliefs; 3) Identify, accommodate, and customize care for patients with language, cognitive, functional, or cultural barriers; 4) Assess and provide education to empower patients to self-manage their chronic conditions; 5) Track and coordinate care by ensuring follow-up on messages, tests, consults, and care at outside facilities; 6) Engage other health professionals in www.npjournal.org
shared patient-centered problem solving; and 7) Use motivational interviewing to help change healthrelated behaviors. The curriculum consists of authentic workplace experiential learning opportunities and didactic session on topics such as motivational interviewing to achieve patient-centered care competence. SHARED DECISION-MAKING DOMAIN
Shared decision making is defined as care that is aligned with the values, preferences, and cultural perspectives of the patient, whereas the curricula focuses on the communication skills necessary to promote the patient’s self-efficacy. The NP resident performs a self-evaluation and is rated by the mentor on the ability to perform the following: 1) Use active listening skills and open-ended question during a patient visit; 2) Counsel and support patients in their self-management of chronic diseases; 3) Facilitate patients’ participation in health care decisions using decision aids; 4) Engage patients in advanced care planning; 5) Activate community resources for patients or populations needs; 6) Engage patients as care team members in tracking and coordinating care; and 7) Share accountability with other professions, patients, and communities for outcomes relevant to prevention and health care. The curriculum consists of authentic workplace experiential learning opportunities and didactic sessions on topics such as The Ottawa Decision Support Tutorial to achieve competence in the shared decision-making domain.22 SUSTAINED RELATIONSHIPS DOMAIN
In the sustained relationship domain, care is designed to promote continuity of care, and the curricula focus on the growth of longitudinal relationships. Sustained relationships develop among patients, learners, faculty, and staff through longitudinal communication (face-to-face encounters, secure messaging, shared medical appointments, workplace learning, and didactic sessions). Ongoing experiences with the team and the assignment of a panel of patients has resulted in the development of accountability or ownership of their practice. The NP resident performs a selfevaluation and is rated by the mentor on the ability to The Journal for Nurse Practitioners - JNP
e271
1) devise, follow, review, and adjust a longitudinal care plan to meet the patient’s needs; 2) develop and sustain a respectful and trusting relationship with the clinic staff, the faculty, their peer learners, and their patients/families; 3) give timely, sensitive instructive feedback to others about their performance on the team; and 4) respond respectfully to feedback from others. The focus is on authentic workplace experiential interprofessional team learning opportunities to achieve competence in the sustained relationship domain. PERFORMANCE IMPROVEMENT DOMAIN
In the performance improvement (PI) domain, care is designed to optimize the health of populations; curricula focus on using the methodology of continuous improvement in redesigning care to achieve quality outcomes. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) access and interpret clinic performance data, 2) improve care through plan-do-study-act cycles, 3) perform root cause analyses and reflect on critical incidents (medical error, near miss, preventable emergency room visits, or readmissions), 4) query registries to determine the health status and care needs of the entire practice and/or specific populations of interest (ie, all diabetic patients), and 5) reflect on individual and team performance and introduce strategies for improvement. Examples of didactic and workplace learning experiences include Lean/Six Sigma certification and the Institute of Healthcare Innovations online modules “Open School,” leading PI projects with an emphasis on interprofessional teams and panel management skills.23 Examples of interprofessional PI projects that NP residents have led and participated in include improving medication reconciliation, rapid access for smoking cessation, decreasing polypharmacy, and reducing nonurgent emergency room visits for musculoskeletal pain. SUMMARY
To date 42 NP residents have completed the VA CoEPCE interprofessional primary care NP residency program, and 38 residents and their mentors have completed the competency tool. The tool was developed to assess change in knowledge, skills, and e272
The Journal for Nurse Practitioners - JNP
behaviors across 7 competency domains as the NP resident moves through the 12-month program and becomes proficient in the CoEPCE core domains and primary care competencies. Preliminary analysis showed that the NP residents have improved significantly in all domains over the 12-month period (P < .0001). Furthermore, NP resident self-evaluation is highly correlated with the mentor’s evaluation across all time points. These results also agree with qualitative comments made by NP residents, as well as other CoEPCE general program evaluation findings, which indicate high satisfaction with the NP residency program. Finally, there are identified areas for program improvement based on the competency tool results. One area includes focusing on strengthening differential diagnostic abilities of the NP residents as they progress through the residency. The assessment and management of veteran-centric health conditions such as posttraumatic stress disorder, military sexual trauma, and traumatic brain injury have also been identified as areas for further development. Curricula to address these needs are being developed. In summary, these positive findings, although preliminary, have encouraged us to share the competency tool with other VA NP residency programs, the private sector, and Health Resources and Services Administration (HRSA)-funded NP residency programs. The next steps include conducting a more detailed psychometric analysis to determine reliability and validity and to further refine tool items. In the interim, the competency tool will continue to be used at the legacy CoEPCE sites and the 2 new CoEPCE sites (West Los Angles and Michael E. DeBakey [Houston] VA Medical Centers). We also plan to expand the use of this tool to evaluate medicine, pharmacy, and psychology trainees involved in the VA CoEPCEs. If you would like to receive a copy of the VA Center of Excellence in Primary Care Education NP Residency competency tool, contact Kathryn Rugen at [email protected] References 1. Heitz LJ, Steiner SH, Burman ME. RN to FNP: a qualitative study of role transition. J Nurs Educ. 2004;43(9):416-420. 2. Huffstutler SY, Varnell G. The imposter phenomenon in new nurse practitioner graduates. Topics Adv Pract Nurs. 2006;6(2). http://www .medscape.com/viewarticle/533648. Accessed August 15, 2015. 3. Hart AM, Macnee CL. How well are nurse practitioners prepared for practice: results of a 2004 questionnaire study. J Am Acad Nurse Pract. 2007;19(1):35-42.
Volume 12, Issue 6, June 2016
4. Rugen KW, Watts SA, Janson SL, et al. Veteran Affairs Centers of Excellence in Primary Care Education: transforming nurse practitioner education. Nurs Outlook. 2014;62(2):78-88. 5. Zapatka SA, Conelius J, Edwards J, Meyer E, Brienza R. Pioneering a primary care adult nurse practitioner interprofessional fellowship. J Nurs Pract. 2014;10(6); 378-38. 6. Brown M, Olshansky EF. From limbo to legitimacy: a theoretical model of the transition to the primary care nurse practitioner role. Nurs Res. 1997;46(1):46-51. 7. Institute of Medicine. The Future of Nursing: Leading Change, Advancing Health. Washington, DC: The National Academies Press; 2010. 8. American Association of Nurse Practitioners, Gerontological Advanced Practice Nurse Association, National Association of Pediatric Nurse Practitioners, National Association of Nurse Practitioners in Women’s Health & National Organization of Nurse Practitioner Faculties. Nurse Practitioner Perspective on Education and Post-Graduate Training. http://c.ymcdn.com/sites/nonpf.site-ym.com/resource/ resmgr/Docs/NPRoundtableStatementPostGra.pdf. Accessed March 21, 2016. 9. Flinter M. From new nurse practitioner to primary care provider: bridging the transition through FQHC-based residency training. Online J Issues Nurs. 2012;17(1):6. 10. Sargent L, Olmedo M. Meeting the needs of new-graduate nurse practitioners: a model to support transition. J Nurs Adm. 2013;43(11):603-610. 11. American Association of Colleges of Nursing. Adult-gerontology primary care nurse practitioner competencies. http://www.nonpf.org/resource/resmgr/ competencies/adult-geropccomps2010.pdf. 2010. Accessed August 16, 2015. 12. National Organization of Nurse Practitioner Faculties. Nurse practitioner core competencies. http://www.nonpf.org/resource/resmgr/competencies/ npcorecompetenciesfinal2012.pdf. 2012. Accessed August 16, 2015. 13. American Association of Colleges of Nursing. The Essentials of Doctoral Education for Advanced Nursing Practice. http://www.aacn.nche.edu/ publications/position/DNPEssentials.pdf. 2006. Accessed August 16, 2015. 14. National Committee for Quality Assurance (NCQA) PCMH standards. http:// www.ncqa.org/Programs/Recognition/Practices/PatientCenteredMedical HomePCMH.aspx. Accessed August 16, 2015. 15. Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, DC: Interprofessional Education Collaborative; 2011. 16. American College of Graduate Medical Education. https://www.acgme.org/ acgmeweb/. Accessed August 16, 2015. 17. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176-1177. 18. Chang A, Bowen JL, Buranosky RA, et al. Transforming primary care trainingepatient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2012;28(6):801-809. 19. Heinemann GD, Schmitt MA, Farrell MP, Brallier SA. Development of an attitudes toward health care teams scale. Eval Health Prof. 1999;22(1):123-142. 20. Team STEPPS. http://teamstepps.ahrq.gov/. Accessed August 16, 2015. 21. University of Toronto Centre for Interprofessional Education. http://www.ipe .utoronto.ca/. Accessed August 16, 2015. 22. The Ottawa Decision Support Tutorial. https://decisionaid.ohri.ca/odsf.html. Accessed. 23. Institute of Healthcare Innovations online modules “Open School.” http:// www.ihi.org/education/ihiopenschool/Pages/default.aspx. Accessed.
www.npjournal.org
Kathryn Wirtz Rugen, PhD, FNP-BC, FAANP, is the national nurse practitioner consultant at VA Centers of Excellence in Primary Care Education and a clinical assistant professor at the College of Nursing at the University of Illinois at Chicago and can be reached at [email protected]. Elena Speroff, DNP, NP-C, is the director of nurse practitioner training at the Boise VA Center of Excellence in Primary Care Education at the Boise VA Medical Center in Boise, ID. Susan A. Zapatka, MSN, ANP-BC, is a liver resource center/hepatitis C advanced practice nurse at VA Connecticut Healthcare System and an associate clinical professor at the School of Nursing at Fairfield University in Fairfield, CT. Rebecca Brienza, MD, MPH is the co-director of the West Haven VA Center of Excellence in Primary Care Education at the VA Connecticut Healthcare System and an assistant professor at Yale University School of Medicine in New Haven, CT. In compliance with national ethical guidelines, the authors report no relationships with business or industry that would pose a conflict of interest. The VA Centers of Excellence in Primary Care Education are funded by the VA Office of Academic Affiliations. The authors would like to acknowledge Sharon A. Watts, DNP, FNP-BC, CDE, Susan L. Janson, PhD, ANP-BC, CNS, FAAN, Laura A. Angelo, MS, ANP-BC, Melanie Nash, DNP, FNP, and Judith L. Bowen, MD, for their participation in the development of the competency tool and Nancy Harada, PhD, PT, for her expertise in the analysis of the competency tool. 1555-4155/16/$ see front matter Published by Elsevier Inc. http://dx.doi.org/10.1016/j.nurpra.2016.02.023
The Journal for Nurse Practitioners - JNP
e273
The Institute of Medicine recommended the implementation of nurse practitioner transition-to-practice programs, either called residency or fellowship, for new graduates. These programs are rapidly expanding on the national level in a variety of practice areas. However, there is a lack of literature on the effectiveness of these programs. The Veterans Affairs Centers of Excellence in Primary Care Education developed a competency-based assessment tool to measure program effectiveness, document the achievement of competency, and promote standardization. This article describes the development of the tool along with curricular examples to promote nurse practitioner transition to practice. Keywords: competency assessment, nurse practitioner postgraduate training Published by Elsevier Inc.
N
ew nurse practitioner (NP) graduates face many challenges as they transition from academia to the complex health care environment, at both the patient and system levels. Despite an excellent academic preparation, the transition to practice can be difficult.1-5 The literature has shown that new NPs described their first year of practice as distressing and tumultuous, with a selfperception that they lacked legitimacy in the NP role.6 The authors concluded that it is imperative to reduce the vulnerability of new NPs.6 Another study revealed that 87% of new NP graduates reported they would be interested in a postgraduate training program if it were available to them.3 Furthermore, the Institute of Medicine recommended advanced practice nurses complete a transition-to-practice program (nurse residency program).7 NP postgraduate training programs are optional and can provide new NPs with “additional management strategies for working with patients, families, or communities where significant financial, social, and emotional co-morbidities make traditional management of health more challenging, such as in www.npjournal.org
community health centers and in the Veterans Affairs (VA) health system.”8 NP postgraduate training programs are in demand as shown by the number of NP trainees who have applied to programs.4,5 Currently, there are NP postgraduate training opportunities in acute care, primary care, psychiatry/ mental health, and specialty areas throughout the United States. These programs offer variable amounts of didactic training, specialty rotations, structured mentorship, and interprofessional collaboration.5,9,10 However, there is a paucity of literature on the effectiveness of these programs. The purpose of this article is to describe the process of the development of a competency-based tool to measure the effectiveness of the VA Center of Excellence in Primary Care Education (CoEPCE) NP residency program. VA CENTERS OF EXCELLENCE IN PRIMARY CARE EDUCATION
In 2011, the VA Office of Academic Affiliations funded the development of 5 VA CoEPCEs. These centers were charged with developing and testing The Journal for Nurse Practitioners - JNP
e267
innovative models of interprofessional education and collaborative practice in the VA primary care setting. In the first year, the West Haven CoEPCE developed a NP residency embedded within the interprofessional learning environment. This NP residency program was developed for new graduates of adult gerontology primary care or family NP programs to learn to work in and lead interprofessional, patient-centered care teams. Interprofessional learning and collaborative practice occurs with physician residents, postdoctorate pharmacy residents, and postdoctorate psychology fellows. Based on the successes of the West Haven NP residency program in recruitment, retention, trainee competency progression, faculty and resident satisfaction, and job placement, the remaining 4 CoEPCEs (Boise, Cleveland, San Francisco, and Seattle) initiated NP postgraduate training programs in subsequent years.4,5 All CoEPCE NP residency training programs are 12 months long and are supported with a trainee stipend that is approximately half of a full-time NP staff position and comparable with the stipend of a first-year physician resident. The stipend includes benefits for health care, vacation, and sick leave. Candidates must have graduated from an accredited master’s or doctor of nursing practice (DNP) program within the past year, have board certification as an adult-gerontology primary care or family NP and state advanced practice registered nurse licensure, have outstanding references from faculty, and have completed a rigorous interviewing process. All the CoEPCEs have been successful in recruiting NP students who have had clinical practicums in CoEPCE to continue in their NP residency program. Across all of the CoEPCE sites, there have been consistently more applicants than positions available. The CoEPCE NP residency curriculum focuses on the advancement of clinical and diagnostic skills as well as leadership and scholarship skills through interprofessional experiential learning opportunities and collaborative care. The NP residents are assigned a panel of patients that they share with their faculty supervisors and at some sites also share or cross cover patients with the physician residents in practice partnership models. They work in a patient-centered medical home model with a teamlet composed of a e268
The Journal for Nurse Practitioners - JNP
registered nurse case manager, licensed practical nurse/licensed vocational nurse, health care technician, and clerical staff. They are also assigned a faculty mentor(s) who could be an NP or physician. At some sites, they are supervised by MD NP dyad faculty mentors. Optional specialty care rotations in areas such as cardiology, dermatology, pulmonology, palliative care, and gastroenterology are available. Several of the CoEPCEs offer an inpatient rotation. NP residents are mentored to lead shared medical appointments and group visits, case conferences, and team huddles. Scholarly pursuits, such as presenting and publishing, are encouraged and mentored. Trainees of all professions learn panel management, performance improvement, and population health skills. Trainees work collaboratively on performance/ quality improvement projects. In the second half of the program, the NP residents participate in precepting NP students and trainees of other professions with supervised mentorship. Development of the NP Residency Competency Tool
In 2012, with the expansion of NP residency programs to all the CoEPCE sites, it became imperative to develop a standardized evaluation process. This was especially important because this was a pilot program incorporated into a high-profile interprofessional education demonstration project. The goal was to develop a tool that would document and show program effectiveness and achievement of competencies essential for interprofessional teambased patient-centered primary care practice. These competencies include the ability to assess, diagnose, treat, and manage common acute and chronic health conditions; patient-centered care; leadership skills; and performance improvement/ population health skills. These competencies are not considered remediation of knowledge and skills already acquired during graduate education but advancement of skills to support the transition to a fully competent and confident provider. The development of the competency tool was an iterative team process led by the NP consultant from the CoEPCE Coordinating Center in collaboration with the NP codirectors from each CoEPCE and the CoEPCE Coordinating Center physician consultant who has expertise in health professions education evaluation. Volume 12, Issue 6, June 2016
Several standards and competencies for NP education were reviewed to ensure the NP residency program would build on the academic curriculum and competencies achieved upon graduation from either a master’s or DNP nurse practitioner program. These included the American Association of Colleges of Nursing competencies for an adult-gerontology primary care NP and the National Organization of Nurse Practitioner Faculties (NONPF) NP core competencies.11,12 It is essential for all NPs, including adult-gerontology primary care NPs, to develop aptitude in the 9 NONPF competency domains before graduation. The 9 domains include scientific foundation, leadership, quality, practice, inquiry, technology and information literacy, policy, health delivery system, ethics, and independent practice.12 The NONPF core competencies are reinforced by the 8 American Association of Colleges of Nursing DNP essential competencies, which include scientific underpinnings for practice, organizational and systems leadership for quality improvement and systems thinking, clinical scholarship and analytical methods for evidence-based practice, information systems and patient care technology, health care policy, and interprofessional collaboration.11,13 Because the NP residency program takes place in the recently redesigned patient-centered primary care setting called patient-aligned care teams (PACT), VA’s model of the patient-centered medical home, the National Committee for Quality Assurance14 patient-centered medical home standards were also reviewed. These standards are 1) enhance access and continuity, 2) identify and manage patient populations, 3) plan and manage care, 4) provide self-care and community support, 5) track and coordinate care, and 6) measure and improve performance. The Interprofessional Education Collaborative Expert Panel core competencies were reviewed to ensure inclusion in the competency tool because the NP residency takes place in an interprofessional learning environment.15 The core competencies for interprofessional collaborative practice include values and ethics, roles and responsibilities, interprofessional communication, and teams and teamwork. The Accreditation Council for Graduate Medical Education core competencies for physician residents were also reviewed and considered when developing the www.npjournal.org
competencies.16 The 6 Accreditation Council for Graduate Medical Education core competencies are medical knowledge, patient care and procedural skills, professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice. Many of these are similar to the NONPF competencies and most are included in this VA CoEPCE NP residency competency tool. The CoEPCEs were mandated to develop and test educational models for all learners that incorporate the core domains of patient-centered team-based care, including shared decision making, sustained relationships, interprofessional collaboration, and performance improvement. Therefore, these domains were also included in the competency tool. The tool development group considered many different rating scales. The goal was to use a scale that was objective and easily measurable. It was decided to use a scale that could show progression to competent/proficient practice without supervision over the course of the training program. The group selected the levels of entrustable professional activities (EPAs) models. EPAs are key professional activities that constitute critical elements that operationally define a profession or discipline.17 EPAs are used in competency-based medical education to operationalize broad complex clinical competencies and allow the clinical supervisor to evaluate tangible, observable clinical activities of the learner over time.18 Examples of EPAs are performing a comprehensive medication review and reconciliation and performing a comprehensive physical examination. Entrustment refers to granting learners the privilege to perform the professional activity and is predicated on the belief that the desired outcome(s) will occur.17 Therefore, the level of entrustment of a learner to perform a professional activity is based on the level of competence he or she has reached. The rating scale is comprised of 5 levels of entrustment: 1, observes task only; 2, needs full supervision; 3, needs supervision periodically; 4, is able to perform without supervision; and 5, able to supervise others. The fifth level, able to supervise others, is considered an aspirational goal for those who exceed the expectations of the program. There is also a not applicable/not performed /not observed rating category. The EPA model seemed especially appropriate for this setting because the NP The Journal for Nurse Practitioners - JNP
e269
residency is embedded in an interprofessional learning environment where physician faculty also precept and evaluate the NP residents. The goal is to be able to eventually use this tool with trainees of other professions. The tool also collects qualitative data. At each time point, the NP resident responds to open-ended questions. These questions ask for comments about what the NP resident perceives he or she does well, could improve, short-term goals, and long-term goals. Additionally, the NP resident is asked how he or she will achieve goals and if there are opportunities and/or obstacles to goal achievement. The competency tool is completed by both the NP resident and the designated mentor (either an NP or physician) independently at 1 month, 6 months, and 12 months of the training program. The NP resident performs a self-evaluation, whereas the mentor evaluates the resident based on direct observation, input from other faculty, and a review of the electronic health record. At each time interval, the NP resident and mentor discuss the competency ratings, progress made, identify learning needs and strategies to achieve competence. Content validity of the competency tool was established by several methods. First, a thorough literature review was conducted, and no existing NP postgraduate competency tools were identified. Second, the competency tool was developed in an iterative process by NP and physician experts in primary care and interprofessional education using the previously described resources. Third, an NP resident completing her training program reviewed the tool and offered suggestions for improvement. Finally, the tool was sent out through the VA advanced practice registered nurse national e-mail group to solicit feedback from both new and experienced NPs working in VA primary care settings. The few who responded thought the tool was satisfactory and did not have substantive comments. The following describes each competency domain and provides examples of the curriculum to achieve competence in each domain.
and manage health conditions that are commonly seen in primary care settings. These health conditions include hypertension, obesity, diabetes mellitus, depression, ischemic heart disease, gastroesophageal reflux, benign prostatic hypertrophy, chronic obstructive pulmonary disease, anemia, chronic renal failure, heart failure, asthma, peripheral arterial disease, and osteoarthritis. Health conditions that are more prevalent in the veteran population such as posttraumatic stress disorder, traumatic brain injury, military sexual trauma, suicidality, and hepatitis C virus are also included. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) perform a comprehensive history and physical examination; 2) construct pertinent differential diagnoses; 3) order appropriate screening diagnostic tests and appropriate medications; 4) perform comprehensive medication review and reconciliation; and 5) present a clear, concise, and organized presentation of the patient. The NP resident also receives monthly data on performance measures for his or her panel of patients on hypertension control, diabetes control, and heart disease management and preventive care measures. Leadership Domain
In 2010, the IOM recommended that nurses assume leadership roles to advance health and nursing practice.7 The CoEPCE NP residency programs build on leadership skills that were learned during formal graduate education by applying them to practice to create future leaders in advanced practice nursing in areas such as patient-centered care, interprofessional education, quality improvement, and health policy. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) lead team huddles, case conferences, team meetings, quality improvement projects, and shared medical/group appointments and 2) apply leadership strategies that support collaborative practice and team effectiveness. The curriculum consists of authentic workplace experiential learning opportunities to achieve leadership competency.
CLINICAL DOMAIN
INTERPROFESSIONAL TEAM COLLABORATION DOMAIN
In the clinical domain, the NP resident is evaluated on his or her competence to assess, diagnose, treat,
Interprofessional team collaboration is guided by 3 assumptions: 1) respect and understanding of
e270
The Journal for Nurse Practitioners - JNP
Volume 12, Issue 6, June 2016
each others roles/scope of practice, 2) improved efficiency (less duplication and fragmentation of effort), and 3) improved quality and cost-effective care.19 Many of the competencies in this domain come from the Interprofessional Education Collaborative Expert Panel.15 The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) develop own professional identity and ability to explain one’s role to patients, families, and other professions; 2) use respectful language and understand and appreciate contributions of other team members; 3) function as a resource to other professions and maintain open communication with team members; 4) safely transition patients among team members; 5) seek feedback from team members; 6) constructively manage disagreements with team; and 7) engage in continuous professional and interprofessional development to enhance team performance. The curriculum consists of authentic workplace experiential learning opportunities and didactic session such as TeamSTEPPS and University of Toronto Centre for Interprofessional Education curricula to achieve interprofessional team collaboration competence.20,21 PATIENT-CENTERED CARE DOMAIN
One of the goals of the CoEPCE was to create an interprofessional workforce that will be capable of working and leading within the PACT model of care to provide quality patient-centered care. Patient-centered care focuses on collaborating with patients to individualize care to the patient’s preferences. The NP resident performs a self-evaluation and is rated by the mentor on the ability to perform the following: 1) Communicate with patients between office visits by telephone, secure messaging, and telehealth monitoring; 2) Elicit patient’s values, preferences, and cultural beliefs; 3) Identify, accommodate, and customize care for patients with language, cognitive, functional, or cultural barriers; 4) Assess and provide education to empower patients to self-manage their chronic conditions; 5) Track and coordinate care by ensuring follow-up on messages, tests, consults, and care at outside facilities; 6) Engage other health professionals in www.npjournal.org
shared patient-centered problem solving; and 7) Use motivational interviewing to help change healthrelated behaviors. The curriculum consists of authentic workplace experiential learning opportunities and didactic session on topics such as motivational interviewing to achieve patient-centered care competence. SHARED DECISION-MAKING DOMAIN
Shared decision making is defined as care that is aligned with the values, preferences, and cultural perspectives of the patient, whereas the curricula focuses on the communication skills necessary to promote the patient’s self-efficacy. The NP resident performs a self-evaluation and is rated by the mentor on the ability to perform the following: 1) Use active listening skills and open-ended question during a patient visit; 2) Counsel and support patients in their self-management of chronic diseases; 3) Facilitate patients’ participation in health care decisions using decision aids; 4) Engage patients in advanced care planning; 5) Activate community resources for patients or populations needs; 6) Engage patients as care team members in tracking and coordinating care; and 7) Share accountability with other professions, patients, and communities for outcomes relevant to prevention and health care. The curriculum consists of authentic workplace experiential learning opportunities and didactic sessions on topics such as The Ottawa Decision Support Tutorial to achieve competence in the shared decision-making domain.22 SUSTAINED RELATIONSHIPS DOMAIN
In the sustained relationship domain, care is designed to promote continuity of care, and the curricula focus on the growth of longitudinal relationships. Sustained relationships develop among patients, learners, faculty, and staff through longitudinal communication (face-to-face encounters, secure messaging, shared medical appointments, workplace learning, and didactic sessions). Ongoing experiences with the team and the assignment of a panel of patients has resulted in the development of accountability or ownership of their practice. The NP resident performs a selfevaluation and is rated by the mentor on the ability to The Journal for Nurse Practitioners - JNP
e271
1) devise, follow, review, and adjust a longitudinal care plan to meet the patient’s needs; 2) develop and sustain a respectful and trusting relationship with the clinic staff, the faculty, their peer learners, and their patients/families; 3) give timely, sensitive instructive feedback to others about their performance on the team; and 4) respond respectfully to feedback from others. The focus is on authentic workplace experiential interprofessional team learning opportunities to achieve competence in the sustained relationship domain. PERFORMANCE IMPROVEMENT DOMAIN
In the performance improvement (PI) domain, care is designed to optimize the health of populations; curricula focus on using the methodology of continuous improvement in redesigning care to achieve quality outcomes. The NP resident performs a self-evaluation and is rated by the mentor on the ability to 1) access and interpret clinic performance data, 2) improve care through plan-do-study-act cycles, 3) perform root cause analyses and reflect on critical incidents (medical error, near miss, preventable emergency room visits, or readmissions), 4) query registries to determine the health status and care needs of the entire practice and/or specific populations of interest (ie, all diabetic patients), and 5) reflect on individual and team performance and introduce strategies for improvement. Examples of didactic and workplace learning experiences include Lean/Six Sigma certification and the Institute of Healthcare Innovations online modules “Open School,” leading PI projects with an emphasis on interprofessional teams and panel management skills.23 Examples of interprofessional PI projects that NP residents have led and participated in include improving medication reconciliation, rapid access for smoking cessation, decreasing polypharmacy, and reducing nonurgent emergency room visits for musculoskeletal pain. SUMMARY
To date 42 NP residents have completed the VA CoEPCE interprofessional primary care NP residency program, and 38 residents and their mentors have completed the competency tool. The tool was developed to assess change in knowledge, skills, and e272
The Journal for Nurse Practitioners - JNP
behaviors across 7 competency domains as the NP resident moves through the 12-month program and becomes proficient in the CoEPCE core domains and primary care competencies. Preliminary analysis showed that the NP residents have improved significantly in all domains over the 12-month period (P < .0001). Furthermore, NP resident self-evaluation is highly correlated with the mentor’s evaluation across all time points. These results also agree with qualitative comments made by NP residents, as well as other CoEPCE general program evaluation findings, which indicate high satisfaction with the NP residency program. Finally, there are identified areas for program improvement based on the competency tool results. One area includes focusing on strengthening differential diagnostic abilities of the NP residents as they progress through the residency. The assessment and management of veteran-centric health conditions such as posttraumatic stress disorder, military sexual trauma, and traumatic brain injury have also been identified as areas for further development. Curricula to address these needs are being developed. In summary, these positive findings, although preliminary, have encouraged us to share the competency tool with other VA NP residency programs, the private sector, and Health Resources and Services Administration (HRSA)-funded NP residency programs. The next steps include conducting a more detailed psychometric analysis to determine reliability and validity and to further refine tool items. In the interim, the competency tool will continue to be used at the legacy CoEPCE sites and the 2 new CoEPCE sites (West Los Angles and Michael E. DeBakey [Houston] VA Medical Centers). We also plan to expand the use of this tool to evaluate medicine, pharmacy, and psychology trainees involved in the VA CoEPCEs. If you would like to receive a copy of the VA Center of Excellence in Primary Care Education NP Residency competency tool, contact Kathryn Rugen at [email protected] References 1. Heitz LJ, Steiner SH, Burman ME. RN to FNP: a qualitative study of role transition. J Nurs Educ. 2004;43(9):416-420. 2. Huffstutler SY, Varnell G. The imposter phenomenon in new nurse practitioner graduates. Topics Adv Pract Nurs. 2006;6(2). http://www .medscape.com/viewarticle/533648. Accessed August 15, 2015. 3. Hart AM, Macnee CL. How well are nurse practitioners prepared for practice: results of a 2004 questionnaire study. J Am Acad Nurse Pract. 2007;19(1):35-42.
Volume 12, Issue 6, June 2016
4. Rugen KW, Watts SA, Janson SL, et al. Veteran Affairs Centers of Excellence in Primary Care Education: transforming nurse practitioner education. Nurs Outlook. 2014;62(2):78-88. 5. Zapatka SA, Conelius J, Edwards J, Meyer E, Brienza R. Pioneering a primary care adult nurse practitioner interprofessional fellowship. J Nurs Pract. 2014;10(6); 378-38. 6. Brown M, Olshansky EF. From limbo to legitimacy: a theoretical model of the transition to the primary care nurse practitioner role. Nurs Res. 1997;46(1):46-51. 7. Institute of Medicine. The Future of Nursing: Leading Change, Advancing Health. Washington, DC: The National Academies Press; 2010. 8. American Association of Nurse Practitioners, Gerontological Advanced Practice Nurse Association, National Association of Pediatric Nurse Practitioners, National Association of Nurse Practitioners in Women’s Health & National Organization of Nurse Practitioner Faculties. Nurse Practitioner Perspective on Education and Post-Graduate Training. http://c.ymcdn.com/sites/nonpf.site-ym.com/resource/ resmgr/Docs/NPRoundtableStatementPostGra.pdf. Accessed March 21, 2016. 9. Flinter M. From new nurse practitioner to primary care provider: bridging the transition through FQHC-based residency training. Online J Issues Nurs. 2012;17(1):6. 10. Sargent L, Olmedo M. Meeting the needs of new-graduate nurse practitioners: a model to support transition. J Nurs Adm. 2013;43(11):603-610. 11. American Association of Colleges of Nursing. Adult-gerontology primary care nurse practitioner competencies. http://www.nonpf.org/resource/resmgr/ competencies/adult-geropccomps2010.pdf. 2010. Accessed August 16, 2015. 12. National Organization of Nurse Practitioner Faculties. Nurse practitioner core competencies. http://www.nonpf.org/resource/resmgr/competencies/ npcorecompetenciesfinal2012.pdf. 2012. Accessed August 16, 2015. 13. American Association of Colleges of Nursing. The Essentials of Doctoral Education for Advanced Nursing Practice. http://www.aacn.nche.edu/ publications/position/DNPEssentials.pdf. 2006. Accessed August 16, 2015. 14. National Committee for Quality Assurance (NCQA) PCMH standards. http:// www.ncqa.org/Programs/Recognition/Practices/PatientCenteredMedical HomePCMH.aspx. Accessed August 16, 2015. 15. Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, DC: Interprofessional Education Collaborative; 2011. 16. American College of Graduate Medical Education. https://www.acgme.org/ acgmeweb/. Accessed August 16, 2015. 17. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176-1177. 18. Chang A, Bowen JL, Buranosky RA, et al. Transforming primary care trainingepatient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2012;28(6):801-809. 19. Heinemann GD, Schmitt MA, Farrell MP, Brallier SA. Development of an attitudes toward health care teams scale. Eval Health Prof. 1999;22(1):123-142. 20. Team STEPPS. http://teamstepps.ahrq.gov/. Accessed August 16, 2015. 21. University of Toronto Centre for Interprofessional Education. http://www.ipe .utoronto.ca/. Accessed August 16, 2015. 22. The Ottawa Decision Support Tutorial. https://decisionaid.ohri.ca/odsf.html. Accessed. 23. Institute of Healthcare Innovations online modules “Open School.” http:// www.ihi.org/education/ihiopenschool/Pages/default.aspx. Accessed.
www.npjournal.org
Kathryn Wirtz Rugen, PhD, FNP-BC, FAANP, is the national nurse practitioner consultant at VA Centers of Excellence in Primary Care Education and a clinical assistant professor at the College of Nursing at the University of Illinois at Chicago and can be reached at [email protected]. Elena Speroff, DNP, NP-C, is the director of nurse practitioner training at the Boise VA Center of Excellence in Primary Care Education at the Boise VA Medical Center in Boise, ID. Susan A. Zapatka, MSN, ANP-BC, is a liver resource center/hepatitis C advanced practice nurse at VA Connecticut Healthcare System and an associate clinical professor at the School of Nursing at Fairfield University in Fairfield, CT. Rebecca Brienza, MD, MPH is the co-director of the West Haven VA Center of Excellence in Primary Care Education at the VA Connecticut Healthcare System and an assistant professor at Yale University School of Medicine in New Haven, CT. In compliance with national ethical guidelines, the authors report no relationships with business or industry that would pose a conflict of interest. The VA Centers of Excellence in Primary Care Education are funded by the VA Office of Academic Affiliations. The authors would like to acknowledge Sharon A. Watts, DNP, FNP-BC, CDE, Susan L. Janson, PhD, ANP-BC, CNS, FAAN, Laura A. Angelo, MS, ANP-BC, Melanie Nash, DNP, FNP, and Judith L. Bowen, MD, for their participation in the development of the competency tool and Nancy Harada, PhD, PT, for her expertise in the analysis of the competency tool. 1555-4155/16/$ see front matter Published by Elsevier Inc. http://dx.doi.org/10.1016/j.nurpra.2016.02.023
The Journal for Nurse Practitioners - JNP
e273