- Email: [email protected]
Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study
Journal Pre-proof
Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study Marleen Smits , Yvonne Peters , Sander Ranke , Erik Plat , Miranda Laurant , Paul Giesen PII: DOI: Reference:
S0020-7489(19)30252-4 https://doi.org/10.1016/j.ijnurstu.2019.103445 NS 103445
To appear in:
International Journal of Nursing Studies
Received date: Revised date: Accepted date:
19 March 2019 20 September 2019 24 September 2019
Please cite this article as: Marleen Smits , Yvonne Peters , Sander Ranke , Erik Plat , Miranda Laurant , Paul Giesen , Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study, International Journal of Nursing Studies (2019), doi: https://doi.org/10.1016/j.ijnurstu.2019.103445
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier Ltd.
Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study Marleen Smits, PhD1*; Yvonne Peters, Msc1; Sander Ranke1, Msc; Erik Plat1, MD, Msc; Miranda Laurant, PhD1,2; Paul Giesen, MD, PhD1 1 Radboud
university medical center, Radboud Institute for Health Sciences, Scientific Center for
Quality of Healthcare (IQ healthcare), Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands 2 HAN
University of Applied Sciences, Faculty of Health and Social Studies, Kapittelweg 33, 6525
EN Nijmegen, The Netherlands *Corresponding author: Dr. Marleen Smits Radboud university medical center P.O Box 9101, 114 IQ healthcare 6500 HB Nijmegen The Netherlands [email protected] #31 (0) 24 3666264 Email addresses: MS: [email protected] YP: [email protected] SR: [email protected] EP: [email protected] ML: [email protected] PG: [email protected] Running title: Substitution out-of-hours primary care home visits Word count: 4429 (6383 including tables, figures and references)
Abstract Background: General practitioners experience a high workload during out-of-hours care. A possible solution is the shifting of care to nurse practitioners. Objectives: To provide insight into patient- and care characteristics, safety, efficiency, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-ofhours primary care services. Design: Quasi-experimental non-randomised study comparing home visits by nurse practitioners (intervention group; one out-of-hours care service) with home visits by general practitioners (control group; two out-of-hours care services) for 24 protocolised health problems. Setting: Three out-of-hours primary care services in the East of the Netherlands. Participants: 1,601 patients who received a home visit by a nurse practitioner (N=386) or a general practitioner (N=1,215). Of these patients, 639 gave informed consent to be included in the protocol adherence assessment and follow-up record review (nurse practitioner: N=358; general practitioner: N=281). Methods: Five nurse practitioners with experience in ambulance care were recruited and trained. From September 2016 to March 2017 the nurse practitioners took over home visits under supervision of a general practitioners. This was evaluated using: (1) data-extraction from the patient registration system, (2) follow-up record review in the patients’ general practices, and (3) patient satisfaction survey. Two general practitioners independently assessed protocol adherence based on the extracted registration data. Results: Nurse practitioners prescribed medication significantly less often than general practitioners (19.9% versus 30.6%), and referred patients significantly more often to the hospital (24.1% versus 15.9%). The mean length of the home visit was significantly longer for nurse practitioners (34.1 versus 21.1 minutes). Nurse practitioners adhered to the protocol significantly more often than general practitioners (84.9% versus 76.2%) and their medication prescribing was significantly more often appropriate (93.7% versus 79.5%). There were no differences in the number of missed diagnoses and complications. The number of follow-up contacts was also similar in both groups. Patient satisfaction was generally high and significantly higher for nurse practitioners on several items. Conclusions: Nurse practitioners with experience in ambulance care can safely, efficiently, and satisfactorily perform low complex out-of-hours primary care home visits. It is recommended to study the safety and efficiency of nurse practitioners’ home visits in other regions and with nurse
practitioners with different educational levels and different specialisations. In addition, we recommend to evaluate the cost-effectiveness and if it leads increased quality of care.
Keywords (MeSH): Emergency Care, General Practitioners, Nurses, Out-of-Hours Medical Care, Practitioner Cooperative, Primary Health Care
Contribution of the paper What is already known about the topic?
General practitioners experience a high workload during out-of-hours primary care.
Previous studies into substitution of general practitioners by nurse practitioners during clinic consultations, both in daytime general practices and in out-of-hours primary care services, have shown positive results in terms of workload for general practitioners, patient experiences and care outcomes.
Little is known on the effects of substitution during out-of-hours primary care home visits.
What this paper adds?
This study demonstrates that nurse practitioners, compared to general practitioners, more often adhered to protocols, less often prescribed medication, and obtained higher patient satisfaction during out-of-hours primary care home visits, but the mean duration of the home visit was longer and they more often referred patients to hospital.
The study shows that the nurse practitioners were equivalent to general practitioners in terms of the number of follow-up contacts, missed diagnoses, and complications.
Introduction In many Western countries, hospital emergency departments are overcrowded, leading to the desire to strengthen primary care, particularly out-of-hours (Moskop et al., 2009; Bellow and Gillespie, 2014). To make the out-of-hours primary care system stronger, many Western nations have reorganised it from individual general practices and rota groups (small groups of GP’s who are on duty alternately) into large-scale General Practitioner Cooperatives (Berchet and Nader, 2016). In these primary care services a large number of general practitioners take turns being on duty during out-of-hours for the patient population of all participating general practitioners. The services are intented for primary care problems that cannot wait until the regular consultation hours of the patient’s own general practitioner. General practitioners in the cooperatives may be supported by nurses, management, drivers etc. Reasons for this restructuring included high workload and reluctance by general practitioners to commit to being on call 24/7, regional shortages of general practitioners, and increasing patient demand for out-of-hours care (Leibowitz et al., 2003; Huibers et al., 2009). The challenge for policymakers is to find a suitable model for out-of-hours care and continuously make adjustments as a reaction to the changing context to ensure accessibility, safety, efficiency, and the satisfaction of both patients and health care professionals (Huibers et al., 2014, Coombes, 2016). Around the year 2000, general practitioners in the Netherlands reorganised their out-of-hours care into large-scaled general practitioner cooperatives, thereby taking their responsibility to provide primary care 24/7 (Giesen et al., 2011; Smits et al., 2017)(Table 1 Online supplementary file). A patient contacting a Dutch cooperative receives a telephone advice, clinic consultation or home visit. The use of these cooperatives has increased over the years. General practitioners now experience a high workload, which could have a negative impact on the quality of care and their motivation to perform out-of-hours shifts (Smits et al., 2014). A range of developments possibly increase workload. These include societal, economic and demographic changes, such as a decreased number of emergency departments, patient flows from hospital to the general practitioner, aging of the population and increased co-morbidity, and staff shortages (Smits et al., 2017). As many health problems in out-of-hours care do not necessarily require the knowledge and skills of a general practitioner, using trained nurses to expand the capacity of the primary care workforce is a topic of increasing interest (Kooienga and Carryer, 2015; Maier et al., 2016; Laurant et al., 2018). A shift of care from general practitioners to advanced practice nursing roles, is one strategy for improving access to care and improve efficiency and quality of care . Previous studies into substitution of general practitioners by nurse practitioners, mostly during clinic consultations in
daytime general practices, have shown positive results in terms of workload for general practitioners, patient experiences and care outcomes (Dierick-van Dale et al., 2009; van der Biezen et al., 2016, 2017; Laurant et al., 2018; Collins, 2019). In addition, there are indications that costs savings can be realised, although further research is recommended on this point (Dierick-van Daele et al., 2008; Martinez-Gonzales et al., 2014; Laurant et al., 2018). A study in one out-of-hours primary care service showed that nurse practitioners treat similar patients as the general practitioner and can do about 77% of all clinic consultations autonomously without consulting a general practitioner (van der Biezen et al., 2016). There are only a few reported studies on the effects of substitution of general practitioners by nurse practitioners during home visits (Edwards et al., 2009; Collins, 2019), and only one of them in out-of-hours (Collins, 2019). The findings of these studies support the indication that nurse practitioners are competent and achieve equivalent clinical outcomes compared to general practitioners, but the studies had some methodological limitations. The expectation is that when experienced and trained nurse practitioners take over low-complex protocolised home visits in outof-hours primary care, this will reduce the workload of general practitioners, while the quality of care and patient satisfaction remains the same. Consequently, general practitioners will have more time for high-complex patient problems, both during home visits and clinic consultations. Out-of-hours home visits of general practitioner cooperatives are regarded as a high-risk primary care setting due to the fact that patients (mostly elderly) are assessed at relatively high urgency and with little diagnostic support in the home setting. Therefore, it is important to evaluate the safety of substitution general practitioners by nurse practitioners in this setting. Our aim was to provide insight into the safety, efficiency, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-of-hours primary care general practitioner cooperatives.
Materials and Methods Design and setting A quasi-experimental non-randomised study was conducted. Nurse practitioners substituted general practitioners to carry out out-of-hours home visits for 24 predefined low complex health problems at one general practitioner cooperative in the east of the Netherlands (intervention group). The control group consisted of two other general practitioner cooperatives where general practitioners provided care as usual. The three general practitioner cooperatives were located in the same region with a similar patient population and policy.
We used three sources of data: (1) data-extraction from the patient registration system of the general practitioner cooperative, (2) follow-up record review by the patient’s own general practitioner, and (3) patient experiences survey. The intervention period was seven months, from September 2016 to March 2017.
Patients Patients who received a home visit for one of the 24 predefined health problems (see Table 2 Online supplementary file) were included in the study . Home visits at night were excluded from the study.
Intervention Five registered nurse practitioners were recruited for the study. They were all experienced as ambulance nurses before they entered the Master of Nurse Practitioner (MANP). In the Netherlands, the MANP is a 2 year master’s programme at universities of applied sciences. The education consists of a dual work-education model, meaning that students are employed within a practice (including general practices) and receive salary. About two-third of the education takes place on the job, learning by doing (Freund et al., 2015). The nurse practitioners were all male and had more than 20 years of experience as ambulance nurses and had two to six years of experience with clinic consultations at a general practitioner cooperative. They substituted general practitioners when visiting patients at home during weekday evenings (5:00 PM‐11:30 PM) and weekends (10:00 AM‐6:00 PM). They were supervised by a general practitioner of the general practitioner cooperative. It was decided that the nurse practitioners could take care of 24 low-complex health problems which could be treated according to a protocol. The triage professionals (often trained practice nurses) and supervising general practitioner of the cooperative assigned patients with one of the 24 eligible health problems to the nurse practitioners after telephone triage. In case of a high-risk health problem or U0 call (resuscitation), a general practitioner performed the home visit or an ambulance was called. The 24 eligible health problems presented at triage were: arm complaints, leg complaints, insult, nosebleed, extremity trauma, neck trauma, back trauma, skull trauma, thorax trauma, face trauma, abdominal trauma, burn/inhalation trauma, intoxication, shortness of breath, palpitations, neurological failure, disrupted diabetes, thorax pain, back pain, urinary tract problems, wound, fainting, dizziness, and headache (Table 2 Online supplementary file). For each of these health problems a protocol was written, based on the national general practitioner guidelines (NHG, 2019), and before the intervention the nurse practitioners received a training in these protocols, offered by the ambulance service and general practitioner cooperative. The general practitioners were used to working with the national guidelines
during daytime and out-of-hours primary care and therefore did not receive any additional training. The nurse practitioners used a car ambulance for the home visit and the general practitioners used the usual general practitioner cooperative car. General practitioners had a chauffeur who could give them medical assistance, if necessary. Nurse practitioners had no chauffeur. Both cars had no possibility to transport patients and the equipment was generally equal (e.g. oxygen, intravenous drip equipment, automated external defibrillator, and medication for acute treatment). Additionally, the car of the nurse practitioners had electrocardiogram equipment, which was not available for general practitioners. Data from the first month of the intervention period were not used in the study, to give the nurse practitioners time to get familiar with performing home visits. Hence, the study period was six months.
Informed consent In both groups patients were asked to confirm participation in non-anonymous parts of the study (protocol adherence assessment and follow-up record review study) and in the patient survey. The procedure to obtain informed consent differed between the groups. In the intervention group, the nurse practitioners obtained informed consent from each patient at the end of the home visit. Patients could give back the signed form directly to the nurse practitioners or they could send it to the research institute in a prepaid return envelope. In the control group, patients received the informed consent form from the researchers by post within one week after the home visit. They could return the informed consent form with a prepaid return envelope to the research institute.
Data collection and outcomes Data extraction During the intervention period anonymous routinely obtained data were extracted from the electronic registration systems of the three general practitioner cooperatives and included the following patient characteristics: age, gender, date home visit, health problem presented during triage, urgency level during triage (U1-U5), medication prescription, ambulance callout, referral to emergency department, International Classification Primary Care code, chronic disease, and free text SOAP note (Subjective, Objective, Assessment, Plan). Nurse practitioners recorded in the electronic registration system if they had contacted the general practitioner from the general practitioner cooperative. The following actions could be reported: telephone consultation with the supervising
general practitioner with no further action or followed by a home visit by a general practitioner (second opinion). These data were also extracted.
Assessment of protocol adherence Two experienced general practitioners independently assessed protocol adherence based on all extracted patient registration data, including free text SOAP notes, following a procedure used in earlier research (Willekens et al., 2011). These reviewers used the 24 protocols as guiding principle. Adequate care was defined as following the protocol or reasoned deviating from the protocol. For cases in which the initial assessment of the reviewers differed, a consensus procedure followed. The reviewers were first asked if they wanted to alter their assessment to conform with the other reviewer. Cases with persistent differences were discussed in a face-to-meeting of the reviewers, together with an expert general practitioner. The expert general practitioner could make a final decision if necessary.
Follow-up record review After the intervention period, a follow-up record review study was performed in patients’ own general practices. The general practitioners received a postal questionnaire for each patient with questions regarding changes in diagnosis or policy, complications, and follow-up contacts within the same disease episode as the index home visit. In case of a complication, general practitioners were asked to give a description. The descriptions of missed diagnoses and complications were assessed by the researchers to determine their preventability and severity. The follow-up duration was one month from the general practitioner cooperative home visit. Research into hospital readmissions has shown that this is the optimal length of the follow-up period (Halfon et al., 2002). General practitioners were asked to send the questionnaires back to the research institute in a prepaid return envelope. If there was no response, a reminder was sent after 14 days. After the first reminder, general practitioners who had more than six patients included in the study were telephonically reminded and were offered assistance. Some general practitioners were visited by one of the researchers who gathered the required data from the patient records.
Patient experiences survey A Patient Reported Experience Measure (PREM) questionnaire developed and validated for general practitioner cooperatives was used to measure patient experiences (Keizer et al., 2017). Patients were asked to respond to statements about their experiences with the care provider during the
home visit. Statements were about communication, patient-centeredness, expertise of care provider, shared decision making, continuity of care, usefulness of advice, maintenance of care, result of care, and waiting time. The answering scale was a five-point Likert scale ranging from totally disagree to totally agree. In addition, patients were asked to give an overall rating of the care provider using a school grade from 1 to 10. Patients received the questionnaire by postal mail within one week after the home visit. They were asked to send the questionnaire back to the research institute in a prepaid return envelope. If there was no response, a reminder was sent after 14 days.
Sample size To calculate the sample size needed, we used a non-inferiority approach. To examine if protocol adherence of nurse practitioners was equal to general practitioners a total number of 333 patients were needed in both the intervention and control group. For this calculation we estimated the mean protocol adherence at 77% (Willekens et al., 2011) and we used a non-inferiority margin of 10%, a power of 80% and a 95% two-sided confidence interval. Based on the incidence of the 24 health problems, we estimated that a study period of six months would yield enough home visits by nurse practitioners. We used two general practitioner cooperatives as control settings, because it was estimated that the loss of patients in the control group would be higher, because patients received the informed consent form by post without faceto-face information about the study. Blinding The assessment of protocol adherence was blinded: the general practitioner reviewers did not know who performed the home visit. It was not possible to blind the patients to study condition. In the follow-up study in the patients’ own general practices, it was also not possible to blind the general practitioners.
Statistical analyses Home visits were the unit of analysis. Patient- and care characteristics, patient experiences, and follow-up data were analysed using descriptive statistics. The duration of the home visit was calculated by subtracting the start time from the end time of the home visit as recorded in the electronic patient file. Home visits with a missing time (N=53) and home visits with an unlikely duration (below five minutes or above two hours; N=66) were excluded in the calculation of the mean home visit duration. For the items on patient experiences, the percentages of positive scores (“agree” and “totally agree”) were calculated.
The total intervention group and control group were compared on several background items, using a T-test for age and Chi-square tests for gender (male/female), urgency (U1-U5) and time of home visit (week/weekend). To compare medication prescription and referral between both groups, we used logistical regression analyses, correcting for differences in urgency level, time of home visit, patients with shortness of breath and trauma. To compare the duration of the home visit, we used linear regression analyses, correcting for urgency level, time of home visit, patients with shortness of breath and trauma. For the subset of patients with informed consent, the analyses on background items (age, gender, urgency, time of home visit and health problem presented) were repeated to check for possible covariates. In the comparative analyses regarding protocol adherence and follow-up we used logistical regression analyses, correcting for urgency level, patients with trauma, back pain, headache and diabetes. Patient experiences from the intervention and control group were compared using Ttests on the total five-point scale. Questionnaires with a missing response on >5 questions were excluded from the analyses. All statistical analyses were carried out using IBM SPSS software version 25.
Results
Patient characteristics A total of 1,601 patients from the three general practitioner cooperatives received a home visit: 386 were visited by a nurse practitioner and 1,215 by a general practitioner (Figure 1). Table 1 shows patient and care characteristics of both groups. Most patients were triaged at urgency level U2 (Acute) and U3 (Urgent). Patients that were visited by a nurse practitioner, had health problems with significantly lower urgency levels than patients that were visited by a general practitioner. The most frequently presented health problem was shortness of breath in both groups, although significantly more frequent in home visits by general practitioners (32.2%) than by nurse practitioners (26.2%). Traumas were significantly more frequent in home visits by nurse practitioners (11.1%) than by general practitioners (6.4%). None of the other health problems differed significantly (Table 1).
Care characteristics Nurse practitioners prescribed significantly less often medication than general practitioners (19.9% nurse practitioner versus 30.6% general practitioner), and significantly more often referred the patient to the hospital emergency department (24.1% nurse practitioner versus 15.9% general practitioner). In addition, the mean duration of the home visit was significantly longer for nurse practitioners (34.1 minutes) than for general practitioners (21.1 minutes)(Table 1). The nurse practitioners consulted the supervising general practitioner in 21.5% of all home visits. In one case this resulted in a visit of a general practitioner to the patient home (not in table).
Table 1. Patient and care characteristics of patients who were visited by a nurse practitioner (intervention group; N=386) and general practitioner (control group; N=1,215) Characteristic
Intervention group
Control group
Nurse practitioner
General practitioner
Total
% (N)
% (N)
% (N)
73.0 (15.1)
74.6 (16.0)
74.2 (15.8)
95% CI
71.4-74.5
73.7-75.5
73.4-75.0
Gender male
45.6 (176)
43.3 (526)
43.8 (702)
0.3 (1)
3.0 (36)
2.3 (37)
U2 – Acute
48.2 (186)
53.1 (645)
51.9 (831)
U3 – Urgent
47.4 (183)
41.8 (508)
43.2 (691)
U4 – Routine
2.8 (11)
1.8 (22)
2.1 (33)
U5 – Advice
1.3 (5)
0.3 (4)
0.6 (9)
26.2 (101)
32.2 (392)
30.8 (493)
11.1 (43)
6.4 (78)
7.6 (121)
Thorax pain
8.3 (32)
10.0 (122)
9.6 (154)
Dizziness
6.5 (25)
5.8 (71)
6.0 (96)
Leg complaints
6.0 (23)
6.1 (74)
6.1 (97)
Urinary tract problems
6.0 (23)
5.9 (72)
5.9 (95)
Wound
5.7 (22)
4.1 (50)
4.5 (72)
Neurological failure
5.2 (20)
7.0 (85)
6.6 (105)
Back pain
4.7 (18)
6.5 (79)
6.1 (97)
Age (years) mean (SD)
Urgency level** U1 – Life-threatening
Health problem presented Shortness of breath* Trauma (8 health problems)**
Palpitations
3.4 (13)
2.7 (33)
2.9 (46)
Syncope
2.1 (8)
3.5 (43)
3.2 (51)
Headache
1.6 (6)
3.0 (36)
2.6 (42)
Disrupted diabetes
1.3 (5)
1.8 (22)
1.7 (27)
Nosebleed
1.0 (4)
2.5 (30)
2.1 (34)
Arm complaints
0.8 (3)
1.1 (13)
1.0 (16)
Intoxication
0.5 (2)
0.3 (4)
0.4 (6)
Insult
0.3 (1)
0.9 (11)
0.7 (12)
9.8 (37)
-
2.3 (37)
Medication prescription**a
19.9 (77)
30.6 (372)
28.0 (449)
Referral*** a
24.1 (93)
15.9 (193)
17.9 (286)
34.1 (13.8)
21.1 (10.4)
24.0 (12.5)
31.9-34.8
20.4-21.6
23.2-24.5
Other/ not known
Duration home visit (minutes) *** a mean (SD) 95% CI a
Corrected for urgency level, time of home visit, shortness of breath and trauma.
*p<0.05; **p<0.01; ***p<0.001.
Protocol adherence To study protocol adherence, 639 patients with signed informed consent forms, were included in the analyses: 358 nurse practitioner visits and 281 general practitioner visits (Figure 1). Nurse practitioners significantly more often adhered to the protocol than general practitioners (84.9% nurse practitioner versus 76.2% general practitioner). A reasoned deviation from the protocol was also seen as adequate care (2.0% nurse practitioner versus 5.7% general practitioners, not significant), but incorrect deviation of the protocol could be harmful to the patient (13.1% nurse practitioner versus 18.1% general practitioners, not significant). Prescription of medication was the most frequent incorrect deviation from the protocol (51%), but did not significantly differ between professionals (Table 2).
Table 2. Protocol adherence, reasoned deviation, incorrect deviation and reason for incorrect deviation by a nurse practitioner (intervention group; N=358) and general practitioner (control group N=281) Item
Intervention group
Control group
Nurse practitioner
General practitioner
% (N)
% (N)
% (N)
84.9 (304)
76.2 (214)
81.1 (518)
2.0 (7)
5.7 (16)
3.6 (23)
13.1 (47)
18.1 (51)
15.3 (98)
Incorrect medication
51.1 (24)
51.0 (26)
51.0 (50)
Incorrect diagnostics
12.8 (6)
7.8 (4)
10.2 (10)
Incorrect referral
6.4 (3)
3.9 (2)
5.1 (5)
Other
21.3 (10)
19.6 (10)
20.4 (20)
Combination of reasons
8.5 (4)
17.7 (9)
13.3 (13)
Protocol adherence* Reasoned deviation Incorrect deviation
Total
Reason for incorrect deviationa
All analyses corrected for urgency level, trauma, back pain, headache and diabetes. *p<0.01 a
Percentage of patients with incorrect deviations.
Follow-up Of 620 patients who gave their informed consent to follow-up with the own general practitioner, the postal address of the general practices was known. A total of 420 forms were returned by the general practitioners (response rate 67.7%) (Figure 1). There were no significant differences between the groups in the frequency of missed diagnoses and complications reported by the patients’ own general practitioners. Based on the descriptions of these cases the researchers assessed that in each group two complications were preventable. The severity of these preventable complications was somewhat higher for home visits by nurse practitioners (see descriptions in footnote of Table 3). There were neither significant differences in the frequency of follow-up contacts within 72 hours nor within one month after the home visit. There were also no significant differences in types of followup contacts <72 hours. Protocol adherence (99.5%) as rated by the patients’ own general practitioner, was significantly higher for nurse practitioners (99.5%) than for general practitioners (92.3%). Appropriate referral and continuity of care as rated by the patients’ own general practitioner was generally equal between both groups. Appropriate medication prescribing (93.7%) was rated significantly higher for nurse practitioners than for general practitioners (79.5%) (Table 3).
Table 3. Assessment of care during home visit by a nurse practitioner (intervention group; N=231) and general practitioner (control group; N=189) by patients own general practitioner Item
Intervention group
Control group
Nurse practitioner
General
% (N)
practitioner
Total
% (N)
% (N) Missed diagnosis
8.5 (19)
6.0 (11)
7.4 (30)
11.0 (25)
9.0 (17)
10.1 (42)
0.9 (2)
1.1 (2)
1.0 (4)
Follow-up contact < 1 month
63.2 (146)
64.0 (121)
63.6 (267)
Follow-up contact <72 hours; contact typeb:
51.5 (119)
56.1 (106)
53.6 (225)
Own general practitioner
34.2 (79)
41.3 (78)
37.4 (157)
General practitioner cooperative
6.9 (16)
6.9 (13)
6.9 (29)
Emergency department
9.1 (21)
4.8 (9)
7.1 (30)
National emergency number (112)
0.9 (2)
0.0 (0)
0.5 (2)
Outpatient hospital service
1.7 (4)
3.2 (6)
2.4 (10)
Hospitalisation
12.1 (28)
16.9 (32)
14.3 (60)
0.4 (1)
1.6 (3)
1.0 (4)
0.9 (2)
0.5 (1)
0.7 (3)
Protocol adherence*
99.5 (192)
92.3 (144)
96.3 (336)
Appropriate referral
85.5 (53)
95.6 (65)
90.8 (118)
98.4 (181)
99.3 (152)
98.8 (333)
93.7 (74)
79.5 (66)
86.4 (140)
Complication Preventable complicationa
Nursing home/Residential care home/ Hospice Other
Continuity of care Appropriate prescribing*
All analyses corrected for urgency level, trauma, back pain, headache and diabetes. a
Nurse practitioner: 1) perforated appendicitis; 2) crisis situation related to secondary dislocated
medial collum fracture. General practitioner: 1) increase in dyspnoea due to insufficient antibiotics and prednisone; 2) urine leakage along inserted catheter. b
More than one type of contact possible per patient.
*p<0.05.
Patient satisfaction Of all 1,573 patient questionnaires sent, 650 were returned (response rate 41.3%). We included 217 questionnaires in the intervention group and 412 in the control group. On most items, the agreement percentages were higher for nurse practitioners than for general practitioners, with significant differences for confidence in expertise of care provider (nurse practitioner 95.8%; general practitioner 94.6%), usefulness of advice (nurse practitioner 95.3%; general practitioner 87.7%), information about possibilities for the patient if complaints would change after the home visit (nurse practitioner 92.9%; general practitioner 85.2%), and interest in personal situation (nurse practitioner 95.7%; general practitioner 91.9%). The mean overall grade patients gave for the care provided by nurse practitioners was 8.6 (range 6-10). This was significantly higher than for general practitioners: 8.3 (range 1-10)(Table 4). Table 4. Percentage of agreementa with items about experiences of patients with home visit by a nurse practitioner (intervention group; N=217) and general practitioner (control group; N=412) Item
Intervention group
Control group
Nurse practitioner
General
% (N)
practitioner
Total
% (N)
% (N) Acceptable waiting time
82.3 (172)
84.8 (335)
83.9 (507)
Treatment/advice had desired effect
84.5 (163)
84.4 (309)
84.4 (472)
Confidence in expertise of care provider*
95.8 (206)
94.6 (383)
95.0 (539)
Useful advice**
95.3 (184)
87.7 (320)
90.3 (504)
92.9 (168)
85.2 (283)
87.9 (451)
85.8 (145)
85.1 (251)
85.3 (396)
Shared decision making
84.8 (156)
82.1 (289)
83.0 (445)
Interest in personal situation*
95.7 (200)
91.9 (364)
89.3 (564)
Clear explanation
96.7 (208)
93.3 (380)
94.5 (588)
Possibility to ask questions
98 .1 (205)
95.7 (381)
96.6 (586)
8.6 (1.0)
8.3 (1.3)
8.4 (1.2)
8.5-8.8
8.2-8.4
8.3-8.5
Information about options if complaints would change** Treatment in correspondence with other treatments
Overall grade** Mean (SD) 95% CI
*p<0.05; **p<0.01; testing of difference between means on five-point scale. a Sum of “agree” and “totally agree”.
Discussion Main findings We evaluated the efficiency, safety, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-of-hours general practitioner cooperatives. A number of results support the claim this substitution is efficient. First, nurse practitioners could perform home visits independently, whereby they telephonically contacted the supervising general practitioner in only one in five home visits. Second, nurse practitioners prescribed less often medication and prescribed more appropriately than general practitioners, according to evidence-based guidelines.. It seemed that home visits by nurse practitioners were less complex (e.g. less urgent, often traumarelated) and this could partly explain the lower prescription rate, as more medication is described in case of complex co-morbidity (Christensen et al.; 2019). On the other hand, efficiency was reduced by the longer mean length of the home visits for nurses and higher referral rate to the hospital emergency department. Many Western countries, including the Netherlands, are already struggling with overcrowding in emergency departments (Hoot & Aronsky, 2008; van der Linden , 2013). Results on the safety of substitution of general practitioners by nurse practitioners during home visits were also generally positive. First, there were no differences in the number of missed diagnoses and complications between both professionals. The percentage of preventable complications was very low in both groups, and in accordance with the literature (Smits et al., 2010). Second, the percentage of follow-up contacts (e.g. with general practice, general practitioner cooperative, emergency department, ambulance) was similar in both groups and is comparable to that of highly urgent contacts at the general practitioner cooperative (Huibers et al., 2013). Third, nurse practitioners more often adhered to the protocol than general practitioners. This can partly be explained by the fact that nurse practitioners only visited patients with one or more of 24 eligible health problems. The nurses were trained in the corresponding protocols before the start of the study. Patient satisfaction was high, in both groups, corresponding with earlier findings (Smits et al., 2012). Patients gave nurse practitioners even higher ratings than general practitioners for expertise, usefulness of advice, information and interest. These differences might be related to the longer time nurse practitioners took for the home visits, enabling them to include these elements of care (Wilson et al., 2002).
Strengths and limitations We performed a quasi-experimental study using an intervention and control group. A strength of this study is the use of three sources of data to answer our research questions, including follow-up data. Moreover, we used an independent blind assessment of patient records by experienced general practitioners. A limitation of our study is that although we exceeded the minimum of 333 cases for the intervention group, we did not reach the minimum for the control group (N=281), because of a loss of patients who did not give informed consent. This could have led to selection bias in the control group for the protocol adherence assessment, follow-up record review and patient survey. In the follow-up study we lost patients who did give informed consent, because not all general practitioners responded to the questionnaire. The response rate among general practitioners was nonetheless good (68%). We could not perform a sample size calculation for differences in follow-up figures. Because of the low frequency of complications and missed diagnoses, our study was likely underpowered to find differences on these outcomes. The required sample size for the incidence of complications would be about 1,200 patients per group (Smits et al., 2010). This was not feasible for this study. Another limitation of the study is that we could not perform a randomised trial with random allocation of patients to either a nurse practitioner or general practitioner. Instead we used one general practitioner cooperative as the intervention setting and two general practitioner cooperatives as control settings. The three general practitioner cooperatives were located in the same region with a similar patient population and policy. Our design could have led to a difference between the intervention and control group in the complexity of the home visits. Therefore we corrected for these differences in the analyses. Furthermore, nurse practitioners with at least 20 years of experience and a specialisation in ambulance care participated. They all had at least two years of experience with clinic consultations at the general practitioner cooperative. It is not known if our results are representative for nurse practitioners with another educational background and experience. Finally, during the assessment of protocol adherence, we blinded the reviewers for type of healthcare provider performing the home visit (nurse practitioner or general practitioner). Blinding of own general practitioners in the follow-up study was not possible. When completing the questionnaire, the general practitioners may have been influenced by their own positive or negative views on substitution of care by nurse practitioners.
Comparison with literature
There are only a few previous studies into the effects of substitution of general practitioners by nurse practitioners during primary care home visits (Edwards et al., 2009; Collins, 2019). Our study is most comparable with the study of Collins, because it was performed in an out-of-hours primary care setting. Contrary to our findings, Collins (2019) did not find differences in home visit length nor in the referral rate between nurse practitioners and general practitioners. Moreover, Edwards et al. (2009) found a higher prescription rate for nurse practitioners as compared to general practitioners. Literature into substitution of general practitioners by nurses in primary care clinic consultations is less scarce (Wijers et al., 2013; Laurant et al., 2018). Consultations by nurse practitioners were found to take longer (Laurant et al., 2018), patient satisfaction to be higher (Laurant et al, 2018), and nurse practitioners were found to adhere more often to the protocols (Wijers et al., 2013). These findings are more in line with our results. There are, however, methodological issues and contextual differences that hinder a good comparison with results from previous research.
However, the general conclusions of previous studies into nurse practitioners substituting general practitioners during daytime or out-of-hours primary care (Edwards et al., 2009; Collins, 2019; Wijers et al., 2013; Laurant et al., 2018), that nurse practitioners meet patient needs, practise in a safe manner, and provide an effective service for a broad range of patient conditions, are in line with our findings.
Implications for practice and future research The results show that nurse practitioners can make a valuable contribution to primary care during urgent home visits. In countries with a need for extra workforce in primary care, substitution by nurse practitioners can be considered a solution equal to the care delivered by general practitioners. However, a careful selection should be made of nurse practitioners who are able to perform home visits and of eligible health problems. Nurse practitioners should have the option to consult a general practitioner. Earlier work experience of nurse practitioners at the general practitioner cooperative seems to be a success factor, because consequently general practitioners gain confidence in the knowledge and skills of the nurse practitioners, which are crucial for collaboration and shared responsibilities. Future research into the safety and efficiency of home visits by nurse practitioners is recommended, over a longer period of time, using nurse practitioners with different types of educational backgrounds and in more regions to increase the number of included patients and to increase the representativeness of the results. The cost-effectiveness should be studied, as nurse practitioners referred more patients to emergency department and the duration of home visits by nurse
practitioners was longer. Finally, it is interesting to examine if the substitution decreases workload for general practitioners at the general practitioner cooperative and if this leads to more time for patients needing more complex care and increased quality of care.
Conclusions The results indicate that experienced nurse practitioners can safely, efficiently and satisfactorily perform low complex out-of-hours primary care home visits. They are practicing at a level equivalent to general practitioners for these home visits. This study adds to the body of evidence for the positive impact of nurse practitioners in primary care.
References Bellow AA Jr, Gillespie GL. The evolution of ED crowding. J. Emerg. Nurs. 2014; 40: 153-160.
Berchet C, Nader C. The Organisation of Out-of-Hours Primary Care in OECD Countries. OECD Health Working Papers no. 89. Paris: OECD Pub; 2016. doi:10.1787/18152015. Accessed at www.oecdilibrary.org/social-issues-migration-health/the-organisation-of-out-of-hours-primary-care-in-oecdcountries_5jlr3czbqw23-en on 11 March 2019.
Christensen LD, Reilev M, Juul-Larsen HG, Jørgensen LM, Kaae S, Andersen O, Pottegård A, Petersen J. Use of prescription drugs in the older adult population-a nationwide pharmacoepidemiological study. Eur. J. Clin. Pharmacol. 2019 Apr 4. doi: 10.1007/s00228-019-02669-2. [Epub ahead of print]
Collins D. Assessing the effectiveness of advanced nurse practitioners undertaking home visits in an out of hours urgent primary care service in England. J. Nurs. Manag. 2019; 27: 450-458.
Coombes R. How to fix out of hours care. BMJ. 2016; 353: i235.
Dierick‐van Daele AT, Spreeuwenberg C, Derckx EW, Metsemakers JF, Vrijhoef BJ. Critical appraisal of the literature on economic evaluations of substitution of skills between professionals: a systematic literature review. J. Eval. Clin. Pract. 2008; 14: 481‐492.
Dierick‐van Daele AT, Metsemakers JF, Derckx EW, Spreeuwenberg C, Vrijhoef HJ. Nurse practitioners substituting for general practitioners: randomized controlled trial. J. Adv. Nurs. 2009; 65: 391‐401.
Edwards M, Bobb C, Robinson SI. Nurse practitioner management of acute in-hours home visit or assessment requests: a pilot study. Br. J. Gen. Pract. 2009; 59: 7-11.
Freund T, Everett C, Griffiths P, Hudon C, Naccarella L, Laurant M. Skill mix, roles and remuneration in the primary care workforce: who are the healthcare professionals in the primary care teams across the world? Int. J. Nurs. Stud. 2015; 52:727-743.
Giesen P, Smits M, Huibers L, Grol R, Wensing M. Quality of after-hours primary care in the Netherlands: a narrative review. Ann. Intern. Med. 2011; 155: 108-113.
Halfon P, Eggli Y, van Melle G, Chevalier J, Wasserfallen JB, Burnand B. Measuring potentially avoidable hospital readmissions. J. Clin. Epidemiol. 2002; 55: 573-587.
Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann. Emerg. Med. 2008; 52: 126–36.1.
Huibers L, Giesen P, Wensing M, Grol R. Out-of-hours care in western countries: assessment of different organizational models. BMC. Health Serv. Res. 2009; 9: 105.
Huibers L, Thijssen W, Koetsenruijter J, Giesen P, Grol R, Wensing M. GP cooperative and emergency department: an exploration of patient flows. J. Eval. Clin. Pract. 2013; 19: 243-249.
Huibers L, Philips H, Giesen P, Remmen R, Christensen MB, Bondevik GT. EurOOHnet—the European research network for out-ofhours primary health care. Eur. J. Gen. Pract. 2014; 20: 229-232.
Keizer E, Oude Bos A, Giesen P, Smits M. [Development of the PREM for general practitioner cooperatives]. Nijmegen: IQ healthcare, Radboudumc; 2017.
Kooienga SA, Carryer JB. Globalization and advancing primary health care nurse practitioner practice. Journal for Nurse Practitioners 2015;11:804–11.
Laurant M, van der Biezen M, Wijers N, Watananirun K, Kontopantelis E, van Vught AJ. Nurses as substitutes for doctors in primary care. Cochrane Database Syst. Rev. 2018; 7: CD001271.
Leibowitz R, Day S, Dunt D. A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam. Pract. 2003; 20: 311-317.
Maier CB, Barnes H, Aiken LH, Busse R. Descriptive, cross-country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential. BMJ Open. 2016; 6: e011901.
Martinez‐Gonzalez NA, Djalali S, Tandjung R, Huber-Geismann F, Markun S, Wnesing M, Rosemann T. Substitution of physicians by nurses in primary care: a systematic review and meta‐analysis. BMC Health. Serv. Res. 2014; 14: 214.
Moskop JC, Sklar DP, Geiderman JM, Schears RM, Bookman KJ. Emergency department crowding, part 1—concept, causes, and moral consequences. Ann. Emerg. Med. 2009; 53: 605-611.
Nederlands Huisartsen Genootschap (NHG). [NHG Standards]. NHG: Utrecht; 2019 https://www.nhg.org/nhg-standaarden (accessed 11 March 2019). Rutten M, Vrielink F, Smits M, Giesen P. Patient and care characteristics of self-referrals treated by the general practitioner cooperative at emergency-care-access-points in the Netherlands. BMC Fam. Pract. 2017; 18: 62.
Smits M, Huibers L, Kerssemeijer B, de Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: a review of patient records. BMC Health Serv. Res. 2010; 10: 335.
Smits M, Huibers L, Oude Bos A, Giesen P. Patient satisfaction with out-of-hours GP cooperatives: a longitudinal study. Scand. J. Prim. Health Care. 2012; 30: 206-213.
Smits M, Keizer E, Huibers L, Giesen P. GPs' experiences with out‐of‐hours GP cooperatives: a survey study from the Netherlands. Eur. J. Gen. Pract. 2014; 20: 196‐201.
Smits M, Rutten M, Keizer E, Wensing M, Westert G, Giesen P. The development and performance of after-hours primary care in the Netherlands: a narrative review. Ann. Intern. Med. 2017; 166: 737742.
van der Biezen M, Schoonhoven L, Wijers N, van der Burgt R, Wensing M, Laurant M. Substitution of general practitioners by nurse practitioners in out‐of‐hours primary care: a quasi‐experimental study. J. Adv. Nurs. 2016; 72: 1813‐1824.
van der Biezen M, Wensing M, van der Burgt R, Laurant M. Towards an optimal composition of general practitioners and nurse practitioners in out-of-hours primary care teams: a quasiexperimental study. BMJ Open 2017; 7: e015509.
van der Linden C, Reijnen R, Derlet RW, Lindeboom R, van der Linden N, Richards JR. Emergency department crowding in the Netherlands: managers’ experiences. Int. J. Emerg. Med. 2013; 6: 41.
Wijers N, van den Burgt R, Laurant M. [Nurse practitioner offers opportunities. Research report on the deployment of nurse practitioners at the Emergency Care Access Point in Eindhoven]. Eindhoven. Nijmegen/Eindhoven: IQ healthcare/Stichting KOH, 2013.
Willekens M, Giesen P, Plat E, Mokkink H, Burgers J, Grol R. Quality of after-hours primary care in the Netherlands: adherence to national guidelines. BMJ Qual. Saf. 2011; 20: 223-227.
Wilson A, Childs S. The relationship between consultation length, process and outcomes in general practice: a systematic review. Br. J. Gen. Pract. 2002; 52: 1012-1020.
Declarations Ethics approval and consent to participate The Ethical Research Committee of the Radboud university medical center Nijmegen was consulted and concluded that this study does not fall within the remit of the Dutch Medical Research Involving Human Subjects Act [Wet Mensgebonden Onderzoek] (file number 2016-2668). Only patients who signed an informed consent form were included in the non-anonymous part of the study.
Consent for publication Not applicable.
Availability of data and material The dataset used during the current study is available from the corresponding author on reasonable request.
Conflict of interests None.
Funding This work was financially supported by ZonMw, the Netherlands.
Authors' contributions MS designed the study, coordinated the data collection, performed the statistical analyses, interpreted the data and drafted the manuscript. YP and SR collected the data, analysed and interpreted the data and revised the manuscript critically for important intellectual content. EP participated in the interpretation of the data and revised the manuscript critically for important intellectual content. ML and PG participated in the conception and design of the study and interpretation of the data, and revised the manuscript critically for important intellectual content. All authors read and approved the final manuscript.
Acknowledgements We thank all participants in the study.
Figures
Figure 1. Flow chart of patients included in the different parts of the study
Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study Marleen Smits , Yvonne Peters , Sander Ranke , Erik Plat , Miranda Laurant , Paul Giesen PII: DOI: Reference:
S0020-7489(19)30252-4 https://doi.org/10.1016/j.ijnurstu.2019.103445 NS 103445
To appear in:
International Journal of Nursing Studies
Received date: Revised date: Accepted date:
19 March 2019 20 September 2019 24 September 2019
Please cite this article as: Marleen Smits , Yvonne Peters , Sander Ranke , Erik Plat , Miranda Laurant , Paul Giesen , Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study, International Journal of Nursing Studies (2019), doi: https://doi.org/10.1016/j.ijnurstu.2019.103445
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier Ltd.
Substitution of general practitioners by nurse practitioners in out-of-hours primary care home visits: a quasi-experimental study Marleen Smits, PhD1*; Yvonne Peters, Msc1; Sander Ranke1, Msc; Erik Plat1, MD, Msc; Miranda Laurant, PhD1,2; Paul Giesen, MD, PhD1 1 Radboud
university medical center, Radboud Institute for Health Sciences, Scientific Center for
Quality of Healthcare (IQ healthcare), Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands 2 HAN
University of Applied Sciences, Faculty of Health and Social Studies, Kapittelweg 33, 6525
EN Nijmegen, The Netherlands *Corresponding author: Dr. Marleen Smits Radboud university medical center P.O Box 9101, 114 IQ healthcare 6500 HB Nijmegen The Netherlands [email protected] #31 (0) 24 3666264 Email addresses: MS: [email protected] YP: [email protected] SR: [email protected] EP: [email protected] ML: [email protected] PG: [email protected] Running title: Substitution out-of-hours primary care home visits Word count: 4429 (6383 including tables, figures and references)
Abstract Background: General practitioners experience a high workload during out-of-hours care. A possible solution is the shifting of care to nurse practitioners. Objectives: To provide insight into patient- and care characteristics, safety, efficiency, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-ofhours primary care services. Design: Quasi-experimental non-randomised study comparing home visits by nurse practitioners (intervention group; one out-of-hours care service) with home visits by general practitioners (control group; two out-of-hours care services) for 24 protocolised health problems. Setting: Three out-of-hours primary care services in the East of the Netherlands. Participants: 1,601 patients who received a home visit by a nurse practitioner (N=386) or a general practitioner (N=1,215). Of these patients, 639 gave informed consent to be included in the protocol adherence assessment and follow-up record review (nurse practitioner: N=358; general practitioner: N=281). Methods: Five nurse practitioners with experience in ambulance care were recruited and trained. From September 2016 to March 2017 the nurse practitioners took over home visits under supervision of a general practitioners. This was evaluated using: (1) data-extraction from the patient registration system, (2) follow-up record review in the patients’ general practices, and (3) patient satisfaction survey. Two general practitioners independently assessed protocol adherence based on the extracted registration data. Results: Nurse practitioners prescribed medication significantly less often than general practitioners (19.9% versus 30.6%), and referred patients significantly more often to the hospital (24.1% versus 15.9%). The mean length of the home visit was significantly longer for nurse practitioners (34.1 versus 21.1 minutes). Nurse practitioners adhered to the protocol significantly more often than general practitioners (84.9% versus 76.2%) and their medication prescribing was significantly more often appropriate (93.7% versus 79.5%). There were no differences in the number of missed diagnoses and complications. The number of follow-up contacts was also similar in both groups. Patient satisfaction was generally high and significantly higher for nurse practitioners on several items. Conclusions: Nurse practitioners with experience in ambulance care can safely, efficiently, and satisfactorily perform low complex out-of-hours primary care home visits. It is recommended to study the safety and efficiency of nurse practitioners’ home visits in other regions and with nurse
practitioners with different educational levels and different specialisations. In addition, we recommend to evaluate the cost-effectiveness and if it leads increased quality of care.
Keywords (MeSH): Emergency Care, General Practitioners, Nurses, Out-of-Hours Medical Care, Practitioner Cooperative, Primary Health Care
Contribution of the paper What is already known about the topic?
General practitioners experience a high workload during out-of-hours primary care.
Previous studies into substitution of general practitioners by nurse practitioners during clinic consultations, both in daytime general practices and in out-of-hours primary care services, have shown positive results in terms of workload for general practitioners, patient experiences and care outcomes.
Little is known on the effects of substitution during out-of-hours primary care home visits.
What this paper adds?
This study demonstrates that nurse practitioners, compared to general practitioners, more often adhered to protocols, less often prescribed medication, and obtained higher patient satisfaction during out-of-hours primary care home visits, but the mean duration of the home visit was longer and they more often referred patients to hospital.
The study shows that the nurse practitioners were equivalent to general practitioners in terms of the number of follow-up contacts, missed diagnoses, and complications.
Introduction In many Western countries, hospital emergency departments are overcrowded, leading to the desire to strengthen primary care, particularly out-of-hours (Moskop et al., 2009; Bellow and Gillespie, 2014). To make the out-of-hours primary care system stronger, many Western nations have reorganised it from individual general practices and rota groups (small groups of GP’s who are on duty alternately) into large-scale General Practitioner Cooperatives (Berchet and Nader, 2016). In these primary care services a large number of general practitioners take turns being on duty during out-of-hours for the patient population of all participating general practitioners. The services are intented for primary care problems that cannot wait until the regular consultation hours of the patient’s own general practitioner. General practitioners in the cooperatives may be supported by nurses, management, drivers etc. Reasons for this restructuring included high workload and reluctance by general practitioners to commit to being on call 24/7, regional shortages of general practitioners, and increasing patient demand for out-of-hours care (Leibowitz et al., 2003; Huibers et al., 2009). The challenge for policymakers is to find a suitable model for out-of-hours care and continuously make adjustments as a reaction to the changing context to ensure accessibility, safety, efficiency, and the satisfaction of both patients and health care professionals (Huibers et al., 2014, Coombes, 2016). Around the year 2000, general practitioners in the Netherlands reorganised their out-of-hours care into large-scaled general practitioner cooperatives, thereby taking their responsibility to provide primary care 24/7 (Giesen et al., 2011; Smits et al., 2017)(Table 1 Online supplementary file). A patient contacting a Dutch cooperative receives a telephone advice, clinic consultation or home visit. The use of these cooperatives has increased over the years. General practitioners now experience a high workload, which could have a negative impact on the quality of care and their motivation to perform out-of-hours shifts (Smits et al., 2014). A range of developments possibly increase workload. These include societal, economic and demographic changes, such as a decreased number of emergency departments, patient flows from hospital to the general practitioner, aging of the population and increased co-morbidity, and staff shortages (Smits et al., 2017). As many health problems in out-of-hours care do not necessarily require the knowledge and skills of a general practitioner, using trained nurses to expand the capacity of the primary care workforce is a topic of increasing interest (Kooienga and Carryer, 2015; Maier et al., 2016; Laurant et al., 2018). A shift of care from general practitioners to advanced practice nursing roles, is one strategy for improving access to care and improve efficiency and quality of care . Previous studies into substitution of general practitioners by nurse practitioners, mostly during clinic consultations in
daytime general practices, have shown positive results in terms of workload for general practitioners, patient experiences and care outcomes (Dierick-van Dale et al., 2009; van der Biezen et al., 2016, 2017; Laurant et al., 2018; Collins, 2019). In addition, there are indications that costs savings can be realised, although further research is recommended on this point (Dierick-van Daele et al., 2008; Martinez-Gonzales et al., 2014; Laurant et al., 2018). A study in one out-of-hours primary care service showed that nurse practitioners treat similar patients as the general practitioner and can do about 77% of all clinic consultations autonomously without consulting a general practitioner (van der Biezen et al., 2016). There are only a few reported studies on the effects of substitution of general practitioners by nurse practitioners during home visits (Edwards et al., 2009; Collins, 2019), and only one of them in out-of-hours (Collins, 2019). The findings of these studies support the indication that nurse practitioners are competent and achieve equivalent clinical outcomes compared to general practitioners, but the studies had some methodological limitations. The expectation is that when experienced and trained nurse practitioners take over low-complex protocolised home visits in outof-hours primary care, this will reduce the workload of general practitioners, while the quality of care and patient satisfaction remains the same. Consequently, general practitioners will have more time for high-complex patient problems, both during home visits and clinic consultations. Out-of-hours home visits of general practitioner cooperatives are regarded as a high-risk primary care setting due to the fact that patients (mostly elderly) are assessed at relatively high urgency and with little diagnostic support in the home setting. Therefore, it is important to evaluate the safety of substitution general practitioners by nurse practitioners in this setting. Our aim was to provide insight into the safety, efficiency, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-of-hours primary care general practitioner cooperatives.
Materials and Methods Design and setting A quasi-experimental non-randomised study was conducted. Nurse practitioners substituted general practitioners to carry out out-of-hours home visits for 24 predefined low complex health problems at one general practitioner cooperative in the east of the Netherlands (intervention group). The control group consisted of two other general practitioner cooperatives where general practitioners provided care as usual. The three general practitioner cooperatives were located in the same region with a similar patient population and policy.
We used three sources of data: (1) data-extraction from the patient registration system of the general practitioner cooperative, (2) follow-up record review by the patient’s own general practitioner, and (3) patient experiences survey. The intervention period was seven months, from September 2016 to March 2017.
Patients Patients who received a home visit for one of the 24 predefined health problems (see Table 2 Online supplementary file) were included in the study . Home visits at night were excluded from the study.
Intervention Five registered nurse practitioners were recruited for the study. They were all experienced as ambulance nurses before they entered the Master of Nurse Practitioner (MANP). In the Netherlands, the MANP is a 2 year master’s programme at universities of applied sciences. The education consists of a dual work-education model, meaning that students are employed within a practice (including general practices) and receive salary. About two-third of the education takes place on the job, learning by doing (Freund et al., 2015). The nurse practitioners were all male and had more than 20 years of experience as ambulance nurses and had two to six years of experience with clinic consultations at a general practitioner cooperative. They substituted general practitioners when visiting patients at home during weekday evenings (5:00 PM‐11:30 PM) and weekends (10:00 AM‐6:00 PM). They were supervised by a general practitioner of the general practitioner cooperative. It was decided that the nurse practitioners could take care of 24 low-complex health problems which could be treated according to a protocol. The triage professionals (often trained practice nurses) and supervising general practitioner of the cooperative assigned patients with one of the 24 eligible health problems to the nurse practitioners after telephone triage. In case of a high-risk health problem or U0 call (resuscitation), a general practitioner performed the home visit or an ambulance was called. The 24 eligible health problems presented at triage were: arm complaints, leg complaints, insult, nosebleed, extremity trauma, neck trauma, back trauma, skull trauma, thorax trauma, face trauma, abdominal trauma, burn/inhalation trauma, intoxication, shortness of breath, palpitations, neurological failure, disrupted diabetes, thorax pain, back pain, urinary tract problems, wound, fainting, dizziness, and headache (Table 2 Online supplementary file). For each of these health problems a protocol was written, based on the national general practitioner guidelines (NHG, 2019), and before the intervention the nurse practitioners received a training in these protocols, offered by the ambulance service and general practitioner cooperative. The general practitioners were used to working with the national guidelines
during daytime and out-of-hours primary care and therefore did not receive any additional training. The nurse practitioners used a car ambulance for the home visit and the general practitioners used the usual general practitioner cooperative car. General practitioners had a chauffeur who could give them medical assistance, if necessary. Nurse practitioners had no chauffeur. Both cars had no possibility to transport patients and the equipment was generally equal (e.g. oxygen, intravenous drip equipment, automated external defibrillator, and medication for acute treatment). Additionally, the car of the nurse practitioners had electrocardiogram equipment, which was not available for general practitioners. Data from the first month of the intervention period were not used in the study, to give the nurse practitioners time to get familiar with performing home visits. Hence, the study period was six months.
Informed consent In both groups patients were asked to confirm participation in non-anonymous parts of the study (protocol adherence assessment and follow-up record review study) and in the patient survey. The procedure to obtain informed consent differed between the groups. In the intervention group, the nurse practitioners obtained informed consent from each patient at the end of the home visit. Patients could give back the signed form directly to the nurse practitioners or they could send it to the research institute in a prepaid return envelope. In the control group, patients received the informed consent form from the researchers by post within one week after the home visit. They could return the informed consent form with a prepaid return envelope to the research institute.
Data collection and outcomes Data extraction During the intervention period anonymous routinely obtained data were extracted from the electronic registration systems of the three general practitioner cooperatives and included the following patient characteristics: age, gender, date home visit, health problem presented during triage, urgency level during triage (U1-U5), medication prescription, ambulance callout, referral to emergency department, International Classification Primary Care code, chronic disease, and free text SOAP note (Subjective, Objective, Assessment, Plan). Nurse practitioners recorded in the electronic registration system if they had contacted the general practitioner from the general practitioner cooperative. The following actions could be reported: telephone consultation with the supervising
general practitioner with no further action or followed by a home visit by a general practitioner (second opinion). These data were also extracted.
Assessment of protocol adherence Two experienced general practitioners independently assessed protocol adherence based on all extracted patient registration data, including free text SOAP notes, following a procedure used in earlier research (Willekens et al., 2011). These reviewers used the 24 protocols as guiding principle. Adequate care was defined as following the protocol or reasoned deviating from the protocol. For cases in which the initial assessment of the reviewers differed, a consensus procedure followed. The reviewers were first asked if they wanted to alter their assessment to conform with the other reviewer. Cases with persistent differences were discussed in a face-to-meeting of the reviewers, together with an expert general practitioner. The expert general practitioner could make a final decision if necessary.
Follow-up record review After the intervention period, a follow-up record review study was performed in patients’ own general practices. The general practitioners received a postal questionnaire for each patient with questions regarding changes in diagnosis or policy, complications, and follow-up contacts within the same disease episode as the index home visit. In case of a complication, general practitioners were asked to give a description. The descriptions of missed diagnoses and complications were assessed by the researchers to determine their preventability and severity. The follow-up duration was one month from the general practitioner cooperative home visit. Research into hospital readmissions has shown that this is the optimal length of the follow-up period (Halfon et al., 2002). General practitioners were asked to send the questionnaires back to the research institute in a prepaid return envelope. If there was no response, a reminder was sent after 14 days. After the first reminder, general practitioners who had more than six patients included in the study were telephonically reminded and were offered assistance. Some general practitioners were visited by one of the researchers who gathered the required data from the patient records.
Patient experiences survey A Patient Reported Experience Measure (PREM) questionnaire developed and validated for general practitioner cooperatives was used to measure patient experiences (Keizer et al., 2017). Patients were asked to respond to statements about their experiences with the care provider during the
home visit. Statements were about communication, patient-centeredness, expertise of care provider, shared decision making, continuity of care, usefulness of advice, maintenance of care, result of care, and waiting time. The answering scale was a five-point Likert scale ranging from totally disagree to totally agree. In addition, patients were asked to give an overall rating of the care provider using a school grade from 1 to 10. Patients received the questionnaire by postal mail within one week after the home visit. They were asked to send the questionnaire back to the research institute in a prepaid return envelope. If there was no response, a reminder was sent after 14 days.
Sample size To calculate the sample size needed, we used a non-inferiority approach. To examine if protocol adherence of nurse practitioners was equal to general practitioners a total number of 333 patients were needed in both the intervention and control group. For this calculation we estimated the mean protocol adherence at 77% (Willekens et al., 2011) and we used a non-inferiority margin of 10%, a power of 80% and a 95% two-sided confidence interval. Based on the incidence of the 24 health problems, we estimated that a study period of six months would yield enough home visits by nurse practitioners. We used two general practitioner cooperatives as control settings, because it was estimated that the loss of patients in the control group would be higher, because patients received the informed consent form by post without faceto-face information about the study. Blinding The assessment of protocol adherence was blinded: the general practitioner reviewers did not know who performed the home visit. It was not possible to blind the patients to study condition. In the follow-up study in the patients’ own general practices, it was also not possible to blind the general practitioners.
Statistical analyses Home visits were the unit of analysis. Patient- and care characteristics, patient experiences, and follow-up data were analysed using descriptive statistics. The duration of the home visit was calculated by subtracting the start time from the end time of the home visit as recorded in the electronic patient file. Home visits with a missing time (N=53) and home visits with an unlikely duration (below five minutes or above two hours; N=66) were excluded in the calculation of the mean home visit duration. For the items on patient experiences, the percentages of positive scores (“agree” and “totally agree”) were calculated.
The total intervention group and control group were compared on several background items, using a T-test for age and Chi-square tests for gender (male/female), urgency (U1-U5) and time of home visit (week/weekend). To compare medication prescription and referral between both groups, we used logistical regression analyses, correcting for differences in urgency level, time of home visit, patients with shortness of breath and trauma. To compare the duration of the home visit, we used linear regression analyses, correcting for urgency level, time of home visit, patients with shortness of breath and trauma. For the subset of patients with informed consent, the analyses on background items (age, gender, urgency, time of home visit and health problem presented) were repeated to check for possible covariates. In the comparative analyses regarding protocol adherence and follow-up we used logistical regression analyses, correcting for urgency level, patients with trauma, back pain, headache and diabetes. Patient experiences from the intervention and control group were compared using Ttests on the total five-point scale. Questionnaires with a missing response on >5 questions were excluded from the analyses. All statistical analyses were carried out using IBM SPSS software version 25.
Results
Patient characteristics A total of 1,601 patients from the three general practitioner cooperatives received a home visit: 386 were visited by a nurse practitioner and 1,215 by a general practitioner (Figure 1). Table 1 shows patient and care characteristics of both groups. Most patients were triaged at urgency level U2 (Acute) and U3 (Urgent). Patients that were visited by a nurse practitioner, had health problems with significantly lower urgency levels than patients that were visited by a general practitioner. The most frequently presented health problem was shortness of breath in both groups, although significantly more frequent in home visits by general practitioners (32.2%) than by nurse practitioners (26.2%). Traumas were significantly more frequent in home visits by nurse practitioners (11.1%) than by general practitioners (6.4%). None of the other health problems differed significantly (Table 1).
Care characteristics Nurse practitioners prescribed significantly less often medication than general practitioners (19.9% nurse practitioner versus 30.6% general practitioner), and significantly more often referred the patient to the hospital emergency department (24.1% nurse practitioner versus 15.9% general practitioner). In addition, the mean duration of the home visit was significantly longer for nurse practitioners (34.1 minutes) than for general practitioners (21.1 minutes)(Table 1). The nurse practitioners consulted the supervising general practitioner in 21.5% of all home visits. In one case this resulted in a visit of a general practitioner to the patient home (not in table).
Table 1. Patient and care characteristics of patients who were visited by a nurse practitioner (intervention group; N=386) and general practitioner (control group; N=1,215) Characteristic
Intervention group
Control group
Nurse practitioner
General practitioner
Total
% (N)
% (N)
% (N)
73.0 (15.1)
74.6 (16.0)
74.2 (15.8)
95% CI
71.4-74.5
73.7-75.5
73.4-75.0
Gender male
45.6 (176)
43.3 (526)
43.8 (702)
0.3 (1)
3.0 (36)
2.3 (37)
U2 – Acute
48.2 (186)
53.1 (645)
51.9 (831)
U3 – Urgent
47.4 (183)
41.8 (508)
43.2 (691)
U4 – Routine
2.8 (11)
1.8 (22)
2.1 (33)
U5 – Advice
1.3 (5)
0.3 (4)
0.6 (9)
26.2 (101)
32.2 (392)
30.8 (493)
11.1 (43)
6.4 (78)
7.6 (121)
Thorax pain
8.3 (32)
10.0 (122)
9.6 (154)
Dizziness
6.5 (25)
5.8 (71)
6.0 (96)
Leg complaints
6.0 (23)
6.1 (74)
6.1 (97)
Urinary tract problems
6.0 (23)
5.9 (72)
5.9 (95)
Wound
5.7 (22)
4.1 (50)
4.5 (72)
Neurological failure
5.2 (20)
7.0 (85)
6.6 (105)
Back pain
4.7 (18)
6.5 (79)
6.1 (97)
Age (years) mean (SD)
Urgency level** U1 – Life-threatening
Health problem presented Shortness of breath* Trauma (8 health problems)**
Palpitations
3.4 (13)
2.7 (33)
2.9 (46)
Syncope
2.1 (8)
3.5 (43)
3.2 (51)
Headache
1.6 (6)
3.0 (36)
2.6 (42)
Disrupted diabetes
1.3 (5)
1.8 (22)
1.7 (27)
Nosebleed
1.0 (4)
2.5 (30)
2.1 (34)
Arm complaints
0.8 (3)
1.1 (13)
1.0 (16)
Intoxication
0.5 (2)
0.3 (4)
0.4 (6)
Insult
0.3 (1)
0.9 (11)
0.7 (12)
9.8 (37)
-
2.3 (37)
Medication prescription**a
19.9 (77)
30.6 (372)
28.0 (449)
Referral*** a
24.1 (93)
15.9 (193)
17.9 (286)
34.1 (13.8)
21.1 (10.4)
24.0 (12.5)
31.9-34.8
20.4-21.6
23.2-24.5
Other/ not known
Duration home visit (minutes) *** a mean (SD) 95% CI a
Corrected for urgency level, time of home visit, shortness of breath and trauma.
*p<0.05; **p<0.01; ***p<0.001.
Protocol adherence To study protocol adherence, 639 patients with signed informed consent forms, were included in the analyses: 358 nurse practitioner visits and 281 general practitioner visits (Figure 1). Nurse practitioners significantly more often adhered to the protocol than general practitioners (84.9% nurse practitioner versus 76.2% general practitioner). A reasoned deviation from the protocol was also seen as adequate care (2.0% nurse practitioner versus 5.7% general practitioners, not significant), but incorrect deviation of the protocol could be harmful to the patient (13.1% nurse practitioner versus 18.1% general practitioners, not significant). Prescription of medication was the most frequent incorrect deviation from the protocol (51%), but did not significantly differ between professionals (Table 2).
Table 2. Protocol adherence, reasoned deviation, incorrect deviation and reason for incorrect deviation by a nurse practitioner (intervention group; N=358) and general practitioner (control group N=281) Item
Intervention group
Control group
Nurse practitioner
General practitioner
% (N)
% (N)
% (N)
84.9 (304)
76.2 (214)
81.1 (518)
2.0 (7)
5.7 (16)
3.6 (23)
13.1 (47)
18.1 (51)
15.3 (98)
Incorrect medication
51.1 (24)
51.0 (26)
51.0 (50)
Incorrect diagnostics
12.8 (6)
7.8 (4)
10.2 (10)
Incorrect referral
6.4 (3)
3.9 (2)
5.1 (5)
Other
21.3 (10)
19.6 (10)
20.4 (20)
Combination of reasons
8.5 (4)
17.7 (9)
13.3 (13)
Protocol adherence* Reasoned deviation Incorrect deviation
Total
Reason for incorrect deviationa
All analyses corrected for urgency level, trauma, back pain, headache and diabetes. *p<0.01 a
Percentage of patients with incorrect deviations.
Follow-up Of 620 patients who gave their informed consent to follow-up with the own general practitioner, the postal address of the general practices was known. A total of 420 forms were returned by the general practitioners (response rate 67.7%) (Figure 1). There were no significant differences between the groups in the frequency of missed diagnoses and complications reported by the patients’ own general practitioners. Based on the descriptions of these cases the researchers assessed that in each group two complications were preventable. The severity of these preventable complications was somewhat higher for home visits by nurse practitioners (see descriptions in footnote of Table 3). There were neither significant differences in the frequency of follow-up contacts within 72 hours nor within one month after the home visit. There were also no significant differences in types of followup contacts <72 hours. Protocol adherence (99.5%) as rated by the patients’ own general practitioner, was significantly higher for nurse practitioners (99.5%) than for general practitioners (92.3%). Appropriate referral and continuity of care as rated by the patients’ own general practitioner was generally equal between both groups. Appropriate medication prescribing (93.7%) was rated significantly higher for nurse practitioners than for general practitioners (79.5%) (Table 3).
Table 3. Assessment of care during home visit by a nurse practitioner (intervention group; N=231) and general practitioner (control group; N=189) by patients own general practitioner Item
Intervention group
Control group
Nurse practitioner
General
% (N)
practitioner
Total
% (N)
% (N) Missed diagnosis
8.5 (19)
6.0 (11)
7.4 (30)
11.0 (25)
9.0 (17)
10.1 (42)
0.9 (2)
1.1 (2)
1.0 (4)
Follow-up contact < 1 month
63.2 (146)
64.0 (121)
63.6 (267)
Follow-up contact <72 hours; contact typeb:
51.5 (119)
56.1 (106)
53.6 (225)
Own general practitioner
34.2 (79)
41.3 (78)
37.4 (157)
General practitioner cooperative
6.9 (16)
6.9 (13)
6.9 (29)
Emergency department
9.1 (21)
4.8 (9)
7.1 (30)
National emergency number (112)
0.9 (2)
0.0 (0)
0.5 (2)
Outpatient hospital service
1.7 (4)
3.2 (6)
2.4 (10)
Hospitalisation
12.1 (28)
16.9 (32)
14.3 (60)
0.4 (1)
1.6 (3)
1.0 (4)
0.9 (2)
0.5 (1)
0.7 (3)
Protocol adherence*
99.5 (192)
92.3 (144)
96.3 (336)
Appropriate referral
85.5 (53)
95.6 (65)
90.8 (118)
98.4 (181)
99.3 (152)
98.8 (333)
93.7 (74)
79.5 (66)
86.4 (140)
Complication Preventable complicationa
Nursing home/Residential care home/ Hospice Other
Continuity of care Appropriate prescribing*
All analyses corrected for urgency level, trauma, back pain, headache and diabetes. a
Nurse practitioner: 1) perforated appendicitis; 2) crisis situation related to secondary dislocated
medial collum fracture. General practitioner: 1) increase in dyspnoea due to insufficient antibiotics and prednisone; 2) urine leakage along inserted catheter. b
More than one type of contact possible per patient.
*p<0.05.
Patient satisfaction Of all 1,573 patient questionnaires sent, 650 were returned (response rate 41.3%). We included 217 questionnaires in the intervention group and 412 in the control group. On most items, the agreement percentages were higher for nurse practitioners than for general practitioners, with significant differences for confidence in expertise of care provider (nurse practitioner 95.8%; general practitioner 94.6%), usefulness of advice (nurse practitioner 95.3%; general practitioner 87.7%), information about possibilities for the patient if complaints would change after the home visit (nurse practitioner 92.9%; general practitioner 85.2%), and interest in personal situation (nurse practitioner 95.7%; general practitioner 91.9%). The mean overall grade patients gave for the care provided by nurse practitioners was 8.6 (range 6-10). This was significantly higher than for general practitioners: 8.3 (range 1-10)(Table 4). Table 4. Percentage of agreementa with items about experiences of patients with home visit by a nurse practitioner (intervention group; N=217) and general practitioner (control group; N=412) Item
Intervention group
Control group
Nurse practitioner
General
% (N)
practitioner
Total
% (N)
% (N) Acceptable waiting time
82.3 (172)
84.8 (335)
83.9 (507)
Treatment/advice had desired effect
84.5 (163)
84.4 (309)
84.4 (472)
Confidence in expertise of care provider*
95.8 (206)
94.6 (383)
95.0 (539)
Useful advice**
95.3 (184)
87.7 (320)
90.3 (504)
92.9 (168)
85.2 (283)
87.9 (451)
85.8 (145)
85.1 (251)
85.3 (396)
Shared decision making
84.8 (156)
82.1 (289)
83.0 (445)
Interest in personal situation*
95.7 (200)
91.9 (364)
89.3 (564)
Clear explanation
96.7 (208)
93.3 (380)
94.5 (588)
Possibility to ask questions
98 .1 (205)
95.7 (381)
96.6 (586)
8.6 (1.0)
8.3 (1.3)
8.4 (1.2)
8.5-8.8
8.2-8.4
8.3-8.5
Information about options if complaints would change** Treatment in correspondence with other treatments
Overall grade** Mean (SD) 95% CI
*p<0.05; **p<0.01; testing of difference between means on five-point scale. a Sum of “agree” and “totally agree”.
Discussion Main findings We evaluated the efficiency, safety, and patient satisfaction of substituting general practitioners with nurse practitioners for home visits by out-of-hours general practitioner cooperatives. A number of results support the claim this substitution is efficient. First, nurse practitioners could perform home visits independently, whereby they telephonically contacted the supervising general practitioner in only one in five home visits. Second, nurse practitioners prescribed less often medication and prescribed more appropriately than general practitioners, according to evidence-based guidelines.. It seemed that home visits by nurse practitioners were less complex (e.g. less urgent, often traumarelated) and this could partly explain the lower prescription rate, as more medication is described in case of complex co-morbidity (Christensen et al.; 2019). On the other hand, efficiency was reduced by the longer mean length of the home visits for nurses and higher referral rate to the hospital emergency department. Many Western countries, including the Netherlands, are already struggling with overcrowding in emergency departments (Hoot & Aronsky, 2008; van der Linden , 2013). Results on the safety of substitution of general practitioners by nurse practitioners during home visits were also generally positive. First, there were no differences in the number of missed diagnoses and complications between both professionals. The percentage of preventable complications was very low in both groups, and in accordance with the literature (Smits et al., 2010). Second, the percentage of follow-up contacts (e.g. with general practice, general practitioner cooperative, emergency department, ambulance) was similar in both groups and is comparable to that of highly urgent contacts at the general practitioner cooperative (Huibers et al., 2013). Third, nurse practitioners more often adhered to the protocol than general practitioners. This can partly be explained by the fact that nurse practitioners only visited patients with one or more of 24 eligible health problems. The nurses were trained in the corresponding protocols before the start of the study. Patient satisfaction was high, in both groups, corresponding with earlier findings (Smits et al., 2012). Patients gave nurse practitioners even higher ratings than general practitioners for expertise, usefulness of advice, information and interest. These differences might be related to the longer time nurse practitioners took for the home visits, enabling them to include these elements of care (Wilson et al., 2002).
Strengths and limitations We performed a quasi-experimental study using an intervention and control group. A strength of this study is the use of three sources of data to answer our research questions, including follow-up data. Moreover, we used an independent blind assessment of patient records by experienced general practitioners. A limitation of our study is that although we exceeded the minimum of 333 cases for the intervention group, we did not reach the minimum for the control group (N=281), because of a loss of patients who did not give informed consent. This could have led to selection bias in the control group for the protocol adherence assessment, follow-up record review and patient survey. In the follow-up study we lost patients who did give informed consent, because not all general practitioners responded to the questionnaire. The response rate among general practitioners was nonetheless good (68%). We could not perform a sample size calculation for differences in follow-up figures. Because of the low frequency of complications and missed diagnoses, our study was likely underpowered to find differences on these outcomes. The required sample size for the incidence of complications would be about 1,200 patients per group (Smits et al., 2010). This was not feasible for this study. Another limitation of the study is that we could not perform a randomised trial with random allocation of patients to either a nurse practitioner or general practitioner. Instead we used one general practitioner cooperative as the intervention setting and two general practitioner cooperatives as control settings. The three general practitioner cooperatives were located in the same region with a similar patient population and policy. Our design could have led to a difference between the intervention and control group in the complexity of the home visits. Therefore we corrected for these differences in the analyses. Furthermore, nurse practitioners with at least 20 years of experience and a specialisation in ambulance care participated. They all had at least two years of experience with clinic consultations at the general practitioner cooperative. It is not known if our results are representative for nurse practitioners with another educational background and experience. Finally, during the assessment of protocol adherence, we blinded the reviewers for type of healthcare provider performing the home visit (nurse practitioner or general practitioner). Blinding of own general practitioners in the follow-up study was not possible. When completing the questionnaire, the general practitioners may have been influenced by their own positive or negative views on substitution of care by nurse practitioners.
Comparison with literature
There are only a few previous studies into the effects of substitution of general practitioners by nurse practitioners during primary care home visits (Edwards et al., 2009; Collins, 2019). Our study is most comparable with the study of Collins, because it was performed in an out-of-hours primary care setting. Contrary to our findings, Collins (2019) did not find differences in home visit length nor in the referral rate between nurse practitioners and general practitioners. Moreover, Edwards et al. (2009) found a higher prescription rate for nurse practitioners as compared to general practitioners. Literature into substitution of general practitioners by nurses in primary care clinic consultations is less scarce (Wijers et al., 2013; Laurant et al., 2018). Consultations by nurse practitioners were found to take longer (Laurant et al., 2018), patient satisfaction to be higher (Laurant et al, 2018), and nurse practitioners were found to adhere more often to the protocols (Wijers et al., 2013). These findings are more in line with our results. There are, however, methodological issues and contextual differences that hinder a good comparison with results from previous research.
However, the general conclusions of previous studies into nurse practitioners substituting general practitioners during daytime or out-of-hours primary care (Edwards et al., 2009; Collins, 2019; Wijers et al., 2013; Laurant et al., 2018), that nurse practitioners meet patient needs, practise in a safe manner, and provide an effective service for a broad range of patient conditions, are in line with our findings.
Implications for practice and future research The results show that nurse practitioners can make a valuable contribution to primary care during urgent home visits. In countries with a need for extra workforce in primary care, substitution by nurse practitioners can be considered a solution equal to the care delivered by general practitioners. However, a careful selection should be made of nurse practitioners who are able to perform home visits and of eligible health problems. Nurse practitioners should have the option to consult a general practitioner. Earlier work experience of nurse practitioners at the general practitioner cooperative seems to be a success factor, because consequently general practitioners gain confidence in the knowledge and skills of the nurse practitioners, which are crucial for collaboration and shared responsibilities. Future research into the safety and efficiency of home visits by nurse practitioners is recommended, over a longer period of time, using nurse practitioners with different types of educational backgrounds and in more regions to increase the number of included patients and to increase the representativeness of the results. The cost-effectiveness should be studied, as nurse practitioners referred more patients to emergency department and the duration of home visits by nurse
practitioners was longer. Finally, it is interesting to examine if the substitution decreases workload for general practitioners at the general practitioner cooperative and if this leads to more time for patients needing more complex care and increased quality of care.
Conclusions The results indicate that experienced nurse practitioners can safely, efficiently and satisfactorily perform low complex out-of-hours primary care home visits. They are practicing at a level equivalent to general practitioners for these home visits. This study adds to the body of evidence for the positive impact of nurse practitioners in primary care.
References Bellow AA Jr, Gillespie GL. The evolution of ED crowding. J. Emerg. Nurs. 2014; 40: 153-160.
Berchet C, Nader C. The Organisation of Out-of-Hours Primary Care in OECD Countries. OECD Health Working Papers no. 89. Paris: OECD Pub; 2016. doi:10.1787/18152015. Accessed at www.oecdilibrary.org/social-issues-migration-health/the-organisation-of-out-of-hours-primary-care-in-oecdcountries_5jlr3czbqw23-en on 11 March 2019.
Christensen LD, Reilev M, Juul-Larsen HG, Jørgensen LM, Kaae S, Andersen O, Pottegård A, Petersen J. Use of prescription drugs in the older adult population-a nationwide pharmacoepidemiological study. Eur. J. Clin. Pharmacol. 2019 Apr 4. doi: 10.1007/s00228-019-02669-2. [Epub ahead of print]
Collins D. Assessing the effectiveness of advanced nurse practitioners undertaking home visits in an out of hours urgent primary care service in England. J. Nurs. Manag. 2019; 27: 450-458.
Coombes R. How to fix out of hours care. BMJ. 2016; 353: i235.
Dierick‐van Daele AT, Spreeuwenberg C, Derckx EW, Metsemakers JF, Vrijhoef BJ. Critical appraisal of the literature on economic evaluations of substitution of skills between professionals: a systematic literature review. J. Eval. Clin. Pract. 2008; 14: 481‐492.
Dierick‐van Daele AT, Metsemakers JF, Derckx EW, Spreeuwenberg C, Vrijhoef HJ. Nurse practitioners substituting for general practitioners: randomized controlled trial. J. Adv. Nurs. 2009; 65: 391‐401.
Edwards M, Bobb C, Robinson SI. Nurse practitioner management of acute in-hours home visit or assessment requests: a pilot study. Br. J. Gen. Pract. 2009; 59: 7-11.
Freund T, Everett C, Griffiths P, Hudon C, Naccarella L, Laurant M. Skill mix, roles and remuneration in the primary care workforce: who are the healthcare professionals in the primary care teams across the world? Int. J. Nurs. Stud. 2015; 52:727-743.
Giesen P, Smits M, Huibers L, Grol R, Wensing M. Quality of after-hours primary care in the Netherlands: a narrative review. Ann. Intern. Med. 2011; 155: 108-113.
Halfon P, Eggli Y, van Melle G, Chevalier J, Wasserfallen JB, Burnand B. Measuring potentially avoidable hospital readmissions. J. Clin. Epidemiol. 2002; 55: 573-587.
Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann. Emerg. Med. 2008; 52: 126–36.1.
Huibers L, Giesen P, Wensing M, Grol R. Out-of-hours care in western countries: assessment of different organizational models. BMC. Health Serv. Res. 2009; 9: 105.
Huibers L, Thijssen W, Koetsenruijter J, Giesen P, Grol R, Wensing M. GP cooperative and emergency department: an exploration of patient flows. J. Eval. Clin. Pract. 2013; 19: 243-249.
Huibers L, Philips H, Giesen P, Remmen R, Christensen MB, Bondevik GT. EurOOHnet—the European research network for out-ofhours primary health care. Eur. J. Gen. Pract. 2014; 20: 229-232.
Keizer E, Oude Bos A, Giesen P, Smits M. [Development of the PREM for general practitioner cooperatives]. Nijmegen: IQ healthcare, Radboudumc; 2017.
Kooienga SA, Carryer JB. Globalization and advancing primary health care nurse practitioner practice. Journal for Nurse Practitioners 2015;11:804–11.
Laurant M, van der Biezen M, Wijers N, Watananirun K, Kontopantelis E, van Vught AJ. Nurses as substitutes for doctors in primary care. Cochrane Database Syst. Rev. 2018; 7: CD001271.
Leibowitz R, Day S, Dunt D. A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam. Pract. 2003; 20: 311-317.
Maier CB, Barnes H, Aiken LH, Busse R. Descriptive, cross-country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential. BMJ Open. 2016; 6: e011901.
Martinez‐Gonzalez NA, Djalali S, Tandjung R, Huber-Geismann F, Markun S, Wnesing M, Rosemann T. Substitution of physicians by nurses in primary care: a systematic review and meta‐analysis. BMC Health. Serv. Res. 2014; 14: 214.
Moskop JC, Sklar DP, Geiderman JM, Schears RM, Bookman KJ. Emergency department crowding, part 1—concept, causes, and moral consequences. Ann. Emerg. Med. 2009; 53: 605-611.
Nederlands Huisartsen Genootschap (NHG). [NHG Standards]. NHG: Utrecht; 2019 https://www.nhg.org/nhg-standaarden (accessed 11 March 2019). Rutten M, Vrielink F, Smits M, Giesen P. Patient and care characteristics of self-referrals treated by the general practitioner cooperative at emergency-care-access-points in the Netherlands. BMC Fam. Pract. 2017; 18: 62.
Smits M, Huibers L, Kerssemeijer B, de Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: a review of patient records. BMC Health Serv. Res. 2010; 10: 335.
Smits M, Huibers L, Oude Bos A, Giesen P. Patient satisfaction with out-of-hours GP cooperatives: a longitudinal study. Scand. J. Prim. Health Care. 2012; 30: 206-213.
Smits M, Keizer E, Huibers L, Giesen P. GPs' experiences with out‐of‐hours GP cooperatives: a survey study from the Netherlands. Eur. J. Gen. Pract. 2014; 20: 196‐201.
Smits M, Rutten M, Keizer E, Wensing M, Westert G, Giesen P. The development and performance of after-hours primary care in the Netherlands: a narrative review. Ann. Intern. Med. 2017; 166: 737742.
van der Biezen M, Schoonhoven L, Wijers N, van der Burgt R, Wensing M, Laurant M. Substitution of general practitioners by nurse practitioners in out‐of‐hours primary care: a quasi‐experimental study. J. Adv. Nurs. 2016; 72: 1813‐1824.
van der Biezen M, Wensing M, van der Burgt R, Laurant M. Towards an optimal composition of general practitioners and nurse practitioners in out-of-hours primary care teams: a quasiexperimental study. BMJ Open 2017; 7: e015509.
van der Linden C, Reijnen R, Derlet RW, Lindeboom R, van der Linden N, Richards JR. Emergency department crowding in the Netherlands: managers’ experiences. Int. J. Emerg. Med. 2013; 6: 41.
Wijers N, van den Burgt R, Laurant M. [Nurse practitioner offers opportunities. Research report on the deployment of nurse practitioners at the Emergency Care Access Point in Eindhoven]. Eindhoven. Nijmegen/Eindhoven: IQ healthcare/Stichting KOH, 2013.
Willekens M, Giesen P, Plat E, Mokkink H, Burgers J, Grol R. Quality of after-hours primary care in the Netherlands: adherence to national guidelines. BMJ Qual. Saf. 2011; 20: 223-227.
Wilson A, Childs S. The relationship between consultation length, process and outcomes in general practice: a systematic review. Br. J. Gen. Pract. 2002; 52: 1012-1020.
Declarations Ethics approval and consent to participate The Ethical Research Committee of the Radboud university medical center Nijmegen was consulted and concluded that this study does not fall within the remit of the Dutch Medical Research Involving Human Subjects Act [Wet Mensgebonden Onderzoek] (file number 2016-2668). Only patients who signed an informed consent form were included in the non-anonymous part of the study.
Consent for publication Not applicable.
Availability of data and material The dataset used during the current study is available from the corresponding author on reasonable request.
Conflict of interests None.
Funding This work was financially supported by ZonMw, the Netherlands.
Authors' contributions MS designed the study, coordinated the data collection, performed the statistical analyses, interpreted the data and drafted the manuscript. YP and SR collected the data, analysed and interpreted the data and revised the manuscript critically for important intellectual content. EP participated in the interpretation of the data and revised the manuscript critically for important intellectual content. ML and PG participated in the conception and design of the study and interpretation of the data, and revised the manuscript critically for important intellectual content. All authors read and approved the final manuscript.
Acknowledgements We thank all participants in the study.
Figures
Figure 1. Flow chart of patients included in the different parts of the study