- Email: [email protected]
Assessment of patient's experiences across the interface between primary and secondary care: Consumer Quality Index Continuum of Care
Patient Education and Counseling 77 (2009) 123–127
Contents lists available at ScienceDirect
Patient Education and Counseling journal homepage: www.elsevier.com/locate/pateducou
Assessment
Assessment of patient’s experiences across the interface between primary and secondary care: Consumer Quality Index Continuum of Care Annette J. Berendsen a,*, Klaas H. Groenier a, G. Majella de Jong a, Betty Meyboom-de Jong a, Willem Jan van der Veen a, Janny Dekker a, Margot W.M. de Waal b, Jan Schuling a a b
Department of General Practice, University Medical Centre Groningen, University of Groningen, The Netherlands Department of Public Health and Primary Care, Leiden University Medical Centre, University of Leiden, The Netherlands
A R T I C L E I N F O
A B S T R A C T
Article history: Received 26 August 2008 Received in revised form 7 January 2009 Accepted 25 January 2009
Objective: Development and validation of a questionnaire that measures patients’ experiences of collaboration between general practitioners (GPs) and specialists. Methods: A questionnaire was developed using the method of the consumer quality index and validated in a cross-sectional study among a random sample of patients referred to medical specialists in the Netherlands. Validation included factor analysis, ascertain internal consistency, and the discriminative ability. Results: The response rate was 65% (1404 patients). Exploratory factor analysis indicated that four domains could be distinguished (i.e. GP Approach; GP Referral; Specialist; Collaboration). Cronbach’s alpha coefficients ranged from 0.51 to 0.93 indicating sufficient internal consistency to make comparison of groups of respondents possible. The Pearson correlation coefficients between the domains were <0.4, except between the domains GP Approach and GP Referral. All domains clearly produced discriminating scores for groups with different characteristics. Conclusions: The Consumer Quality Index (CQ-index) Continuum of Care can be a useful instrument to assess aspects of the collaboration between GPs and specialists from patients’ perspective. Practice implications: It can be used to give feedback to both medical professionals and policy makers. Such feedback creates an opportunity for implementing specific improvements and evaluating quality improvement projects. ß 2009 Elsevier Ireland Ltd. All rights reserved.
Keywords: Patients’ perceptions Assessing experiences Interface between primary and secondary care Consumer quality index
1. Introduction Increasingly often, patients are treated by several care providers simultaneously. Ageing of the population, which is accompanied by multimorbidity, plays an important role in explaining this phenomenon. In order to provide quality of care, collaboration between the parties involved is essential. The need for sound collaboration is acknowledged by all stakeholders in health care [1]. The same goes for the continuity of care between primary and secondary care. Our earlier qualitative study showed that improvements are necessary in the Continuum of Care [2]. Other studies among patients on the continuity of care were done in the UK [3–5]. Further research concerned experiences with parts of the Continuum of Care, such as the referral [6], hospital stay and
* Corresponding author at: Department of General Practice, University Medical Centre Groningen, University of Groningen, Ant. Deusinglaan 1, 9713 AV Groningen, The Netherlands. Tel.: +31 50 3632975; fax: +31 50 3637445. E-mail address: [email protected] (A.J. Berendsen). 0738-3991/$ – see front matter ß 2009 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.pec.2009.01.011
discharge [7–11]. As well, experiences of patients with new models of collaboration were researched [12]. The concepts continuity and progress are central to patients’ view of care. Three types of continuity were identified: informational, management, and relational continuity. These three types together form an important parameter of care. Detailed information on the quality of care provided will be beneficial in two ways: it offers care providers a starting point on how to improve care, and it may help other parties involved, such as patients and insurance companies, to make choices. Not only medical indicators determine the quality of care: the opinion of those receiving care is equally important. Not surprisingly, increasing attention is paid to the development of questionnaires used in research among patients [13,14]. There are two types of questionnaires that are used to this end: the first is concerned with ‘the importance of the care provided, combined with how this care is experienced’, and the second is used to ‘determine the level of satisfaction with the care provided’ [15]. Questioning patients on actual experiences with the care provided seems to demonstrate the quality of care more than questioning patients on satisfaction. Also, actual experiences are easier to interpret, and they form a
124
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
good basis for quality enhancing measures [16]. In order to measure both the importance of the medical care provided and the experiences patients have with this care, a new ‘family’ of questionnaires was created in the Netherlands, based on two existing ‘families’ of questionnaires (QUOTE and CAHPS1). This new ‘family’ is the consumers quality index (CQ-index) [17,18]. So, the objective of this study was to develop and to validate an instrument that measures how patients experience the collaboration between general practitioners (GPs) and specialists. As a contribution to construct validity, we tested a few hypotheses generated from our qualitative research [2]. Patients who are seen quickly have more positive experiences. Patients are generally more positive about GPs than about specialists, particularly patients with MUPS. Many patients think the specialist should report more quickly to the GP. In addition, the report should include more information. Patients wish to receive better information on the timing of discharge and on what they can expect after discharge with respect to symptoms and continuing medical complaints. This will be most evident in patients with a treatable condition. Besides, it is well-known that older people, low-educated people and people in good health test more positive on experiences [24]. 2. Methods The study design was reviewed by the Medical Ethics Committee of the University Medical Centre Groningen, however, ethical approval was not required.
three solo and six group practices in the North and the West of the Netherlands. The practices participating in these networks register the daily care to their patients (about 60,000) and put these registration data (on contacts, morbidity, medicine prescriptions, and referrals) at disposal of research activities. From the databases, patients were selected who were over 18 years of age, had been referred to a medical specialist in the last 2 years (excluding paediatricians and psychiatrists) and had actually visited the specialist concerned. We decided on a 2-year time period as our focus group meetings showed that memories after such a time span were still very vivid. Experiences ‘on the extremes’ (very negative/positive) might be remembered better. These extremes were seldom described in our focus group meetings. Even the opposite was true: there was a wide range of experiences. The questionnaire was sent to two random samples of 2159 patients from the selected population. This sample was stratified for GP and the age of the patient. Those who did not respond were sent a reminder later on. This procedure was repeated after some weeks (Dillman method [21]). A total number of 1534 patients returned the questionnaire, of which 87 indicated that they did not wish to participate. Another 43 questionnaires could not be used due to incomplete information, leaving 1404 questionnaires for analysis (a net response of 65%). By means of the databases, age and gender of the non-respondents were determined. Only patients who had been admitted to hospital answered questions on their experiences at discharge from the hospital. The study was conducted between April and September 2007. 2.3. Questionnaire validation
In our earlier qualitative, explorative research among patients, we defined the themes for the questionnaire [2]. Based on these themes items were formulated, which could be responded to on a dichotomic or ordinal scale (always, usually, sometimes, never). The concept questionnaire comprised 36 questions on how patients were approached and referred by the GP, waiting times for specialist care, approach and referral back by the specialist and questions concerning how they experienced collaboration. To increase content validity, the list was presented to key figures of 13 different organizations in the Netherlands, both patient associations and care providers. No items were added. Thirteen test questionnaires were filled out by patients with varying educational backgrounds. Some adjustments were made (some answer categories were made dichotomous instead of ordinal). General characteristics of respondents (age, gender, etc.) were also included, as well as patients’ information regarding conditions the patient was referred for. Two researchers categorized these conditions independent from each other into five groups. The diagnoses were divided into the following segments: chronic condition, treatable condition, medically unexplained physical symptoms (MUPS), cancer and other. The care given to these patient groups is fundamentally different. So, differences in individual experiences may therefore also be contributed to these segments.
The questionnaire was validated in a cross-sectional study among referred patients. The data were analyzed using the SPSS statistical package. The distribution of answer categories was first reviewed per item, keeping the distribution of the dichotomic variables in mind. The negatively posed questions were mirrored, so that a high score corresponds to a positive attitude. We conducted a factor analysis in order to asses whether the clustering of items was as expected. Items were excluded from the factor analysis when over 5% of the answers were missing or when more than 90% of the answers fell in the most extreme answering category. Based on the screeplot the number of factors was determined and subsequently rotated according to the varimax criterion. The explained variance of the factors was calculated. Subscales were constructed based on the factor loadings. Factor loadings >0.4 were considered important. The internal consistency of these subscales was checked by correlating the items with the total scores of every scale minus the score of the item concerned (item-rest correlation). Cronbach’s alpha coefficients were determined [22,23]. The ability to discriminate between subgroups of patients was tested by calculating differences in total scores for the abovementioned patient characteristics. Testing was done through non-parametric tests (because of skewed variables). When comparing two groups, the Chi-square and Mann–Whitney test were used, when comparing more than two groups the Kruskal–Wallis test and in case of dependent observations the Wilcoxon signed-rank test. A p-value of <0.05 was considered significant.
2.2. Study population
3. Results
We used the databases of the registration networks of the Departments of General Practice of the Universities of Groningen (Registration Network Groningen [19]) and Leiden (RNUH/LEO [20]) to select patients. These two databases contain data from
3.1. Respondent characteristics
2.1. Questionnaire development
Table 1 shows the characteristics of the 1404 respondents. The male–female ratio is 40/60%. The age and gender of the 712 non-
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127 Table 1 Characteristics of respondents.
Table 2 Spearman correlation coefficients between factors. Respondents (%)
Non-respondents (%)
n = 1404
n = 712
Age 18–34 35–64 65 or over
13 58 29
31 55 14
Female
60
55 National (%) statistics Netherlands (CBS)
Education Primary school/GCSE (lower educated) GCSE A-levels College/university (higher educated)
125
27
29
19 27 27
43
Type of illness Chronic illness Treatable condition MUPS Cancer Other
18 36 2 3 41
Self-reported health Excellent Very good Good Reasonably well Bad
9 16 55 18 2
28
MUPS = medically unexplained physical symptoms.
respondents are also shown in Table 1. The proportion of old people is higher in the respondent group than in the nonrespondents. The average education level of the general population of the Netherlands is also shown in Table 1. The educational level of the respondents does not differ from the average of the general population. The older respondents (65+) are largely from a lower educational background (39%). The self-reported diagnoses are divided into chronic condition (18%), treatable condition (36%), MUPS (2%), cancer (3%) and other including unclear diagnosis (41%). The self-reported health is in 80% of the cases excellent through good (Table 1). 3.2. Factor analysis and item analysis The questions regarding the GP, specialist and their collaboration were suitable for factor analysis, as these questions were answered by all respondents. This was not true for the question regarding the referral back to the GP, as not all respondents answered this question (n = 357; 26%). Many patients (74%) indicated they have no insight in this referral back. The four
GP Approach GP Referral Specialist Collaboration GP Referral 0.58 Specialist 0.32 Collaboration 0.35 Discharge after Admittance 0.10
0.25 0.36 0.11
0.37 0.26
0.18
questions regarding discharge after admittance were not suitable for factor analysis either because only part of the respondents had inpatient experiences (n = 505). For the remaining 31 questions, five factors produced the best results, explaining 56% of the total variance. Two questions were excluded because their factor loadings were less than 0.4. Based on the content of the questions, we gave the factors the following titles: GP (14 items), Specialist (11 items), Collaboration (2 items) and Waiting Time (2 items). Since a part of the 14 questions concerning the GP had high loadings on more than one factor, we did a separate factor analysis on those 14 questions. Two factors were rotated (eigenvalue > 1). This also shows that the clustering of the items was not as expected, so two clusters emerged, titled: GP Approach (9) and GP Referral (5). From the domain GP Approach, one item was removed due to low item-rest correlation. Another two items were deleted for reasons of content (redundancy), and to keep the list as concise as possible, so this domain eventually comprises six items (a 0.85). The domain GP Referral consists of five questions (a 0.66). From the domain Specialist another two items were deleted due to redundancy; the remaining nine items make up the total (a 0.93). The domain Collaboration comprises two items (a 0.51). The domain Waiting Time had a low internal consistency (a 0.2). This domain was removed. The domain Discharge after Admittance went from four to three items (a 0.45). The complete questionnaire comprises 25 items divided into five domains. The correlation coefficients (Spearman’s rho) between the domains are <0.4, except for the two concerning the GP (Table 2). 3.3. Validation/total scores Table 3 shows the statistical significance testing of the differences in scores between subgroups. Older people scored higher on GP Approach, GP Referral, Specialist and Collaboration. Low-educated people scored higher on Specialist and Collaboration. Patients who scored higher on experienced health, also scored higher on GP Approach, Specialist and Collaboration. Patients who were referred within 3 days, scored higher on GP Approach, GP Referral and Collaboration than patients who had an appointment after a few weeks. Patients with MUPS scored lower on GP Referral.
Table 3 Differences in scores.
Older people Low-educated people Self-reported physical health: high Speed of referral: high Disease: MUPS Women Hospitals State of being: good
GP Approach
GP Referral
Specialist
Collaboration
p-Value
p-Value
p-Value
p-Value
<0.0005 "
0.011 " 0.001 " <0.0005 "
<0.0005 " 0.008 " <0.0005 " <0.0005 "
0.003 " <0.0005 " 0.001 "
<0.0005 "
": significant higher score and #: significant lower score.
0.003 " 0.031 #
<0.0005 "
<0.0005 # 0.007 <0.0005 "
<0.0005 "
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
126 Table 4 CQI Continuum of Care. Statements
Mean scores (S.D.) n = 1404
Usually + always/yes (%) n = 1404
Item rest correlation
GP Approach My GP listened to me carefully My GP gave me enough time My GP took me seriously My GP explained matters to me in a logical manner I have confidence in my GP’s medical expertise I have noticed my GP is bothered when I stand up for myself
3.7 3.8 3.7 3.8 3.7 3.6 1.2
(0.4) (0.5) (0.5) (0.5) (0.6) (0.6) (0.5)
97 96 97 96 96 3
0.71 0.67 0.71 0.61 0.61 0.54
GP Referral My GP gave me sufficient information about my illness/treatment My GP included me in the decision on the referrala My GP made clear why I was being referreda My GP referred me on timea I think my GP gave the specialist all necessary information when he/she referred mea
1.4 3.6 0.8 0.9 0.8 0.7
(0.3) (0.6) (0.4) (0.3) (0.4) (0.4)
94 76 87 79 74
0.47 0.32 0.51 0.43 0.39
Specialist My specialist listened to me carefully My specialist gave me enough time My specialist took me seriously My specialist explained matters to me in a logical manner My specialist(s) gave me sufficient information on my illness My specialist(s) gave me sufficient information on my treatment I have confidence in my specialist’s medical expertise Were you satisfied with your specialist?a I have noticed my specialist is bothered when I stand up for myself
3.2 3.6 3.5 3.6 3.6 3.5 3.5 3.6 0.8 1.3
(0.6) (0.7) (0.7) (0.6) (0.7) (0.8) (0.8) (0.6) (0.4) (0.6)
90 89 93 91 87 89 93 76 4
0.78 0.79 0.79 0.77 0.80 0.79 0.70 0.63 0.56
Collaboration I experienced the collaboration between GPs and specialists as follows (good/very good) Score the entire referral process from GP tot specialist(s) and back again (0–10)
5.9 (1.2) 3.0 (0.5) 8.7 (1.5)
91
0.60 0.60
a
Cronbach’s alpha 0.85
0.66
0.93
0.51
Binary questions (yes/no). All other questions except the last question (0–10) are on a 4-point scale.
Women scored lower on Specialist. Hospitals differed among each other on Specialist. Patients who were able to discuss important matters (yes/a little/no) scored higher on GP Approach, GP Referral, Specialist and Collaboration. The difference between the means of the domains GP Approach and Specialist is significant (p < 0.0005). Patients scored on average higher on GP Approach. The ability of the domain Discharge after Admittance to discriminate between subgroups is not shown. So, we decided to remove this domain as Cronbach’s alpha was low too. The definitive questionnaire comprises 22 items divided into four domains (Table 4). The item-rest correlations are also reported in this table. 4. Discussion and conclusion 4.1. Discussion This study shows that the CQ-index is a valid instrument to assess certain aspects of the collaboration between GPs and specialists from patients’ perspective. Several domains were defined with sufficient internal consistency to make comparison of different groups of respondents possible (Cronbach’s alpha > 0.6). One domain has a lower internal consistency, possibly due to the small number of questions. There is a weak correlation between the domains. Further analysis shows that the domains clearly produce discriminating scores for groups with different characteristics. Face and content validity were successfully achieved. Construct validity was reasonably well achieved for the domains, as shown by the discriminative ability. Construct validity of the questionnaire was supported by the results of the tested hypotheses. Patients who were seen quickly by the specialist scored higher on almost all domains. This is consistent with the findings from
earlier qualitative research, that patients prefer a smooth transition through the system [2,3]. The impact a referral procedure has on the patient himself is important. Subsequently, this also influences the scores on experiences, for patients who were able to discuss important matters scored higher on all domains. Patients suffering from MUPS yield lower scores on GP Referral. Perhaps this means that this group had to make more effort in order to be referred. We were not able to find other differences between experiences from patients with different conditions. As in other research, older people, low-educated people and people who consider themselves healthier score higher on various domains [24,25]. Patients score significantly higher on GP Approach than on Specialist. This finding confirms the hypothesis of our qualitative research [2]. Except for the relative overrepresentation of older people and underrepresentation of young people, our study population is not very different from the non-respondent/general Dutch population. Though the elder respondents (65+) are largely from a lower educational background. Only clear diagnoses that were filled out by the patient were processed. This explains why the MUPS group is small and the other group is big. It has to be kept in mind that this study was conducted in The Netherlands, where the GP functions as a gatekeeper to secondary care. 4.2. Conclusion Our qualitative study has provided us with rich material to construct a questionnaire regarding patient experiences with the Continuum of Care. This study shows that assessment by patients is limited to items concerning the doctor’s approach and his ability to inform and involve the patient.
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
The definitive questionnaire comprises the domains GP Approach, GP Referral, Specialist and Collaboration. The questions of the removed domains (Waiting Time and Discharge after Admittance) may be differently interpreted by patients. On the other hand, it could be that patients are not able to judge these items properly. Our last two hypotheses referring to these two domains were therefore not tested. The validation process should be completed with data about test–retest reliability, responsiveness and certain aspects of construct validity (convergent, divergent). In order to be able to compare different regions (GPs and hospitals), multilevel analysis may be needed. Afterwards it might be used as a performance indicator. The CQ-index Continuum of Care can be a useful instrument to assess aspects of the collaboration between GPs and specialists from patients’ perspective. 4.3. Practice implications The CQI Continuum of Care can be a useful instrument to give feedback to both medical professionals and policy makers. Such feedback creates an opportunity for implementing specific improvements. It could also be used as an instrument evaluating quality improvement projects. Acknowledgements We would like to thank all the patients, who participated in the test phase and the definitive study. As well, we would like to thank Wim Benneker, for aiding in the formulating of questions, the key figures of various organizations who judged the questionnaires, the GPs and their assistants who were willing to invite their patients, the staff members of the registration networks of Groningen and Leiden and the secretarial staff in Groningen and Leiden, who helped sending out all the questionnaires. This study has been funded by ZonMw, Innovatiefonds Zorgverzekeraars and Menzis, the Netherlands. I confirm all patient/personal identifiers have been removed or disguised so the patient/person(s) described are not identifiable and cannot be identified through the details of the story. References [1] Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. Brit Med J 2003;327:1219–21. [2] Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Continuity and coordination of care: experiences and preferences of patients referred to medical specialists—a qualitative study. BMC Health Serv Res 2009;9:62.
127
[3] Preston C, Cheater F, Baker R, Hearnshaw H. Left in limbo: patients’ views on care across the primary/secondary interface. Qual Health Care 1999;8:16–21. [4] Baker R, Preston C, Cheater F, Hearnshaw H. Measuring patients’ attitudes to care across the primary/secondary interface: the development of the patient career diary. Qual Health Care 1999;8:154–60. [5] Freeman GK, Woloshynowych M, Baker R, Boulton M, Guthrie B, Car J, et al. Continuity of care 2006: what have we learned since 2000 and what are policy imperatives now? Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D. London: NCCSDO; 2007. [6] Grumbach K, Selby JV, Damberg C, Bindman AB, Quesenberry C, Truman A, et al. Resolving the gatekeeper conundrum—what patients value in primary care and referrals to specialists. J Am Med Assoc 1999;282:261–6. [7] Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care 2002;11:335–9. [8] Bruster S, Jarman B, Bosanquet N, Weston D, Erens R, Delbanco TL. National survey of hospital patients. Brit Med J 1994;309:1542–6. [9] Hruby M, Pantilat SZ, Lo B. How do patients view the role of the primary care physician in inpatient care? Am J Med 2007;111:21S–5S. [10] Hadjistavropoulos H, Biem H, Sharpe D, Bourgault-Fagnou M, Janzen J. Patient perceptions of hospital discharge: reliability and validity of a patient continuity of care questionnaire. Int J Qual Health Care 2008. [11] Mistiaen P, Duijnhouwer E, Wijkel D, de BM, Veeger A. The problems of elderly people at home one week after discharge from an acute care setting. J Adv Nurs 1997;25:1233–40. [12] Nielsen JD, Palshof T, Mainz J, Jensen AB, Olesen F. Randomised controlled trial of a shared care programme for newly referred cancer patients: bridging the gap between general practice and hospital. Qual Saf Health Care 2003;12:263– 72. [13] Sixma HJ, Kerssens JJ, van Campen C, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expect 1998;1:82–95. [14] Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garfinkel S. Increasing the impact of health plan report cards by addressing consumers’ concerns. Health Aff (Millwood) 2000;19:138–43. [15] Cleary PD, EdgmanLevitan S. Health care quality—incorporating consumer perspectives. J Am Med Assoc 1997;278:1608–12. [16] Cleary PD, Edgman-Levitan S, Roberts M, Moloney TW, McMullen W, Walker JD, et al. Patients evaluate their hospital care: a national survey. Health Aff (Millwood) 1991;10:254–67. [17] Stubbe JH, Gelsema T, Delnoij DM. The consumer quality index hip knee questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv Res 2007;7:60. [18] Stubbe JH, Brouwer W, Delnoij DM. Patients’ experiences with quality of hospital care: the consumer quality index cataract questionnaire. BMC Ophthalmol 2007;7:14. [19] Veen van der WJ, Meyboom-de Jong B. Age and gender. In: Jones R, Britten N, Culpepper L, Gass DA, Grol R, Mant D, editors. Oxford textbook of primary medical care. Oxford: Oxford University Press; 2004. [20] Gijsen R, Poos MJ. Using registries in general practice to estimate countrywide morbidity in The Netherlands. Public Health 2006;120:923–36. [21] Dillman DA. Mail and telephone surveys: the total design method. New York: Wiley & Sons; 1978. [22] Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1967. [23] Streiner DL, Norman GR. Health measurements scales. A practical guide to their development and use. Oxford: Oxford University Press; 2003. [24] Zaslavsky AM. Statistical issues in reporting quality data: small samples and casemix variation. Int J Qual Health Care 2001;13:481–8. [25] Carlson MJ, Shaul JA, Eisen SV, Cleary PD. The influence of patient characteristics on ratings of managed behavioral health care. J Behav Health Serv Res 2002;29:481–9.
Contents lists available at ScienceDirect
Patient Education and Counseling journal homepage: www.elsevier.com/locate/pateducou
Assessment
Assessment of patient’s experiences across the interface between primary and secondary care: Consumer Quality Index Continuum of Care Annette J. Berendsen a,*, Klaas H. Groenier a, G. Majella de Jong a, Betty Meyboom-de Jong a, Willem Jan van der Veen a, Janny Dekker a, Margot W.M. de Waal b, Jan Schuling a a b
Department of General Practice, University Medical Centre Groningen, University of Groningen, The Netherlands Department of Public Health and Primary Care, Leiden University Medical Centre, University of Leiden, The Netherlands
A R T I C L E I N F O
A B S T R A C T
Article history: Received 26 August 2008 Received in revised form 7 January 2009 Accepted 25 January 2009
Objective: Development and validation of a questionnaire that measures patients’ experiences of collaboration between general practitioners (GPs) and specialists. Methods: A questionnaire was developed using the method of the consumer quality index and validated in a cross-sectional study among a random sample of patients referred to medical specialists in the Netherlands. Validation included factor analysis, ascertain internal consistency, and the discriminative ability. Results: The response rate was 65% (1404 patients). Exploratory factor analysis indicated that four domains could be distinguished (i.e. GP Approach; GP Referral; Specialist; Collaboration). Cronbach’s alpha coefficients ranged from 0.51 to 0.93 indicating sufficient internal consistency to make comparison of groups of respondents possible. The Pearson correlation coefficients between the domains were <0.4, except between the domains GP Approach and GP Referral. All domains clearly produced discriminating scores for groups with different characteristics. Conclusions: The Consumer Quality Index (CQ-index) Continuum of Care can be a useful instrument to assess aspects of the collaboration between GPs and specialists from patients’ perspective. Practice implications: It can be used to give feedback to both medical professionals and policy makers. Such feedback creates an opportunity for implementing specific improvements and evaluating quality improvement projects. ß 2009 Elsevier Ireland Ltd. All rights reserved.
Keywords: Patients’ perceptions Assessing experiences Interface between primary and secondary care Consumer quality index
1. Introduction Increasingly often, patients are treated by several care providers simultaneously. Ageing of the population, which is accompanied by multimorbidity, plays an important role in explaining this phenomenon. In order to provide quality of care, collaboration between the parties involved is essential. The need for sound collaboration is acknowledged by all stakeholders in health care [1]. The same goes for the continuity of care between primary and secondary care. Our earlier qualitative study showed that improvements are necessary in the Continuum of Care [2]. Other studies among patients on the continuity of care were done in the UK [3–5]. Further research concerned experiences with parts of the Continuum of Care, such as the referral [6], hospital stay and
* Corresponding author at: Department of General Practice, University Medical Centre Groningen, University of Groningen, Ant. Deusinglaan 1, 9713 AV Groningen, The Netherlands. Tel.: +31 50 3632975; fax: +31 50 3637445. E-mail address: [email protected] (A.J. Berendsen). 0738-3991/$ – see front matter ß 2009 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.pec.2009.01.011
discharge [7–11]. As well, experiences of patients with new models of collaboration were researched [12]. The concepts continuity and progress are central to patients’ view of care. Three types of continuity were identified: informational, management, and relational continuity. These three types together form an important parameter of care. Detailed information on the quality of care provided will be beneficial in two ways: it offers care providers a starting point on how to improve care, and it may help other parties involved, such as patients and insurance companies, to make choices. Not only medical indicators determine the quality of care: the opinion of those receiving care is equally important. Not surprisingly, increasing attention is paid to the development of questionnaires used in research among patients [13,14]. There are two types of questionnaires that are used to this end: the first is concerned with ‘the importance of the care provided, combined with how this care is experienced’, and the second is used to ‘determine the level of satisfaction with the care provided’ [15]. Questioning patients on actual experiences with the care provided seems to demonstrate the quality of care more than questioning patients on satisfaction. Also, actual experiences are easier to interpret, and they form a
124
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
good basis for quality enhancing measures [16]. In order to measure both the importance of the medical care provided and the experiences patients have with this care, a new ‘family’ of questionnaires was created in the Netherlands, based on two existing ‘families’ of questionnaires (QUOTE and CAHPS1). This new ‘family’ is the consumers quality index (CQ-index) [17,18]. So, the objective of this study was to develop and to validate an instrument that measures how patients experience the collaboration between general practitioners (GPs) and specialists. As a contribution to construct validity, we tested a few hypotheses generated from our qualitative research [2]. Patients who are seen quickly have more positive experiences. Patients are generally more positive about GPs than about specialists, particularly patients with MUPS. Many patients think the specialist should report more quickly to the GP. In addition, the report should include more information. Patients wish to receive better information on the timing of discharge and on what they can expect after discharge with respect to symptoms and continuing medical complaints. This will be most evident in patients with a treatable condition. Besides, it is well-known that older people, low-educated people and people in good health test more positive on experiences [24]. 2. Methods The study design was reviewed by the Medical Ethics Committee of the University Medical Centre Groningen, however, ethical approval was not required.
three solo and six group practices in the North and the West of the Netherlands. The practices participating in these networks register the daily care to their patients (about 60,000) and put these registration data (on contacts, morbidity, medicine prescriptions, and referrals) at disposal of research activities. From the databases, patients were selected who were over 18 years of age, had been referred to a medical specialist in the last 2 years (excluding paediatricians and psychiatrists) and had actually visited the specialist concerned. We decided on a 2-year time period as our focus group meetings showed that memories after such a time span were still very vivid. Experiences ‘on the extremes’ (very negative/positive) might be remembered better. These extremes were seldom described in our focus group meetings. Even the opposite was true: there was a wide range of experiences. The questionnaire was sent to two random samples of 2159 patients from the selected population. This sample was stratified for GP and the age of the patient. Those who did not respond were sent a reminder later on. This procedure was repeated after some weeks (Dillman method [21]). A total number of 1534 patients returned the questionnaire, of which 87 indicated that they did not wish to participate. Another 43 questionnaires could not be used due to incomplete information, leaving 1404 questionnaires for analysis (a net response of 65%). By means of the databases, age and gender of the non-respondents were determined. Only patients who had been admitted to hospital answered questions on their experiences at discharge from the hospital. The study was conducted between April and September 2007. 2.3. Questionnaire validation
In our earlier qualitative, explorative research among patients, we defined the themes for the questionnaire [2]. Based on these themes items were formulated, which could be responded to on a dichotomic or ordinal scale (always, usually, sometimes, never). The concept questionnaire comprised 36 questions on how patients were approached and referred by the GP, waiting times for specialist care, approach and referral back by the specialist and questions concerning how they experienced collaboration. To increase content validity, the list was presented to key figures of 13 different organizations in the Netherlands, both patient associations and care providers. No items were added. Thirteen test questionnaires were filled out by patients with varying educational backgrounds. Some adjustments were made (some answer categories were made dichotomous instead of ordinal). General characteristics of respondents (age, gender, etc.) were also included, as well as patients’ information regarding conditions the patient was referred for. Two researchers categorized these conditions independent from each other into five groups. The diagnoses were divided into the following segments: chronic condition, treatable condition, medically unexplained physical symptoms (MUPS), cancer and other. The care given to these patient groups is fundamentally different. So, differences in individual experiences may therefore also be contributed to these segments.
The questionnaire was validated in a cross-sectional study among referred patients. The data were analyzed using the SPSS statistical package. The distribution of answer categories was first reviewed per item, keeping the distribution of the dichotomic variables in mind. The negatively posed questions were mirrored, so that a high score corresponds to a positive attitude. We conducted a factor analysis in order to asses whether the clustering of items was as expected. Items were excluded from the factor analysis when over 5% of the answers were missing or when more than 90% of the answers fell in the most extreme answering category. Based on the screeplot the number of factors was determined and subsequently rotated according to the varimax criterion. The explained variance of the factors was calculated. Subscales were constructed based on the factor loadings. Factor loadings >0.4 were considered important. The internal consistency of these subscales was checked by correlating the items with the total scores of every scale minus the score of the item concerned (item-rest correlation). Cronbach’s alpha coefficients were determined [22,23]. The ability to discriminate between subgroups of patients was tested by calculating differences in total scores for the abovementioned patient characteristics. Testing was done through non-parametric tests (because of skewed variables). When comparing two groups, the Chi-square and Mann–Whitney test were used, when comparing more than two groups the Kruskal–Wallis test and in case of dependent observations the Wilcoxon signed-rank test. A p-value of <0.05 was considered significant.
2.2. Study population
3. Results
We used the databases of the registration networks of the Departments of General Practice of the Universities of Groningen (Registration Network Groningen [19]) and Leiden (RNUH/LEO [20]) to select patients. These two databases contain data from
3.1. Respondent characteristics
2.1. Questionnaire development
Table 1 shows the characteristics of the 1404 respondents. The male–female ratio is 40/60%. The age and gender of the 712 non-
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127 Table 1 Characteristics of respondents.
Table 2 Spearman correlation coefficients between factors. Respondents (%)
Non-respondents (%)
n = 1404
n = 712
Age 18–34 35–64 65 or over
13 58 29
31 55 14
Female
60
55 National (%) statistics Netherlands (CBS)
Education Primary school/GCSE (lower educated) GCSE A-levels College/university (higher educated)
125
27
29
19 27 27
43
Type of illness Chronic illness Treatable condition MUPS Cancer Other
18 36 2 3 41
Self-reported health Excellent Very good Good Reasonably well Bad
9 16 55 18 2
28
MUPS = medically unexplained physical symptoms.
respondents are also shown in Table 1. The proportion of old people is higher in the respondent group than in the nonrespondents. The average education level of the general population of the Netherlands is also shown in Table 1. The educational level of the respondents does not differ from the average of the general population. The older respondents (65+) are largely from a lower educational background (39%). The self-reported diagnoses are divided into chronic condition (18%), treatable condition (36%), MUPS (2%), cancer (3%) and other including unclear diagnosis (41%). The self-reported health is in 80% of the cases excellent through good (Table 1). 3.2. Factor analysis and item analysis The questions regarding the GP, specialist and their collaboration were suitable for factor analysis, as these questions were answered by all respondents. This was not true for the question regarding the referral back to the GP, as not all respondents answered this question (n = 357; 26%). Many patients (74%) indicated they have no insight in this referral back. The four
GP Approach GP Referral Specialist Collaboration GP Referral 0.58 Specialist 0.32 Collaboration 0.35 Discharge after Admittance 0.10
0.25 0.36 0.11
0.37 0.26
0.18
questions regarding discharge after admittance were not suitable for factor analysis either because only part of the respondents had inpatient experiences (n = 505). For the remaining 31 questions, five factors produced the best results, explaining 56% of the total variance. Two questions were excluded because their factor loadings were less than 0.4. Based on the content of the questions, we gave the factors the following titles: GP (14 items), Specialist (11 items), Collaboration (2 items) and Waiting Time (2 items). Since a part of the 14 questions concerning the GP had high loadings on more than one factor, we did a separate factor analysis on those 14 questions. Two factors were rotated (eigenvalue > 1). This also shows that the clustering of the items was not as expected, so two clusters emerged, titled: GP Approach (9) and GP Referral (5). From the domain GP Approach, one item was removed due to low item-rest correlation. Another two items were deleted for reasons of content (redundancy), and to keep the list as concise as possible, so this domain eventually comprises six items (a 0.85). The domain GP Referral consists of five questions (a 0.66). From the domain Specialist another two items were deleted due to redundancy; the remaining nine items make up the total (a 0.93). The domain Collaboration comprises two items (a 0.51). The domain Waiting Time had a low internal consistency (a 0.2). This domain was removed. The domain Discharge after Admittance went from four to three items (a 0.45). The complete questionnaire comprises 25 items divided into five domains. The correlation coefficients (Spearman’s rho) between the domains are <0.4, except for the two concerning the GP (Table 2). 3.3. Validation/total scores Table 3 shows the statistical significance testing of the differences in scores between subgroups. Older people scored higher on GP Approach, GP Referral, Specialist and Collaboration. Low-educated people scored higher on Specialist and Collaboration. Patients who scored higher on experienced health, also scored higher on GP Approach, Specialist and Collaboration. Patients who were referred within 3 days, scored higher on GP Approach, GP Referral and Collaboration than patients who had an appointment after a few weeks. Patients with MUPS scored lower on GP Referral.
Table 3 Differences in scores.
Older people Low-educated people Self-reported physical health: high Speed of referral: high Disease: MUPS Women Hospitals State of being: good
GP Approach
GP Referral
Specialist
Collaboration
p-Value
p-Value
p-Value
p-Value
<0.0005 "
0.011 " 0.001 " <0.0005 "
<0.0005 " 0.008 " <0.0005 " <0.0005 "
0.003 " <0.0005 " 0.001 "
<0.0005 "
": significant higher score and #: significant lower score.
0.003 " 0.031 #
<0.0005 "
<0.0005 # 0.007 <0.0005 "
<0.0005 "
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
126 Table 4 CQI Continuum of Care. Statements
Mean scores (S.D.) n = 1404
Usually + always/yes (%) n = 1404
Item rest correlation
GP Approach My GP listened to me carefully My GP gave me enough time My GP took me seriously My GP explained matters to me in a logical manner I have confidence in my GP’s medical expertise I have noticed my GP is bothered when I stand up for myself
3.7 3.8 3.7 3.8 3.7 3.6 1.2
(0.4) (0.5) (0.5) (0.5) (0.6) (0.6) (0.5)
97 96 97 96 96 3
0.71 0.67 0.71 0.61 0.61 0.54
GP Referral My GP gave me sufficient information about my illness/treatment My GP included me in the decision on the referrala My GP made clear why I was being referreda My GP referred me on timea I think my GP gave the specialist all necessary information when he/she referred mea
1.4 3.6 0.8 0.9 0.8 0.7
(0.3) (0.6) (0.4) (0.3) (0.4) (0.4)
94 76 87 79 74
0.47 0.32 0.51 0.43 0.39
Specialist My specialist listened to me carefully My specialist gave me enough time My specialist took me seriously My specialist explained matters to me in a logical manner My specialist(s) gave me sufficient information on my illness My specialist(s) gave me sufficient information on my treatment I have confidence in my specialist’s medical expertise Were you satisfied with your specialist?a I have noticed my specialist is bothered when I stand up for myself
3.2 3.6 3.5 3.6 3.6 3.5 3.5 3.6 0.8 1.3
(0.6) (0.7) (0.7) (0.6) (0.7) (0.8) (0.8) (0.6) (0.4) (0.6)
90 89 93 91 87 89 93 76 4
0.78 0.79 0.79 0.77 0.80 0.79 0.70 0.63 0.56
Collaboration I experienced the collaboration between GPs and specialists as follows (good/very good) Score the entire referral process from GP tot specialist(s) and back again (0–10)
5.9 (1.2) 3.0 (0.5) 8.7 (1.5)
91
0.60 0.60
a
Cronbach’s alpha 0.85
0.66
0.93
0.51
Binary questions (yes/no). All other questions except the last question (0–10) are on a 4-point scale.
Women scored lower on Specialist. Hospitals differed among each other on Specialist. Patients who were able to discuss important matters (yes/a little/no) scored higher on GP Approach, GP Referral, Specialist and Collaboration. The difference between the means of the domains GP Approach and Specialist is significant (p < 0.0005). Patients scored on average higher on GP Approach. The ability of the domain Discharge after Admittance to discriminate between subgroups is not shown. So, we decided to remove this domain as Cronbach’s alpha was low too. The definitive questionnaire comprises 22 items divided into four domains (Table 4). The item-rest correlations are also reported in this table. 4. Discussion and conclusion 4.1. Discussion This study shows that the CQ-index is a valid instrument to assess certain aspects of the collaboration between GPs and specialists from patients’ perspective. Several domains were defined with sufficient internal consistency to make comparison of different groups of respondents possible (Cronbach’s alpha > 0.6). One domain has a lower internal consistency, possibly due to the small number of questions. There is a weak correlation between the domains. Further analysis shows that the domains clearly produce discriminating scores for groups with different characteristics. Face and content validity were successfully achieved. Construct validity was reasonably well achieved for the domains, as shown by the discriminative ability. Construct validity of the questionnaire was supported by the results of the tested hypotheses. Patients who were seen quickly by the specialist scored higher on almost all domains. This is consistent with the findings from
earlier qualitative research, that patients prefer a smooth transition through the system [2,3]. The impact a referral procedure has on the patient himself is important. Subsequently, this also influences the scores on experiences, for patients who were able to discuss important matters scored higher on all domains. Patients suffering from MUPS yield lower scores on GP Referral. Perhaps this means that this group had to make more effort in order to be referred. We were not able to find other differences between experiences from patients with different conditions. As in other research, older people, low-educated people and people who consider themselves healthier score higher on various domains [24,25]. Patients score significantly higher on GP Approach than on Specialist. This finding confirms the hypothesis of our qualitative research [2]. Except for the relative overrepresentation of older people and underrepresentation of young people, our study population is not very different from the non-respondent/general Dutch population. Though the elder respondents (65+) are largely from a lower educational background. Only clear diagnoses that were filled out by the patient were processed. This explains why the MUPS group is small and the other group is big. It has to be kept in mind that this study was conducted in The Netherlands, where the GP functions as a gatekeeper to secondary care. 4.2. Conclusion Our qualitative study has provided us with rich material to construct a questionnaire regarding patient experiences with the Continuum of Care. This study shows that assessment by patients is limited to items concerning the doctor’s approach and his ability to inform and involve the patient.
A.J. Berendsen et al. / Patient Education and Counseling 77 (2009) 123–127
The definitive questionnaire comprises the domains GP Approach, GP Referral, Specialist and Collaboration. The questions of the removed domains (Waiting Time and Discharge after Admittance) may be differently interpreted by patients. On the other hand, it could be that patients are not able to judge these items properly. Our last two hypotheses referring to these two domains were therefore not tested. The validation process should be completed with data about test–retest reliability, responsiveness and certain aspects of construct validity (convergent, divergent). In order to be able to compare different regions (GPs and hospitals), multilevel analysis may be needed. Afterwards it might be used as a performance indicator. The CQ-index Continuum of Care can be a useful instrument to assess aspects of the collaboration between GPs and specialists from patients’ perspective. 4.3. Practice implications The CQI Continuum of Care can be a useful instrument to give feedback to both medical professionals and policy makers. Such feedback creates an opportunity for implementing specific improvements. It could also be used as an instrument evaluating quality improvement projects. Acknowledgements We would like to thank all the patients, who participated in the test phase and the definitive study. As well, we would like to thank Wim Benneker, for aiding in the formulating of questions, the key figures of various organizations who judged the questionnaires, the GPs and their assistants who were willing to invite their patients, the staff members of the registration networks of Groningen and Leiden and the secretarial staff in Groningen and Leiden, who helped sending out all the questionnaires. This study has been funded by ZonMw, Innovatiefonds Zorgverzekeraars and Menzis, the Netherlands. I confirm all patient/personal identifiers have been removed or disguised so the patient/person(s) described are not identifiable and cannot be identified through the details of the story. References [1] Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. Brit Med J 2003;327:1219–21. [2] Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Continuity and coordination of care: experiences and preferences of patients referred to medical specialists—a qualitative study. BMC Health Serv Res 2009;9:62.
127
[3] Preston C, Cheater F, Baker R, Hearnshaw H. Left in limbo: patients’ views on care across the primary/secondary interface. Qual Health Care 1999;8:16–21. [4] Baker R, Preston C, Cheater F, Hearnshaw H. Measuring patients’ attitudes to care across the primary/secondary interface: the development of the patient career diary. Qual Health Care 1999;8:154–60. [5] Freeman GK, Woloshynowych M, Baker R, Boulton M, Guthrie B, Car J, et al. Continuity of care 2006: what have we learned since 2000 and what are policy imperatives now? Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D. London: NCCSDO; 2007. [6] Grumbach K, Selby JV, Damberg C, Bindman AB, Quesenberry C, Truman A, et al. Resolving the gatekeeper conundrum—what patients value in primary care and referrals to specialists. J Am Med Assoc 1999;282:261–6. [7] Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care 2002;11:335–9. [8] Bruster S, Jarman B, Bosanquet N, Weston D, Erens R, Delbanco TL. National survey of hospital patients. Brit Med J 1994;309:1542–6. [9] Hruby M, Pantilat SZ, Lo B. How do patients view the role of the primary care physician in inpatient care? Am J Med 2007;111:21S–5S. [10] Hadjistavropoulos H, Biem H, Sharpe D, Bourgault-Fagnou M, Janzen J. Patient perceptions of hospital discharge: reliability and validity of a patient continuity of care questionnaire. Int J Qual Health Care 2008. [11] Mistiaen P, Duijnhouwer E, Wijkel D, de BM, Veeger A. The problems of elderly people at home one week after discharge from an acute care setting. J Adv Nurs 1997;25:1233–40. [12] Nielsen JD, Palshof T, Mainz J, Jensen AB, Olesen F. Randomised controlled trial of a shared care programme for newly referred cancer patients: bridging the gap between general practice and hospital. Qual Saf Health Care 2003;12:263– 72. [13] Sixma HJ, Kerssens JJ, van Campen C, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expect 1998;1:82–95. [14] Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garfinkel S. Increasing the impact of health plan report cards by addressing consumers’ concerns. Health Aff (Millwood) 2000;19:138–43. [15] Cleary PD, EdgmanLevitan S. Health care quality—incorporating consumer perspectives. J Am Med Assoc 1997;278:1608–12. [16] Cleary PD, Edgman-Levitan S, Roberts M, Moloney TW, McMullen W, Walker JD, et al. Patients evaluate their hospital care: a national survey. Health Aff (Millwood) 1991;10:254–67. [17] Stubbe JH, Gelsema T, Delnoij DM. The consumer quality index hip knee questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv Res 2007;7:60. [18] Stubbe JH, Brouwer W, Delnoij DM. Patients’ experiences with quality of hospital care: the consumer quality index cataract questionnaire. BMC Ophthalmol 2007;7:14. [19] Veen van der WJ, Meyboom-de Jong B. Age and gender. In: Jones R, Britten N, Culpepper L, Gass DA, Grol R, Mant D, editors. Oxford textbook of primary medical care. Oxford: Oxford University Press; 2004. [20] Gijsen R, Poos MJ. Using registries in general practice to estimate countrywide morbidity in The Netherlands. Public Health 2006;120:923–36. [21] Dillman DA. Mail and telephone surveys: the total design method. New York: Wiley & Sons; 1978. [22] Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1967. [23] Streiner DL, Norman GR. Health measurements scales. A practical guide to their development and use. Oxford: Oxford University Press; 2003. [24] Zaslavsky AM. Statistical issues in reporting quality data: small samples and casemix variation. Int J Qual Health Care 2001;13:481–8. [25] Carlson MJ, Shaul JA, Eisen SV, Cleary PD. The influence of patient characteristics on ratings of managed behavioral health care. J Behav Health Serv Res 2002;29:481–9.