- Email: [email protected]
Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Available online at www.sciencedirect.com
Public Health journal homepage: www.elsevier.com/puhe
Original Research
Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients J. Wei a, L. Shen b, H.-B. Yang a, J.-B. Qin a, W. Huang a, J.-J. Zhang a, Q.-Y. Gong a, X.-X. Li a, T.-B. Yang a,* a
Department of Epidemiology and Health Statistics, School of Public Health, Central South University, Changsha, Hunan Province, 410008, China b Xiangya Medical School, Central South University, Changsha, Hunan Province, China
article info
abstract
Article history:
Objective: To develop and test a reliable and practical self-administrated questionnaire in
Received 11 June 2014
Chinese to evaluate outpatient satisfaction in China.
Received in revised form
Design: In order to ensure content validity, the 19 items of this questionnaire were gener-
16 December 2014
ated based on literature review, interview and group discussion. Acceptability was tested
Accepted 6 March 2015
by the percentage of missing item responses and the time and operability of administra-
Available online xxx
tion. Construct validity was evaluated using principal component exploratory factor analysis and item-total correlations. The reliability of the questionnaire was assessed
Keywords:
using Cronbach's alpha coefficient and inter-subscale correlation.
Outpatient
Setting: This study was conducted in 46 public general hospitals that report directly to the
Satisfaction
Provincial Health Department in Hunan Province, China.
Questionnaire
Subjects: In total, 5151 outpatients were selected at random and asked to complete the
Medical care quality
questionnaire after they finished their medical procedures in the outpatient area. Results: The extremely low rate of missing data (0e0.04%) suggested good acceptability of the questionnaire. Factor analysis generated six dimensions, and all item-total coefficients were >0.8. Cronbach's alpha exceeded 0.7 for all dimensions, and the inter-subscale correlation coefficients were all lower than the Cronbach's alpha coefficients of the corresponding scale. According to the results, outpatients were least satisfied with waiting time (86.8%) and most satisfied with the quality of medical care (90.1%). Conclusion: The Chinese outpatient satisfaction questionnaire has good acceptability, validity and reliability. It was effective and efficient for measuring outpatient satisfaction in a Chinese population. © 2015 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Tel.: þ86 13517497107. E-mail address: [email protected] (T.-B. Yang). http://dx.doi.org/10.1016/j.puhe.2015.03.004 0033-3506/© 2015 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
2
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Introduction It is crucial for a humane healthcare system to consider patients' needs and wishes, and patients' perceptions of medical processes can provide unique and valuable information to improve the quality of care. Many studies have suggested that patient satisfaction should be regarded as an important indicator for the quality of care.1e4 There are several motives for evaluating patient satisfaction. Firstly, although the quality of medical services can be evaluated from multiple perspectives (e.g. patient, provider, insurer, etc.), patients should still be considered as the ultimate estimator of the quality of care.5 Secondly, patient satisfaction represents an assessment of their medical experience based on their values, perceptions and interactions with the healthcare environment. Thirdly, patients' thoughts can affect their future behaviours,6 and patients with differing levels of satisfaction may behave differently.7 Patients with a high level of satisfaction tend to be more cooperative with medical staff and the recommended treatment plans.8e12 In contrast, patients with a low level of satisfaction may have less effective medical services, refuse to seek care, or disobey physicians' suggestions and prescribed course of treatment.13 It has been shown previously that patients' health status, frequency and length of hospital stay, compliance with medical suggestions and other medical outcomes could be greatly influenced by their own experience and satisfaction.4,9,14 Lastly, healthcare providers can improve the effectiveness of medical services based on feedback regarding patient satisfaction.2 Patient satisfaction is generally subjective and intangible, so researchers often consider it as an abstract and multidimensional phenomenon.9,12,15 It can only be measured by indirect measurement tools such as suggestion boxes, formal complaints, qualitative methods, audits and self-report questionnaires.16 However, many studies on patient satisfaction have been criticized for poor validity and reliability due to shortcomings in their measurement approaches,17,18 and some studies have not included a clear definition of patient satisfaction.19 The need to develop and test a new satisfaction questionnaire, rather than use a measurement tool that has been published previously, is because of the unique nature of China's outpatient care service. Most questionnaires to measure patient satisfaction were developed in Western countries,20e25 and they may not be suitable for use in developing countries due to differences in medical conditions and facilities. In developed countries, most patients have an appointment before they attend a hospital, and medical insurance may cover their medical expenses. As such, they generally experience shorter waiting times and lower costs compared with patients in developing countries. Several developing countries, such as India, Thailand and Tanzania,26e29 have developed outpatient satisfaction questionnaires (OPSQs). However, there are several differences between China and other developing countries. First, the organization of Chinese general hospital outpatient services is very complicated. Certain services, such as self-services and convenience-for-patients services, may be unique to China, so items related to these aspects need to be included in the
Chinese OPSQ (CH-OPSQ). In addition, the daily outpatient volume is extremely high due to the large Chinese population. Lack of a primary care system in the community results in more patients attending general hospitals. Waiting time, waiting environment and facilities in the waiting room are crucial in patient satisfaction. Furthermore, most medical insurance in China does not cover outpatient expenses, so the costs may have a greater impact on patient satisfaction in China compared with other countries. Therefore, it is essential to provide a new measurement tool to evaluate outpatient satisfaction in China. As such, the objective of this study was to develop an outpatient satisfaction questionnaire for the Chinese population, and to test its reliability, validity and acceptability through a large multicentre sample.
Method Questionnaire development Three members of the research team identified items for inclusion in the Ch-OPSQ from previous studies. A literature search was conducted in MEDLINE and EMBASE between June and July 2012. The following keywords were used in the literature search: ‘outpatient’, ‘satisfaction’, ‘hospital’ and ‘questionnaire’. Relevant studies on outpatient satisfaction were screened and useful information was extracted to build an item pool. An expert panel, including two professors, two experts from the government health department and four postgraduate students, was established to develop an item pool for the Ch-OPSQ (65 items were included). The research team interviewed five patients, five administrators from different hospitals and five officers from the government health department, selected at random from Hunan Province. They were asked to rate the importance of each item, give their opinions and suggestions about the items in the item pool, and comment on the relevance of the issues covered and comprehensibility, including the response options. All the consultation sessions were conducted via faceto-face interview or e-mail. To ensure the content validity of the questionnaire, the interviewees were also asked to provide other potential sources of satisfaction and assign significance to individual items. After the interviews, qualitative analysis was conducted by reviewing the suggestions from the interviews, and refining the words and content of the questions according to the interviewees' feedback. The researchers reached consensus on the items and response options to be listed in the Ch-OPSQ. A draft version of the Ch-OPSQ including 22 items was created, and a pilot survey was undertaken to assess the questionnaire. According to the results of the pilot study, some items were eliminated because of a high non-response rate, and some items were amended if the responses showed multiple answers or poor variability. The final version of the Ch-OPSQ contains 19 items, covering nearly all aspects of outpatient medical care. The questionnaire asks patients to rate their feelings about each statement on a five-point Liket scale: very satisfied (5), relatively satisfied (4), neither satisfied nor dissatisfied (3), relatively dissatisfied (2) and very dissatisfied (1).
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
3
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Fig. 1 shows the questionnaire development process.
approved by the Ethics Committee of Central South University.
Participants and study design
Statistical analysis
The study was conducted between October and December 2012 in 46 hospitals in Hunan Province, all of which report directly to the provincial department of health. The plan was to select 120 outpatients at random from each hospital (total 7680 subjects) to answer the questionnaire. Before the survey started, the research team recruited 30 investigators who were not employed by the hospitals or the provincial department of health. After professional training, they were assigned to the hospitals and asked to perform the survey in outpatient areas. Patients who were conscious and willing to participate in this survey were selected at random in different departments. The investigators gave the Ch-OPSQ to patients after their medical procedures, and collected it upon completion. Strict quality control rules were executed during the survey. Investigators were asked to sign their names on the questionnaire and take responsibility for their survey questionnaires. Every received questionnaire was checked and verified by the investigators and supervisors. Questionnaires with more than 20% of missing data were excluded. The research was
Feasibility and acceptability were examined using the percentage of missing item responses, interviewer-reported acceptability, and the time and operability of administration. In accordance with previous studies,21e24,26,28,30 principal component exploratory factor analysis by varimax rotation was used to establish the structure and test the construct validity of the Ch-OPSQ. Factors were generated with an eigenvalue >1 or a cumulative contribution rate >70%. An individual item had to have a loading 0.60 on one factor to be retained. Items with poor factor loading (<0.60) were eliminated from the questionnaire. The item-total correlation coefficient was calculated to evaluate construct validity. To incorporate an item into a related scale, the corresponding item-total correlation coefficient had to be 0.50.21 Reliability and internal consistency were evaluated using Cronbach's alpha coefficient and inter-subscale correlation. Cronbach's alpha coefficient tested the overall correlation between items within a scale. Cronbach's alpha >0.70 was considered satisfactory.31 As dimensions could be interpreted independently, the recommended inter-subscale correlation was <0.70 or less than the corresponding Cronbach's alpha coefficient.32,33 Number and frequency were calculated for categorical variables, and mean, standard deviation, median and interquartile range were calculated for continuous variables. Satisfaction scores and satisfaction rates for individual items and six dimensions were reported. The satisfaction rate was calculated in accordance with the following formulae from previous studies29: Satisfaction rate ¼
mean score 100% ðfor single itemÞ 5
overall satisfaction rate ¼
mean total score 100% ðfor dimensionÞ 5number of items
Patient responses were input into Epidata Version 3.1 (Centres for Disease Control and Prevention, Atlanta, GA, USA), and data analysis was performed using Statistical Package for the Social Sciences Version 17.0 (IBM Corp., Armonk, NY, USA).
Results
Fig. 1 e Flow chart depicting the process used to develop the Chinese outpatient satisfaction questionnaire (ChOPSQ).
In total, 5151 outpatients from 46 hospitals in Central South China were included in this study. The questionnaire had a high response rate, as the missing data rate for each item ranged from 0% to 0.04%. Patients completed the questionnaire within 5e15 min, and very few patients complained about the intelligibility of the questionnaire. Construct validity was tested by factor analysis, which eventually generated six dimensions and retained 19 items. The six dimensions explained 78.43% of the total variation between patients. The dimensions were interpreted as
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
4
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
waiting time (four items), service attitude (four items), quality of medical care (three items), quality of special services (three items), quality of environment (two items) and global assessment (three items). The loading values of the included items are listed in Table 1. All item-total correlations exceeded 0.80, which was greater than the standard of 0.50. The specific values for each item are displayed in Table 2. Cronbach's alpha coefficients for each dimension ranged from 0.774 to 0.895, satisfying the criterion of 0.7. The intersubscale correlation coefficients were all lower than the alpha coefficient of the corresponding scale, and the values were between 0.673 and 0.871. The results are shown in Table 3. The outcomes of the outpatient satisfaction survey are reported in Table 2. Patients were most satisfied with doctors' respect for the patient's privacy (mean 4.56, satisfaction rate 91.2%) and least satisfied with waiting for radiographic and/or laboratory examinations (mean 4.2, satisfaction rate 83.9%). Satisfaction with medical costs was relatively low (mean 4.2, satisfaction rate 84.5%). According to the results of the six dimensions (Fig. 2), patients were least satisfied with waiting time (86.8%), and most satisfied with the quality of medical care (90.1%). Patients' overall assessment of the quality of outpatient services was relatively low (87.6%) compared with the other dimensions.
Discussion The authors developed a new outpatient satisfaction questionnaire in Chinese (Ch-OPSQ), and used it to investigate samples from 46 public hospitals in Hunan Province. Patients were asked to rate their level of satisfaction with several important aspects of their outpatient experience. The Ch-OPSQ went through a strict development process and a series of procedures to test its reliability and validity.
The initial items were generated based on literature review and group discussion. In order to refine the items and improve the quality of the questionnaire, professional opinions and useful suggestions were acquired by interviewing experts and patients. Content validity should be guaranteed after these procedures. The structure of the questionnaire was established and construct validity was ensured using principal component exploratory factor analysis. Six dimensions were generated: waiting time, service attitude, quality of medical care, quality of special services, quality of environmental and global assessment. These aspects offered a good representation of the multidimensional and multiperspective construct of patient satisfaction with outpatient services. Compared with previous studies,21,22 this study found that the acceptability of the Ch-OPSQ was satisfactory, based on the extremely high response rate and low rate for missing data, and the interviewees indicated that the questionnaire was easy to understand and answer. In addition, this questionnaire was designed to be concise so that patients could complete it within 15 min. The reliability of the questionnaire was tested by a series of indexes. Cronbach's alpha coefficients of all dimensions were >0.7, which demonstrated excellent internal consistency. The values of inter-subscale correlation coefficients were all lower than their corresponding Cronbach's alpha coefficients, suggesting that the independence of each dimension was good. The item-total correlation, with all values >0.5, showed evidence for convergent validity. It also indicated good internal consistency because of the good strength of association between an item and the remainder of its scale. Based on the results of this survey, waiting time appears to be a major issue for outpatients in China, mainly due to the overwhelming number of outpatients, the lack of a primary care system in the community and the weak appointment registration system. The Chinese Government and hospitals need to improve the medical efficiency of outpatient
Table 1 e Factor analysis loadings of retaining items. Item number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Content
Factor*
Loading
Waiting for registration Waiting for a doctor Waiting to get medicine Waiting for radiographic and/or laboratory examination Doctor's attitude towards patients Nurse's attitude towards patients Medical technician's attitude towards patients Other hospital staff's attitude towards patients Patient's capability to choose doctors Doctor's respect for patient's privacy Professional and scientific skillfulness of doctors Self-service of outpatients (registration, paying fee, printing result of examinations) Convenience-for-patient service (simple outpatient, information inquiry) Guiding facilities Cleaner and security's service quality Cleanliness and comfort of the environment Reasonableness of medical costs Global assessment of doctors' medical morality Global assessment of outpatient service quality
1 1 1 1 2 2 2 2 3 3 3 4
0.781 0.573 0.719 0.620 0.586 0.605 0.656 0.634 0.737 0.729 0.551 0.747
4 4 5 5 6 6 6
0.627 0.577 0.753 0.623 0.689 0.665 0.596
*1: waiting time, 2: service attitude, 3: medical care quality, 4: special service quality, 5: environment quality, 6: global assessment.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
Item number 1 2 3 4 5 6 7 8 9 10 11 12
13
14 15 16 17 18 19
Content
Very dissatisfied (n/%)
Rather dissatisfied (n/%)
Neither satisfied nor dissatisfied (n/%)
Rather satisfied (n/%)
Very satisfied (n/%)
Mean ± standard deviation
Waiting for registration Waiting for a doctor Waiting to get medicine Waiting for radiographic and/or laboratory examination Doctor's attitude towards patients Nurse's attitude towards patients Medical technician's attitude towards patients Other hospital staff's attitude towards patients Patient's capability to choose doctors Doctor's respect for patient's privacy Professional and scientific skillfulness of doctors Self-service of outpatients (registration, paying fee, printing result of examinations) Convenience-for-patient service (simple outpatient, information inquiry) Guiding facilities Cleaner and security's service quality Cleanliness and comfort of the environment Reasonableness of medical costs Global assessment of doctors' medical morality Global assessment of outpatient service quality
42/0.8 42/0.8 37/0.7 94/1.8
125/2.4 109/2.1 103/2.0 237/4.6
605/11.7 565/11.0 665/12.9 800/15.5
1403/27.2 1433/27.8 1501/29.1 1448/28.1
2976/57.8 3002/58.3 2845/55.2 2570/49.9
4.39 4.41 4.36 4.2
24/0.5
54/1.0
465/9.0
1274/24.7
3334/64.7
34/0.7 26/0.5
64/1.2 53/1.0
532/10.3 570/11.1
1350/26.2 1475/28.6
28/0.5
91/1.8
671/13.0
30/0.6
65/13
19/0.4
Satisfaction rate (%)
Item-total correlation
(4e5) (4e5) (4e5) (4e5)
87.7 88.1 87.2 83.9
0.857 0.826 0.871 0.859
4.52 ± 0.74
5 (4e5)
90.4
0.849
3171/61.6 3072/58.8
4.47 ± 0.78 4.44 ± 0.77
5 (4e5) 5 (4e5)
89.3 88.8
0.871 0.881
1513/29.4
2847/55.3
4.37 ± 0.82
5 (4e5)
87.4
0.863
538/10.4
1235/24.0
3283/63.7
4.49 ± 0.78
5 (4e5)
89.8
0.880
39/0.8
396/7.7
1293/25.1
3404/66.1
4.56 ± 0.7
5 (4e5)
91.2
0.883
27/0.5
55/1.1
494/9.6
1469/28.5
3106/60.3
4.47 ± 0.75
5 (4e5)
89.4
0.877
20/0.4
65/1.3
535/10.4
1388/26.9
3143/61.0
4.47 ± 0.76
5 (4e5)
89.4
0.884
20/0.4
75/1.5
641/12.4
1441/28.0
2974/57.7
4.41 ± 0.79
5 (4e5)
88.2
0.891
21/0.4 28/0.5
81/1.6 56/1.1
533/10.3 636/12.3
1505/29.2 1669/32.4
3010/58.4 2762/53.6
4.44 ± 0.77 4.37 ± 0.78
5 (4e5) 5 (4e5)
88.7 87.5
0.867 0.903
25/0.5
65/1.3
610/11.8
1549/30.1
2902/56.3
4.41 ± 0.78
5 (4e5)
88.1
0.903
71/1.4 35/0.7
199/3.9 77/1.5
874/17.0 544/10.6
1352/26.2 1326/25.7
2655/51.5 3169/61.5
4.23 ± 0.96 4.46 ± 0.79
5 (4e5) 5 (4e5)
84.5 89.1
0.917 0.913
22/0.4
50/1.0
482/9.4
1599/31.0
2996/58.2
4.46 ± 0.74
5 (4e5)
89.1
0.912
± 0.85 ± 0.83 ± 0.83 ± 0.98
Median (interquartile range) 5 5 5 5
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
5
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
Table 2 e Satisfaction characteristics and item-total correlation.
6
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Table 3 e Internal consistency of final version of the questionnaire. Factors
Waiting time
Service attitude
Waiting time Service attitude
0.873 0.780
0.888
Quality of medical care Quality of special services Quality of environment Global assessment
0.730 0.716 0.684 0.770
0.806 0.765 0.740 0.817
Quality of medical care
Quality of special services
Quality of environment
Global assessment
0.853 0.729 0.673 0.756
0.855 0.755 0.796
0.774 0.724
0.895
Numbers in bold represent the Cronbach's alpha coefficients.
departments and reduce waiting times. Possible measures include optimizing treatment processes, developing appointed registration, balancing medical resources, and increasing spending on health care. To the authors' knowledge, this is the first standard general hospital OPSQ to be developed in China and tested among Chinese outpatients. Although researchers from other countries have developed several measurement tools to assess outpatient satisfaction, a questionnaire is needed specifically for the Chinese population. Developed countries have primary care systems in the community (such as family doctors and community clinics), appointment systems and medical insurance. As such, OPSQs for developed countries would evaluate the appointment system and would not assess medical expenses.21e25 Meanwhile, the definition of waiting time differs between developed countries and developing countries, as patients in developed countries mainly wait at home. However, in developing countries, there is no appointment system, and the volume of outpatients is much larger than in developed countries, so numerous patients wait in hospital. Thus, OPSQs in developing countries should pay more attention to evaluating the waiting experience of patients. A few developing countries have generated their own OPSQs.26e28 However, these OPSQs were not suitable for use by Chinese outpatients. The OPSQ in India26 did not evaluate medical expenses and waiting time for registration and examination, the OPSQ in Thailand27 was translated directly from a foreign OPSQ and validated, and the OPSQ in Tanzania28 was too long and complicated to be used directly in
Chinese outpatient departments. As such, there was an urgent need to develop a new OPSQ to assess outpatient satisfaction in China. This study had several strengths. Firstly, the questionnaire was developed specifically for outpatients, so it was more suitable for the measurement of outpatient satisfaction compared with non-specific questionnaires.1 Secondly, instead of evaluating a specific service or disease like other studies,30,34 the Ch-OPSQ evaluated all procedures and aspects related to outpatients. The versatility of the questionnaire was ensured, so it can be used widely in different types of hospitals. Finally, the Ch-OPSQ was surveyed with a large sample size (5151 outpatients in 46 medical centres), and found to have good acceptability, validity and reliability. Limitations of the present study should also be acknowledged. The primary limitation was that basic patient information was not recorded. This was because this study aimed to develop the questionnaire, and aimed to improve the response rate by providing anonymity. Secondly, testeretest reliability was not assessed in this study. Outpatients were mobile, and follow-up was difficult, particularly given the lack of personal information. Ensuring truthfulness of questionnaire responses is in conflict with the acquisition of basic information. In conclusion, the Ch-OPSQ was found to have good validity, reliability and acceptability. The items in this questionnaire differ from those in other studies. This instrument is recommended to assess outpatient experiences in China.
Author statements Ethical approval The research was approved by the Ethics Committee of Central South University.
Funding This work was supported by the Fundamental Research Funds for the Central Universities of Central South University (2014zzts070), Hunan Provincial Innovation Foundation for Postgraduates (CX2014B096).
Competing interests Fig. 2 e Satisfaction rates for six dimensions.
None declared.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
references
1. Kleefstra SM, Kool RB, Veldkamp C, Winters-van Der Meer A, Mens M, Blijham GH, de Haes J. A core questionnaire for the assessment of patient satisfaction in academic hospitals in The Netherlands: development and first results in a nationwide study. Qual Saf Health Care 2010;19:e24. 2. Drain M. Quality improvement in primary care and the importance of patient perceptions. J Ambul Care Manag 2001;24:30e46. 3. Coulter A. Can patients assess the quality of health care? patients' surveys should ask about real experiences of medical care. BMJ 2006;333:1. 4. Aharony L, Strasser S. Patient satisfaction: what we know about and what we still need to explore. Med Care Rev 1993;50:49e79. 5. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44:166e206. 6. Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: a review of current methodology. Anesth Analg 1998;87:1089e98. 7. Ferris LE. Health services research group. A guide to direct measures of patient satisfaction in clinical practice. Can Med Assoc J 1992;146:1727e31. 8. Bush T, Cherkin D, Barlow W. The impact of physician attitudes on patient satisfaction with care for low back pain. Arch Fam Med 1993;2:301. 9. Hudak PL, Wright JG. The characteristics of patient satisfaction measures. Spine 2000;25:3167e77. 10. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehab 1998;79:1122e8. 11. Nitse PS, Rushing V. Patient satisfaction: the new area of focus for the physician's office. Health Mark Q 1997;14:73e84. 12. Hall JA, Dornan MC. Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels. Soc Sci Med 1988;27:637e44. 13. Rivkin MO, Bush PJ. The satisfaction continuum in health care: consumer and provider preferences. New York: Prodist. In: Mushkin EJ, editor. Consumer incentives for health care; 1974. 14. Lochman JE. Factors related to patients' satisfaction with their medical care. J Commun Health 1983;9:91e109. 15. Linder-Pelz S, Struening EL. The multidimensionality of patient satisfaction with a clinic visit. J Community Health 1985;10:42e54. 16. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, Thomas H. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess 2002;6(32). 17. Lilford RJ, Brown CA, Nicholl J. Use of process measures to monitor the quality of clinical practice. BMJ 2007;335:648. 18. Sitzia J. How valid and reliable are patient satisfaction data? an analysis of 195 studies. Int J Qual Health Care 1999;11:319e28.
7
19. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med 1997;45:1829e43. 20. Chapko MK, Bergner M, Green K, Beach B, Milgrom P, Skalabrin N. Development and validation of a measure of dental patient satisfaction. Med Care 1985;23:39e49. 21. Aletras VH, Papadopoulos EA, Niakas DA. Development and preliminary validation of a Greek-language outpatient satisfaction questionnaire with principal components and multi-trait analyses. BMC Health Serv Res 2006;6:66. 22. Garratt AM, Bjærtnes ØA, Krogstad U, Gulbrandsen P. The outpatient experiences questionnaire (OPEQ): data quality, reliability, and validity in patients attending 52 Norwegian hospitals. Qual Saf Health Care 2005;14:433e7. 23. Kleefstra S, Kool R, Zandbelt L, de Haes J. An instrument assessing patient satisfaction with day care in hospitals. BMC Health Serv Res 2012;12:125. 24. Margaritis E, Katharaki M, Katharakis G. Exceeding parents' expectations in ear-nose-throat outpatient facilities: the development and analysis of a questionnaire. Eval Program Plann 2012;35:246e55. 25. Rahmqvist M, Bara AC. Patient characteristics and quality dimensions related to patient satisfaction. Int J Qual Health Care 2010;22:86e92. 26. Goel S, Sharma D, Singh A. Development and validation of a patient satisfaction questionnaire for outpatients attending health centres in North Indian cities. J Health Serv Res Policy 2014;19:85e93. 27. Jaturapatporn D, Hathirat S, Manataweewat B, Dellow AC, Leelaharattanarak S, Sirimothya S, Dellow J, Udomsubpayakul U. Reliability and validity of a Thai version of the general practice assessment questionnaire (GPAQ). J Med Assoc Thai 2006;89:1491e6. 28. Khamis K, Njau B. Patients' level of satisfaction on quality of health care at Mwananyamala hospital in Dar es Salaam, Tanzania. BMC Health Serv Res 2014;14:400. 29. Mohd A, Chakravarty A. Patient satisfaction with services of the outpatient department. Med J Armed Forces India 2014;70:237e42. 30. Bitzer E, Volkmer S, Petrucci M, Weissenrieder N, Dierks M. Patient satisfaction in pediatric outpatient settings from the parents' perspective e the Child ZAP: a psychometrically validated standardized questionnaire. BMC Health Serv Res 2012;12:347. 31. Nunnally JC, Bernstein IH. Psychometric theory. New York: McGraw-Hill; 1994. 32. Labarere J, Francois P, Auquier P, Robert C, Fourny M. Development of a French inpatient satisfaction questionnaire. Int J Qual Health Care 2001;13:99e108. 33. Carey RG, Seibert JH. A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care 1993;31:834e45. 34. Beattie PF, Pinto MB, Nelson MK, Nelson R. Patient satisfaction with outpatient physical therapy: instrument validation. Phys Ther 2002;82:557e65.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
Available online at www.sciencedirect.com
Public Health journal homepage: www.elsevier.com/puhe
Original Research
Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients J. Wei a, L. Shen b, H.-B. Yang a, J.-B. Qin a, W. Huang a, J.-J. Zhang a, Q.-Y. Gong a, X.-X. Li a, T.-B. Yang a,* a
Department of Epidemiology and Health Statistics, School of Public Health, Central South University, Changsha, Hunan Province, 410008, China b Xiangya Medical School, Central South University, Changsha, Hunan Province, China
article info
abstract
Article history:
Objective: To develop and test a reliable and practical self-administrated questionnaire in
Received 11 June 2014
Chinese to evaluate outpatient satisfaction in China.
Received in revised form
Design: In order to ensure content validity, the 19 items of this questionnaire were gener-
16 December 2014
ated based on literature review, interview and group discussion. Acceptability was tested
Accepted 6 March 2015
by the percentage of missing item responses and the time and operability of administra-
Available online xxx
tion. Construct validity was evaluated using principal component exploratory factor analysis and item-total correlations. The reliability of the questionnaire was assessed
Keywords:
using Cronbach's alpha coefficient and inter-subscale correlation.
Outpatient
Setting: This study was conducted in 46 public general hospitals that report directly to the
Satisfaction
Provincial Health Department in Hunan Province, China.
Questionnaire
Subjects: In total, 5151 outpatients were selected at random and asked to complete the
Medical care quality
questionnaire after they finished their medical procedures in the outpatient area. Results: The extremely low rate of missing data (0e0.04%) suggested good acceptability of the questionnaire. Factor analysis generated six dimensions, and all item-total coefficients were >0.8. Cronbach's alpha exceeded 0.7 for all dimensions, and the inter-subscale correlation coefficients were all lower than the Cronbach's alpha coefficients of the corresponding scale. According to the results, outpatients were least satisfied with waiting time (86.8%) and most satisfied with the quality of medical care (90.1%). Conclusion: The Chinese outpatient satisfaction questionnaire has good acceptability, validity and reliability. It was effective and efficient for measuring outpatient satisfaction in a Chinese population. © 2015 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Tel.: þ86 13517497107. E-mail address: [email protected] (T.-B. Yang). http://dx.doi.org/10.1016/j.puhe.2015.03.004 0033-3506/© 2015 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
2
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Introduction It is crucial for a humane healthcare system to consider patients' needs and wishes, and patients' perceptions of medical processes can provide unique and valuable information to improve the quality of care. Many studies have suggested that patient satisfaction should be regarded as an important indicator for the quality of care.1e4 There are several motives for evaluating patient satisfaction. Firstly, although the quality of medical services can be evaluated from multiple perspectives (e.g. patient, provider, insurer, etc.), patients should still be considered as the ultimate estimator of the quality of care.5 Secondly, patient satisfaction represents an assessment of their medical experience based on their values, perceptions and interactions with the healthcare environment. Thirdly, patients' thoughts can affect their future behaviours,6 and patients with differing levels of satisfaction may behave differently.7 Patients with a high level of satisfaction tend to be more cooperative with medical staff and the recommended treatment plans.8e12 In contrast, patients with a low level of satisfaction may have less effective medical services, refuse to seek care, or disobey physicians' suggestions and prescribed course of treatment.13 It has been shown previously that patients' health status, frequency and length of hospital stay, compliance with medical suggestions and other medical outcomes could be greatly influenced by their own experience and satisfaction.4,9,14 Lastly, healthcare providers can improve the effectiveness of medical services based on feedback regarding patient satisfaction.2 Patient satisfaction is generally subjective and intangible, so researchers often consider it as an abstract and multidimensional phenomenon.9,12,15 It can only be measured by indirect measurement tools such as suggestion boxes, formal complaints, qualitative methods, audits and self-report questionnaires.16 However, many studies on patient satisfaction have been criticized for poor validity and reliability due to shortcomings in their measurement approaches,17,18 and some studies have not included a clear definition of patient satisfaction.19 The need to develop and test a new satisfaction questionnaire, rather than use a measurement tool that has been published previously, is because of the unique nature of China's outpatient care service. Most questionnaires to measure patient satisfaction were developed in Western countries,20e25 and they may not be suitable for use in developing countries due to differences in medical conditions and facilities. In developed countries, most patients have an appointment before they attend a hospital, and medical insurance may cover their medical expenses. As such, they generally experience shorter waiting times and lower costs compared with patients in developing countries. Several developing countries, such as India, Thailand and Tanzania,26e29 have developed outpatient satisfaction questionnaires (OPSQs). However, there are several differences between China and other developing countries. First, the organization of Chinese general hospital outpatient services is very complicated. Certain services, such as self-services and convenience-for-patients services, may be unique to China, so items related to these aspects need to be included in the
Chinese OPSQ (CH-OPSQ). In addition, the daily outpatient volume is extremely high due to the large Chinese population. Lack of a primary care system in the community results in more patients attending general hospitals. Waiting time, waiting environment and facilities in the waiting room are crucial in patient satisfaction. Furthermore, most medical insurance in China does not cover outpatient expenses, so the costs may have a greater impact on patient satisfaction in China compared with other countries. Therefore, it is essential to provide a new measurement tool to evaluate outpatient satisfaction in China. As such, the objective of this study was to develop an outpatient satisfaction questionnaire for the Chinese population, and to test its reliability, validity and acceptability through a large multicentre sample.
Method Questionnaire development Three members of the research team identified items for inclusion in the Ch-OPSQ from previous studies. A literature search was conducted in MEDLINE and EMBASE between June and July 2012. The following keywords were used in the literature search: ‘outpatient’, ‘satisfaction’, ‘hospital’ and ‘questionnaire’. Relevant studies on outpatient satisfaction were screened and useful information was extracted to build an item pool. An expert panel, including two professors, two experts from the government health department and four postgraduate students, was established to develop an item pool for the Ch-OPSQ (65 items were included). The research team interviewed five patients, five administrators from different hospitals and five officers from the government health department, selected at random from Hunan Province. They were asked to rate the importance of each item, give their opinions and suggestions about the items in the item pool, and comment on the relevance of the issues covered and comprehensibility, including the response options. All the consultation sessions were conducted via faceto-face interview or e-mail. To ensure the content validity of the questionnaire, the interviewees were also asked to provide other potential sources of satisfaction and assign significance to individual items. After the interviews, qualitative analysis was conducted by reviewing the suggestions from the interviews, and refining the words and content of the questions according to the interviewees' feedback. The researchers reached consensus on the items and response options to be listed in the Ch-OPSQ. A draft version of the Ch-OPSQ including 22 items was created, and a pilot survey was undertaken to assess the questionnaire. According to the results of the pilot study, some items were eliminated because of a high non-response rate, and some items were amended if the responses showed multiple answers or poor variability. The final version of the Ch-OPSQ contains 19 items, covering nearly all aspects of outpatient medical care. The questionnaire asks patients to rate their feelings about each statement on a five-point Liket scale: very satisfied (5), relatively satisfied (4), neither satisfied nor dissatisfied (3), relatively dissatisfied (2) and very dissatisfied (1).
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
3
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Fig. 1 shows the questionnaire development process.
approved by the Ethics Committee of Central South University.
Participants and study design
Statistical analysis
The study was conducted between October and December 2012 in 46 hospitals in Hunan Province, all of which report directly to the provincial department of health. The plan was to select 120 outpatients at random from each hospital (total 7680 subjects) to answer the questionnaire. Before the survey started, the research team recruited 30 investigators who were not employed by the hospitals or the provincial department of health. After professional training, they were assigned to the hospitals and asked to perform the survey in outpatient areas. Patients who were conscious and willing to participate in this survey were selected at random in different departments. The investigators gave the Ch-OPSQ to patients after their medical procedures, and collected it upon completion. Strict quality control rules were executed during the survey. Investigators were asked to sign their names on the questionnaire and take responsibility for their survey questionnaires. Every received questionnaire was checked and verified by the investigators and supervisors. Questionnaires with more than 20% of missing data were excluded. The research was
Feasibility and acceptability were examined using the percentage of missing item responses, interviewer-reported acceptability, and the time and operability of administration. In accordance with previous studies,21e24,26,28,30 principal component exploratory factor analysis by varimax rotation was used to establish the structure and test the construct validity of the Ch-OPSQ. Factors were generated with an eigenvalue >1 or a cumulative contribution rate >70%. An individual item had to have a loading 0.60 on one factor to be retained. Items with poor factor loading (<0.60) were eliminated from the questionnaire. The item-total correlation coefficient was calculated to evaluate construct validity. To incorporate an item into a related scale, the corresponding item-total correlation coefficient had to be 0.50.21 Reliability and internal consistency were evaluated using Cronbach's alpha coefficient and inter-subscale correlation. Cronbach's alpha coefficient tested the overall correlation between items within a scale. Cronbach's alpha >0.70 was considered satisfactory.31 As dimensions could be interpreted independently, the recommended inter-subscale correlation was <0.70 or less than the corresponding Cronbach's alpha coefficient.32,33 Number and frequency were calculated for categorical variables, and mean, standard deviation, median and interquartile range were calculated for continuous variables. Satisfaction scores and satisfaction rates for individual items and six dimensions were reported. The satisfaction rate was calculated in accordance with the following formulae from previous studies29: Satisfaction rate ¼
mean score 100% ðfor single itemÞ 5
overall satisfaction rate ¼
mean total score 100% ðfor dimensionÞ 5number of items
Patient responses were input into Epidata Version 3.1 (Centres for Disease Control and Prevention, Atlanta, GA, USA), and data analysis was performed using Statistical Package for the Social Sciences Version 17.0 (IBM Corp., Armonk, NY, USA).
Results
Fig. 1 e Flow chart depicting the process used to develop the Chinese outpatient satisfaction questionnaire (ChOPSQ).
In total, 5151 outpatients from 46 hospitals in Central South China were included in this study. The questionnaire had a high response rate, as the missing data rate for each item ranged from 0% to 0.04%. Patients completed the questionnaire within 5e15 min, and very few patients complained about the intelligibility of the questionnaire. Construct validity was tested by factor analysis, which eventually generated six dimensions and retained 19 items. The six dimensions explained 78.43% of the total variation between patients. The dimensions were interpreted as
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
4
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
waiting time (four items), service attitude (four items), quality of medical care (three items), quality of special services (three items), quality of environment (two items) and global assessment (three items). The loading values of the included items are listed in Table 1. All item-total correlations exceeded 0.80, which was greater than the standard of 0.50. The specific values for each item are displayed in Table 2. Cronbach's alpha coefficients for each dimension ranged from 0.774 to 0.895, satisfying the criterion of 0.7. The intersubscale correlation coefficients were all lower than the alpha coefficient of the corresponding scale, and the values were between 0.673 and 0.871. The results are shown in Table 3. The outcomes of the outpatient satisfaction survey are reported in Table 2. Patients were most satisfied with doctors' respect for the patient's privacy (mean 4.56, satisfaction rate 91.2%) and least satisfied with waiting for radiographic and/or laboratory examinations (mean 4.2, satisfaction rate 83.9%). Satisfaction with medical costs was relatively low (mean 4.2, satisfaction rate 84.5%). According to the results of the six dimensions (Fig. 2), patients were least satisfied with waiting time (86.8%), and most satisfied with the quality of medical care (90.1%). Patients' overall assessment of the quality of outpatient services was relatively low (87.6%) compared with the other dimensions.
Discussion The authors developed a new outpatient satisfaction questionnaire in Chinese (Ch-OPSQ), and used it to investigate samples from 46 public hospitals in Hunan Province. Patients were asked to rate their level of satisfaction with several important aspects of their outpatient experience. The Ch-OPSQ went through a strict development process and a series of procedures to test its reliability and validity.
The initial items were generated based on literature review and group discussion. In order to refine the items and improve the quality of the questionnaire, professional opinions and useful suggestions were acquired by interviewing experts and patients. Content validity should be guaranteed after these procedures. The structure of the questionnaire was established and construct validity was ensured using principal component exploratory factor analysis. Six dimensions were generated: waiting time, service attitude, quality of medical care, quality of special services, quality of environmental and global assessment. These aspects offered a good representation of the multidimensional and multiperspective construct of patient satisfaction with outpatient services. Compared with previous studies,21,22 this study found that the acceptability of the Ch-OPSQ was satisfactory, based on the extremely high response rate and low rate for missing data, and the interviewees indicated that the questionnaire was easy to understand and answer. In addition, this questionnaire was designed to be concise so that patients could complete it within 15 min. The reliability of the questionnaire was tested by a series of indexes. Cronbach's alpha coefficients of all dimensions were >0.7, which demonstrated excellent internal consistency. The values of inter-subscale correlation coefficients were all lower than their corresponding Cronbach's alpha coefficients, suggesting that the independence of each dimension was good. The item-total correlation, with all values >0.5, showed evidence for convergent validity. It also indicated good internal consistency because of the good strength of association between an item and the remainder of its scale. Based on the results of this survey, waiting time appears to be a major issue for outpatients in China, mainly due to the overwhelming number of outpatients, the lack of a primary care system in the community and the weak appointment registration system. The Chinese Government and hospitals need to improve the medical efficiency of outpatient
Table 1 e Factor analysis loadings of retaining items. Item number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Content
Factor*
Loading
Waiting for registration Waiting for a doctor Waiting to get medicine Waiting for radiographic and/or laboratory examination Doctor's attitude towards patients Nurse's attitude towards patients Medical technician's attitude towards patients Other hospital staff's attitude towards patients Patient's capability to choose doctors Doctor's respect for patient's privacy Professional and scientific skillfulness of doctors Self-service of outpatients (registration, paying fee, printing result of examinations) Convenience-for-patient service (simple outpatient, information inquiry) Guiding facilities Cleaner and security's service quality Cleanliness and comfort of the environment Reasonableness of medical costs Global assessment of doctors' medical morality Global assessment of outpatient service quality
1 1 1 1 2 2 2 2 3 3 3 4
0.781 0.573 0.719 0.620 0.586 0.605 0.656 0.634 0.737 0.729 0.551 0.747
4 4 5 5 6 6 6
0.627 0.577 0.753 0.623 0.689 0.665 0.596
*1: waiting time, 2: service attitude, 3: medical care quality, 4: special service quality, 5: environment quality, 6: global assessment.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
Item number 1 2 3 4 5 6 7 8 9 10 11 12
13
14 15 16 17 18 19
Content
Very dissatisfied (n/%)
Rather dissatisfied (n/%)
Neither satisfied nor dissatisfied (n/%)
Rather satisfied (n/%)
Very satisfied (n/%)
Mean ± standard deviation
Waiting for registration Waiting for a doctor Waiting to get medicine Waiting for radiographic and/or laboratory examination Doctor's attitude towards patients Nurse's attitude towards patients Medical technician's attitude towards patients Other hospital staff's attitude towards patients Patient's capability to choose doctors Doctor's respect for patient's privacy Professional and scientific skillfulness of doctors Self-service of outpatients (registration, paying fee, printing result of examinations) Convenience-for-patient service (simple outpatient, information inquiry) Guiding facilities Cleaner and security's service quality Cleanliness and comfort of the environment Reasonableness of medical costs Global assessment of doctors' medical morality Global assessment of outpatient service quality
42/0.8 42/0.8 37/0.7 94/1.8
125/2.4 109/2.1 103/2.0 237/4.6
605/11.7 565/11.0 665/12.9 800/15.5
1403/27.2 1433/27.8 1501/29.1 1448/28.1
2976/57.8 3002/58.3 2845/55.2 2570/49.9
4.39 4.41 4.36 4.2
24/0.5
54/1.0
465/9.0
1274/24.7
3334/64.7
34/0.7 26/0.5
64/1.2 53/1.0
532/10.3 570/11.1
1350/26.2 1475/28.6
28/0.5
91/1.8
671/13.0
30/0.6
65/13
19/0.4
Satisfaction rate (%)
Item-total correlation
(4e5) (4e5) (4e5) (4e5)
87.7 88.1 87.2 83.9
0.857 0.826 0.871 0.859
4.52 ± 0.74
5 (4e5)
90.4
0.849
3171/61.6 3072/58.8
4.47 ± 0.78 4.44 ± 0.77
5 (4e5) 5 (4e5)
89.3 88.8
0.871 0.881
1513/29.4
2847/55.3
4.37 ± 0.82
5 (4e5)
87.4
0.863
538/10.4
1235/24.0
3283/63.7
4.49 ± 0.78
5 (4e5)
89.8
0.880
39/0.8
396/7.7
1293/25.1
3404/66.1
4.56 ± 0.7
5 (4e5)
91.2
0.883
27/0.5
55/1.1
494/9.6
1469/28.5
3106/60.3
4.47 ± 0.75
5 (4e5)
89.4
0.877
20/0.4
65/1.3
535/10.4
1388/26.9
3143/61.0
4.47 ± 0.76
5 (4e5)
89.4
0.884
20/0.4
75/1.5
641/12.4
1441/28.0
2974/57.7
4.41 ± 0.79
5 (4e5)
88.2
0.891
21/0.4 28/0.5
81/1.6 56/1.1
533/10.3 636/12.3
1505/29.2 1669/32.4
3010/58.4 2762/53.6
4.44 ± 0.77 4.37 ± 0.78
5 (4e5) 5 (4e5)
88.7 87.5
0.867 0.903
25/0.5
65/1.3
610/11.8
1549/30.1
2902/56.3
4.41 ± 0.78
5 (4e5)
88.1
0.903
71/1.4 35/0.7
199/3.9 77/1.5
874/17.0 544/10.6
1352/26.2 1326/25.7
2655/51.5 3169/61.5
4.23 ± 0.96 4.46 ± 0.79
5 (4e5) 5 (4e5)
84.5 89.1
0.917 0.913
22/0.4
50/1.0
482/9.4
1599/31.0
2996/58.2
4.46 ± 0.74
5 (4e5)
89.1
0.912
± 0.85 ± 0.83 ± 0.83 ± 0.98
Median (interquartile range) 5 5 5 5
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
5
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
Table 2 e Satisfaction characteristics and item-total correlation.
6
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
Table 3 e Internal consistency of final version of the questionnaire. Factors
Waiting time
Service attitude
Waiting time Service attitude
0.873 0.780
0.888
Quality of medical care Quality of special services Quality of environment Global assessment
0.730 0.716 0.684 0.770
0.806 0.765 0.740 0.817
Quality of medical care
Quality of special services
Quality of environment
Global assessment
0.853 0.729 0.673 0.756
0.855 0.755 0.796
0.774 0.724
0.895
Numbers in bold represent the Cronbach's alpha coefficients.
departments and reduce waiting times. Possible measures include optimizing treatment processes, developing appointed registration, balancing medical resources, and increasing spending on health care. To the authors' knowledge, this is the first standard general hospital OPSQ to be developed in China and tested among Chinese outpatients. Although researchers from other countries have developed several measurement tools to assess outpatient satisfaction, a questionnaire is needed specifically for the Chinese population. Developed countries have primary care systems in the community (such as family doctors and community clinics), appointment systems and medical insurance. As such, OPSQs for developed countries would evaluate the appointment system and would not assess medical expenses.21e25 Meanwhile, the definition of waiting time differs between developed countries and developing countries, as patients in developed countries mainly wait at home. However, in developing countries, there is no appointment system, and the volume of outpatients is much larger than in developed countries, so numerous patients wait in hospital. Thus, OPSQs in developing countries should pay more attention to evaluating the waiting experience of patients. A few developing countries have generated their own OPSQs.26e28 However, these OPSQs were not suitable for use by Chinese outpatients. The OPSQ in India26 did not evaluate medical expenses and waiting time for registration and examination, the OPSQ in Thailand27 was translated directly from a foreign OPSQ and validated, and the OPSQ in Tanzania28 was too long and complicated to be used directly in
Chinese outpatient departments. As such, there was an urgent need to develop a new OPSQ to assess outpatient satisfaction in China. This study had several strengths. Firstly, the questionnaire was developed specifically for outpatients, so it was more suitable for the measurement of outpatient satisfaction compared with non-specific questionnaires.1 Secondly, instead of evaluating a specific service or disease like other studies,30,34 the Ch-OPSQ evaluated all procedures and aspects related to outpatients. The versatility of the questionnaire was ensured, so it can be used widely in different types of hospitals. Finally, the Ch-OPSQ was surveyed with a large sample size (5151 outpatients in 46 medical centres), and found to have good acceptability, validity and reliability. Limitations of the present study should also be acknowledged. The primary limitation was that basic patient information was not recorded. This was because this study aimed to develop the questionnaire, and aimed to improve the response rate by providing anonymity. Secondly, testeretest reliability was not assessed in this study. Outpatients were mobile, and follow-up was difficult, particularly given the lack of personal information. Ensuring truthfulness of questionnaire responses is in conflict with the acquisition of basic information. In conclusion, the Ch-OPSQ was found to have good validity, reliability and acceptability. The items in this questionnaire differ from those in other studies. This instrument is recommended to assess outpatient experiences in China.
Author statements Ethical approval The research was approved by the Ethics Committee of Central South University.
Funding This work was supported by the Fundamental Research Funds for the Central Universities of Central South University (2014zzts070), Hunan Provincial Innovation Foundation for Postgraduates (CX2014B096).
Competing interests Fig. 2 e Satisfaction rates for six dimensions.
None declared.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004
p u b l i c h e a l t h x x x ( 2 0 1 5 ) 1 e7
references
1. Kleefstra SM, Kool RB, Veldkamp C, Winters-van Der Meer A, Mens M, Blijham GH, de Haes J. A core questionnaire for the assessment of patient satisfaction in academic hospitals in The Netherlands: development and first results in a nationwide study. Qual Saf Health Care 2010;19:e24. 2. Drain M. Quality improvement in primary care and the importance of patient perceptions. J Ambul Care Manag 2001;24:30e46. 3. Coulter A. Can patients assess the quality of health care? patients' surveys should ask about real experiences of medical care. BMJ 2006;333:1. 4. Aharony L, Strasser S. Patient satisfaction: what we know about and what we still need to explore. Med Care Rev 1993;50:49e79. 5. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44:166e206. 6. Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: a review of current methodology. Anesth Analg 1998;87:1089e98. 7. Ferris LE. Health services research group. A guide to direct measures of patient satisfaction in clinical practice. Can Med Assoc J 1992;146:1727e31. 8. Bush T, Cherkin D, Barlow W. The impact of physician attitudes on patient satisfaction with care for low back pain. Arch Fam Med 1993;2:301. 9. Hudak PL, Wright JG. The characteristics of patient satisfaction measures. Spine 2000;25:3167e77. 10. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehab 1998;79:1122e8. 11. Nitse PS, Rushing V. Patient satisfaction: the new area of focus for the physician's office. Health Mark Q 1997;14:73e84. 12. Hall JA, Dornan MC. Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels. Soc Sci Med 1988;27:637e44. 13. Rivkin MO, Bush PJ. The satisfaction continuum in health care: consumer and provider preferences. New York: Prodist. In: Mushkin EJ, editor. Consumer incentives for health care; 1974. 14. Lochman JE. Factors related to patients' satisfaction with their medical care. J Commun Health 1983;9:91e109. 15. Linder-Pelz S, Struening EL. The multidimensionality of patient satisfaction with a clinic visit. J Community Health 1985;10:42e54. 16. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, Thomas H. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess 2002;6(32). 17. Lilford RJ, Brown CA, Nicholl J. Use of process measures to monitor the quality of clinical practice. BMJ 2007;335:648. 18. Sitzia J. How valid and reliable are patient satisfaction data? an analysis of 195 studies. Int J Qual Health Care 1999;11:319e28.
7
19. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med 1997;45:1829e43. 20. Chapko MK, Bergner M, Green K, Beach B, Milgrom P, Skalabrin N. Development and validation of a measure of dental patient satisfaction. Med Care 1985;23:39e49. 21. Aletras VH, Papadopoulos EA, Niakas DA. Development and preliminary validation of a Greek-language outpatient satisfaction questionnaire with principal components and multi-trait analyses. BMC Health Serv Res 2006;6:66. 22. Garratt AM, Bjærtnes ØA, Krogstad U, Gulbrandsen P. The outpatient experiences questionnaire (OPEQ): data quality, reliability, and validity in patients attending 52 Norwegian hospitals. Qual Saf Health Care 2005;14:433e7. 23. Kleefstra S, Kool R, Zandbelt L, de Haes J. An instrument assessing patient satisfaction with day care in hospitals. BMC Health Serv Res 2012;12:125. 24. Margaritis E, Katharaki M, Katharakis G. Exceeding parents' expectations in ear-nose-throat outpatient facilities: the development and analysis of a questionnaire. Eval Program Plann 2012;35:246e55. 25. Rahmqvist M, Bara AC. Patient characteristics and quality dimensions related to patient satisfaction. Int J Qual Health Care 2010;22:86e92. 26. Goel S, Sharma D, Singh A. Development and validation of a patient satisfaction questionnaire for outpatients attending health centres in North Indian cities. J Health Serv Res Policy 2014;19:85e93. 27. Jaturapatporn D, Hathirat S, Manataweewat B, Dellow AC, Leelaharattanarak S, Sirimothya S, Dellow J, Udomsubpayakul U. Reliability and validity of a Thai version of the general practice assessment questionnaire (GPAQ). J Med Assoc Thai 2006;89:1491e6. 28. Khamis K, Njau B. Patients' level of satisfaction on quality of health care at Mwananyamala hospital in Dar es Salaam, Tanzania. BMC Health Serv Res 2014;14:400. 29. Mohd A, Chakravarty A. Patient satisfaction with services of the outpatient department. Med J Armed Forces India 2014;70:237e42. 30. Bitzer E, Volkmer S, Petrucci M, Weissenrieder N, Dierks M. Patient satisfaction in pediatric outpatient settings from the parents' perspective e the Child ZAP: a psychometrically validated standardized questionnaire. BMC Health Serv Res 2012;12:347. 31. Nunnally JC, Bernstein IH. Psychometric theory. New York: McGraw-Hill; 1994. 32. Labarere J, Francois P, Auquier P, Robert C, Fourny M. Development of a French inpatient satisfaction questionnaire. Int J Qual Health Care 2001;13:99e108. 33. Carey RG, Seibert JH. A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care 1993;31:834e45. 34. Beattie PF, Pinto MB, Nelson MK, Nelson R. Patient satisfaction with outpatient physical therapy: instrument validation. Phys Ther 2002;82:557e65.
Please cite this article in press as: Wei J, et al., Development and validation of a Chinese outpatient satisfaction questionnaire: evidence from 46 public general hospitals and 5151 outpatients, Public Health (2015), http://dx.doi.org/10.1016/ j.puhe.2015.03.004