- Email: [email protected]
The Impact of Demographic Characteristics on Nonresponse in an Ambulatory Patient Satisfaction Survey
The Joint Commission Journal on Quality and Patient Safety Performance Measures
The Impact of Demographic Characteristics on Nonresponse in an Ambulatory Patient Satisfaction Survey Christy K. Boscardin, PhD; Ralph Gonzales, MD, MSPH
P
atient satisfaction surveys are increasingly becoming one of the most important and widely used indicators of health care quality1–3 and are increasingly being used in pay-for-performance schemes. With the US federal mandate for the Physician Compare initiative,4 which will provide information to patients about the quality of care, including patient experience, under Section 10331 of the Patient Protection and Affordable Care Act (Pub L 111148) of 2010,5 the emphasis on patient satisfaction data will be an even more critical component of practice- and systems-based improvement strategies in the future. The reliance on the Consumer Assessment of Healthcare Providers and Systems (CAHPS) data by the federal government to compare health care quality across the health plans serving Medicare beneficiaries is a prime example of the current emphasis on patient satisfaction data as a primary source of health care quality.6 Previous studies have also shown that patient satisfaction is associated with better health outcomes and greater adherence to treatment recommendations and medication.7,8 The positive link between patient satisfaction and compliance has been one of the key arguments for the utility of patient satisfaction surveys. With health care reform and the rise of consumerist ethos in the health care industry, patient satisfaction data will remain an important aspect of health care quality. Despite increasing use of patient satisfaction data for compensation and comparisons across institutions, medical groups, and providers, the low response rate and potential bias of the respondents pose a significant threat to validity of the data. With the response rate for surveys typically ranging from 25% to 50%,9 the potential for response bias in the data is substantial. In a previous study of hospitalized patients, satisfaction scores were significantly associated with certain demographic characteristics such as age, sex, and race (ethnicity).10–13 Despite these concerns, few studies have examined the potential bias and the impact of nonresponse, and studies examining the impact of nonresponse have been limited to hospital settings.3,12,14,15 In this study, we compared the demographic profiles of
Article-at-a-Glance Background: Despite the increasing use of patient satisfac-
tion data for compensation and comparison of performance, low response rate and potential bias of the respondents pose a significant threat to validity of the data. The demographic profiles of respondents and nonrespondents to a patient satisfaction survey in the ambulatory care setting were compared to explore the impact of nonresponse bias. Methods: Patient satisfaction survey data were collected from October through December 2010 for outpatient facilities at a large academic medical center. The association between respondent characteristics and satisfaction ratings on three dimensions of the clinical care process—(1) interpersonal communication (clarity of language), (2) service delivery (overall care during visit), and (3) likelihood of recommending practice to others—were assessed with bivariate and multivariate linear regression. Weighted analyses were performed to examine the impact of nonresponse. Results: Surveys were mailed to 15,549 patients, of which 4,952 (32%) were returned. Respondents had greater proportions of elderly, female, and English speakers. Bivariate analyses showed significant difference in satisfaction ratings by age, language, and insurance type. Multivariate regression analysis showed significant confounding across variables. On the basis of the calculated weighted means, mean satisfaction ratings were discrepant for language and age; however, the overall satisfaction ratings for each dimension were minimally affected. Conclusion: Nonresponse rates and satisfaction ratings differed by age, language, and insurance type. However, if it is assumed that nonrespondents within these demographic groups have similar satisfaction ratings as respondents, then nonresponse levels appear to have minimal effects on overall satisfaction ratings.
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
123
The Joint Commission Journal on Quality and Patient Safety respondents and nonrespondents to a patient satisfaction survey administered in an ambulatory care setting to determine whether systematic differences exist between the two groups.
Methods SURVEY DATA Patient satisfaction survey data were collected by a health care performance improvement vendor for a one-quarter period (October 2010–December 2010) for outpatient facilities at a large academic medical center. These surveys are conducted continuously, and data were aggregated by quarter for monitoring and reporting purposes. The survey was mailed by the vendor to a stratified random sample of 15,549 patients who completed ambulatory visits at a large academic medical center during the observation period. Completed surveys were returned by 4,952 respondents (response rate, 32%). The 35-item survey (Appendix 1, available in online article) includes items on the patient’s satisfaction with registration, facility, test/treatment, personal issues (sensitivity to needs), physician, and overall quality assessment of his or her care. The following demographic information was available for both respondents and nonrespondents: (1) age, (2) sex, (3) language of the survey administered, and (4) insurance type. The demographic data for the analysis were limited to the information collected during registration because these data were available for both groups. Race and ethnicity information were not available for the nonrespondents because this information is collected as part of the survey administration.
the demographic characteristics (age, sex, language, and insurance type) and satisfaction ratings separately. Then we conducted multivariable linear regression analyses to examine the independent effect of the demographic characteristics on the satisfaction ratings for each of the three specific questions. Finally, to examine the impact of nonresponse on reported satisfaction ratings, we performed weighted analyses, weighting by the inverse probability of response. Specifically, we calculated weights through a poststratification approach to adjust the characteristics of the response sample to those of the total sample (including the nonresponse sample) by using the four key demographic variables (language, sex, age, and insurance type). The nonresponse weights were first calculated using the package in Stata version 11 (StataCorp LP, College Station, Texas). We performed subsequent analyses with these weights using Stata’s pweights (probability weights, also known as sampling weights).
Results RESPONDENT AND NONRESPONDENT PROFILES The profiles of respondent and nonrespondents were compared on the four key demographic variables. We found substantial differences between respondents and nonrespondent profiles on the basis of age, sex, language, and insurance type. As shown in Table 1 (page 125), a greater proportion of respondents than nonrespondents were elderly, female, and English speakers (Table 1, p < .001 for all comparisons). A greater proportion of respondents reported Medicare insurance, and a smaller proportion reported Medicaid insurance (p < .001).
ASSOCIATION BETWEEN THE DEMOGRAPHIC CHARACTERISTICS AND SATISFACTION RATINGS Descriptive statistics (relative frequency) were calculated for de- ACROSS THE THREE DIMENSIONS Data Analysis
mographic data. The chi-square test for categorical data between respondents and nonrespondents was used to compare the proportional difference. To examine the association between respondent characteristics and satisfaction ratings, we selected 3 specific questions from among the 35 questions which reflect distinct dimensions of the clinical care process—(1) interpersonal communication (clarity of language), (2) service delivery (overall care during visit), and (3) likelihood of recommending the practice to others. In the survey, patients were asked to rate each item on a 5point scale ranging from very poor (1), poor (2), fair (3), good (4), and very good (5). For consistency with the vendor’s reporting format, we rescaled the 5-point scale to 100 for ease of interpretation: 1 = 0, 2 = 25, 3 = 50, 4 = 75, and 5 = 100. First, we conducted t-tests to examine the relationship between each of 124
To examine whether the differences between respondent and nonrespondent demographic characteristics could potentially bias the respondents’ satisfaction ratings, we examined the association between the demographic characteristics and satisfaction ratings across the three dimensions. 1. Satisfaction with Clarity of Language Spoken by Providers In unadjusted bivariate analyses, language, age, and insurance type were all associated with the satisfaction rating on the clarity of language (Table 2a, page 126). Spanish-speaking patients had a significantly lower satisfaction rating than English-speaking patients. Chinese-speaking patients also reported a significantly lower satisfaction rating on the clarity of the language spoken by the providers. Older patients (65–79 years of age) reported higher satisfaction ratings than younger patients. Medicaid patients had significantly lower satisfaction ratings on the
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Table 1. Proportion of Respondents vs. Nonrespondents by Demographic Characteristics Variable Sex/Gender Male Female Language English Spanish Russian Chinese Age, Years 0–17 18–34 35–49 50–64 65–79 80+ Insurance Medicare Medicaid Insurance
Respondent
Nonrespondent
P Value
42.5 57.5
45 55
< .001
94.5 2.5 2.1 0.9
93.1 3.8 2.1 1.0
< .001
8.3 7.8 16.0 31.3 27.8 8.7
7.7 13.9 22.3 29.5 19.9 6.6
< .001
31.2 8.3 60.5
26.3 11.7 62.0
< .001
clarity of the language spoken by the providers than did privately insured patients. Sex was not associated with the satisfaction rating on the clarity of language. In multivariate linear regression analyses, language, age, and insurance type remained independently associated with the satisfaction ratings on the clarity of language spoken by the provider. Chinese-speaking (adjusted mean, 81.8) and Spanishspeaking (84.7) patients were less satisfied than English-speaking patients (92.0). Patients in the 65–75-year age group (90.1) had a higher satisfaction rating than the other age groups. Medicaid patients (86.6) reported lower satisfaction with clarity of language than privately insured patients (88.5). 2. Satisfaction with Care Received During the Visit In bivariate analyses, all four demographic variables were associated with the satisfaction rating on the care received during the visit (Table 2b, page 126). Chinese-speaking patients reported significantly lower satisfaction ratings than English-speaking patients. Russian speakers, on the other hand, provided higher ratings than English-speaking patients. Older patients (65–79 years of age) were more satisfied than younger patients. In accordance with age effect, Medicare patients were more satisfied than privately insured patients. Male patients also reported higher satisfaction compared to the female patients. In multivariate analyses, the language and age variables remained independently associated with the satisfaction rating on the care the
patients received during the visit. Younger patients (18–35 years of age) were significantly less satisfied (adjusted mean, 86.5) compared to the older patients. Chinese-speaking patients (83.0) were less satisfied than English-speaking patients (89.7). 3. Likelihood of Recommending the Practice to Others In bivariate analyses, patients 18–35 years of age and 36–49 years of age were less likely to recommend than older patients (Table 2c, page 127). Medicare patients were also more likely to recommend the practice than privately insured patients. Male patients reported a significantly higher rate of likelihood of recommending the practice than female patients. In multivariate analyses, sex and age remained significantly associated with the likelihood of recommending the practice to others. Male patients (adjusted mean, 90.1) were more likely to recommend the practice than female patients (88.9). Again, patients 18–35 years of age were less likely to recommend the practice (86.1) than patients 65–79 years of age (92.2).
ADJUSTED MEANS FOR NONRESPONSE Given the significant differences in the sample characteristics between respondents and nonrespondents, we examined the impact of nonrespondents on the reported means for all three dimensions within our patient population by calculating the weighted means, adjusting for the group demographic differences. On the basis of the calculated weighted means, significant differences were found for language and age. As shown in Appendix 2, Figure A (available in online article), the weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes. Yet, the means for Russian speakers decreased slightly when adjusted for nonresponse. As shown in Figure 3, Chinese speakers were significantly less likely to recommend the practice when compared with the overall sample, as well as the state (California) and UHC (https://www.uhc.edu/) means. The satisfaction ratings increased slightly for the 18–35 age group when adjusted for the nonresponse. For all three dimensions, all other calculated weighted means were comparable to the means on the basis of the respondents only.
Discussion Systematic differences between respondents and nonrespondents may produce biased results on patient satisfaction surveys and other measures, possibly threatening the validity of the data. Our findings highlight three important unresolved issues in the use of patient satisfaction data: (1) the high overall proportion of nonresponse, (2) the nonresponse rating bias associated with demographic characteristics, and (3) overestimation of the satisfac-
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
125
The Joint Commission Journal on Quality and Patient Safety Table 2a. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Clarity of Language Spoken by Provider” (N = 4,086)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
91.9 91.1 92.3 92.4 94.6§ 90.8
90.3, 93.4 89.5, 92.7 91.2, 93.4 91.6, 93.2 93.7, 95.4 89.3, 92.4
87.4 86.0 87.2 87.3 90.1‡ 86.6
85.1, 89.6 83.7, 88.4 85.2, 89.2 85.5, 89.2 88.2, 92.0 84.3, 88.8
92.6 92.9
92.0, 93.2 92.2, 93.6
87.3 87.5
85.6, 89.1 85.7, 89.3
93.0 85.0§ 92.1 82.6§
92.5, 93.5 82.0, 87.9 88.8, 95.4 77.6, 87.6
92.0 84.7§ 91.3 81.8§
91.3, 92.7 81.8, 87.6 88.0, 94.6 76.8, 86.8
92.9 89.9§ 93.0
92.4, 93.5 88.3, 91.4 92.2, 93.8
88.5 86.6† 87.2
86.7, 90.3 84.5, 88.7 85.3, 89.2
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
Table 2b. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Care Received During Visit” (N = 4,118)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
90.3 86.5‡ 88.0† 89.7 92.4† 91.1
88.5, 92.1 84.7, 88.4 86.7, 89.3 88.8, 90.7 91.4, 93.4 89.3, 92.9
90.0 86.5‡ 87.9 89.4 91.4 89.9
87.4, 92.5 83.9, 89.2 85.6, 90.1 87.4, 91.5 89.3, 93.5 87.3, 92.5
89.6 90.8†
88.9, 90.3 90.0, 91.6
88.8 89.6
86.8, 90.7 87.6, 91.6
90.0 91.4 94.2† 84.0†
89.5, 90.6 88.1, 94.7 90.6, 97.8 78.5, 89.6
89.7 91.3 92.7 83.0†
88.8, 90.5 87.9, 94.6 89.1, 96.4 77.5, 88.7
89.3 89.0 92.1§
88.6, 89.9 87.2, 90.8 91.1, 93.0
88.9 88.6 90.0
86.9, 91.0 86.2, 91.0 87.8, 92.2
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
126
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Table 2c. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Likelihood of Recommending the Practice” (N = 4,060)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
90.7 85.8§ 87.6‡ 90.1 92.9 90.8
88.7, 92.7 83.7, 87.8 86.2, 89.0 89.1, 91.1 91.8, 94.0 88.8, 92.7
90.7 86.1‡ 87.7 90.1 92.2 89.9
87.9, 93.5 83.2, 89.0 85.2, 90.3 87.8, 92.4 89.9, 94.5 87.1, 92.8
89.5 91.2†
88.7, 90.3 90.3, 92.1
88.9 90.1†
86.7, 91.0 87.9, 92.3
90.2 91.8 93.6 86.1
89.6, 90.7 88.2, 95.5 89.5, 97.7 80.0, 92.2
89.5 91.6 92.0 84.9
88.6, 90.3 87.9, 95.2 87.9, 96.0 78.8, 91.0
89.4 88.5 92.3§
88.7, 90.2 86.5, 90.5 91.2, 93.3‡
89.5 88.5 90.4
87.3, 91.8 85.8, 91.1 88.0, 92.9
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
tion due to proportional differences in the demographic profiles of respondents and nonrespondents. Our findings are consistent with previous studies, which also showed that respondents were more likely to be elderly, female, and English speakers than nonrespondents.3,16 Similarly, other studies have found a significant association between ethnicity/ race and nonresponse rates on the CAHPS survey data.17,18 The results indicate that the differences in the proportion of Spanish speakers and the Medicaid patients in the respondent sample could contribute to inflation of the satisfaction ratings. Also, consistent with other studies using CAHPS data, we found significantly lower patient satisfaction ratings by Chinese-speaking patients compared with whites.19 Depending on the distribution of demographic profiles of patients whom an organization serves, the differential response rates among certain demographic groups may bias results in overall satisfaction ratings. Young et al. suggested that the race-associated differences in patient satisfaction scores may reflect both the differences in the expectations and value system associated with care, as well as possibly actual differences in care.3 Also, younger patients’ lower satisfaction ratings might reflect lack of experience and associated unrealistic expectations for the health care system.3 Although such background factors may contribute to overall satisfaction rat-
ings, additional research is needed to determine the nature of interaction between a patient’s background and previous health care experiences, as well as underlying factors contributing to perceptions of care.
LIMITATIONS Our findings should be interpreted in the light of several limitations to the study. First, the demographic variables examined in the study were limited to the demographic information collected during registration. This limited the analysis to those four variables and prevented consideration of other key variables such as race/ethnicity, education, and socioeconomic status. Moreover, any attempts to generalize results of a study sample with response rates below 50% will always be limited.20 In addition, in order to be consistent with the vendor’s reporting and interpretation of the results, we rescaled the outcome using the vendor’s format. Given this rescaling of the responses, we also conducted top-box analyses, in which topbox responses (5) on the 5–point scale were compared against all others (1–4). The two scoring methods indicated overall consistency in the findings. Although we lacked information about race/ethnicity, language variable and insurance type provided some indication of
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
127
The Joint Commission Journal on Quality and Patient Safety the potential influence of ethnicity and socioeconomic status on satisfaction and likelihood of response. Also, the data used in the study came from a single time period from one academic medical center. Given the characteristics of academic centers and their patient population, the results of the findings may not be generalizable to other health care organizations and providers. For one institution, where the goal is to obtain unbiased satisfaction ratings, weighting should suffice; however, for institutional comparisons, case-mix adjustment may be necessary. Finally, the analyses were based on the assumption that the nonresponse patterns would be similar to those of responders. To verify this assumption, further studies on the response patterns of nonresponders will be necessary. The next phase of our study will explicitly test this assumption by collecting patient satisfaction data on the nonresponders using different mode of data collection such as email follow-up and using tablets at the time of care.
Conclusion Although the nonresponse rate in this study did not significantly affect the overall satisfaction score, the effect of nonresponse rates within subgroups, particularly for providers who serve a diverse patient population, will continue to be a challenge. Nonrespondents to an ambulatory care satisfaction survey were more likely to be younger, be non-English-speakers, and have Medicaid insurance. Respondents within these subgroups also had different satisfaction ratings than their counterparts. These findings suggest that overall scores may reflect nonresponse bias, so that comparisons of institutions or providers with patient populations that are significantly different in terms of demographic characteristics may be invalid. Adjustment of satisfaction scores according to patient characteristics should be considered. Future studies of surveys that can achieve higher response rates (for example, > 70%) would be helpful to assess whether overall satisfaction ratings change significantly when a greater proportion of a given patient population is captured in the measurement. J The authors gratefully acknowledge Josh Adler, MD, for his helpful suggestions regarding earlier versions of this manuscript, and Jason Philips, MA, for his assistance with the preparation of the data.
Online-Only Content
8
See the online version of this article for
Appendix 1. Sample Medical Practice Survey Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings
128
Christy K. Boscardin, PhD, is Assistant Professor, Department of Medicine, Office of Medical Education, University of California, San Francisco (UCSF). Ralph Gonzales, MD, MSPH, is Professor of Medicine; Director, Department of Epidemiology and Biostatistics, Program in Implementation and Dissemination Sciences; and Associate Chair for Ambulatory Care and Clinical Innovation, Division of General Internal Medicine, UCSF. Please address correspondence to Christy K. Boscardin, [email protected].
References 1. Carlson MJ, et al. Socioeconomic status and dissatisfaction among HMO enrollees. Med Care. 2000;38(5):508–516. 2. Lasek RJ, et al. An evaluation of the impact of nonresponse bias on patient satisfaction surveys. Med Care. 1997;35(6):646–652. 3. Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: Effects of demographic and institutional characteristics. Med Care. 2000;38(3):325–334. 4. Centers for Medicare & Medicaid Services. Physician Compare Initiative. (Updated: Jan 14, 2013.) Accessed Jan 29, 2013. http://www.cms.gov/Medicare /Quality-Initiatives-Patient-Assessment-Instruments/physician-compare -initiative/index.html. 5. H.R. 3590. The Patient Protection and Affordable Care Act (PPACA) of 2010, Public Law 111–148, Mar 23, 2010. Accessed Jan 29, 2013, http://www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS111hr3590enr.pdf. 6. Darby C, Hays RD, Kletke P. Development and evaluation of the CAHPS hospital survey. Health Serv Res. 2005;40(6 Pt 2):1973–1976. 7. Gary TL, et al. Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Dis Manag. 2005;8(6): 361–371. 8. Beach MC, et al. Do patients treated with dignity report higher satisfaction, adherence, and receipt of preventive care? Ann Fam Med. 2005;3(4):331–338. 9. Chen AY, et al. Differences in CAHPS reports and ratings of health care provided to adults and children. Med Care. 2012 Nov;50(Suppl):S35–39. 10. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry. 1988;25(1):25–36. 11. Sixma HJ, Spreeuwenberg PM, van der Pasch MA. Patient satisfaction with the general practitioner: A two-level analysis. Med Care. 1998;36(2):212–229. 12. van den Akker M, et al. Morbidity in responders and non-responders in a register-based population survey. Fam Pract. 1998;15(3):261–263. 13. Hall JA, Dornan MC. Patient sociodemographic characteristics as predictors of satisfaction with medical care: A meta-analysis. Soc Sci Med. 1990;30(7):811–818. 14. Perneger TV, Chamot E, Bovier PA. Nonresponse bias in a survey of patient perceptions of hospital care. Med Care. 2005;43(4):374–380. 15. Roland M, et al. Reliability of patient responses in pay for performance schemes: Analysis of national General Practitioner Patient Survey Data in England. BMJ. 2009 Sep 29;339:b3851. 16. Hays RD, et al. Hospital quality trends: A short-form patient-based measure. Med Care. 1991;29(7):661–668. 17. Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care. 2002;40(6):485–499. 18. Klein DJ, et al. Understanding nonresponse to the 2007 Medicare CAHPS Survey. Gerontologist. 2011;51(6):843–855. 19. Morales LS, et al. Differences in CAHPS adult survey reports and ratings by race and ethnicity: An analysis of the National CAHPS benchmarking data 1.0. Health Serv Res. 2001;36(3): 595–617. 20. Grady KE, Wallston BS. Research in Health Care Settings. Newbury Park, CA: Sage, 1988.
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8 Appendix 1. Sample Medical Practice Survey
(continued on page AP2)
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
AP1
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8 Appendix 1. Sample Medical Practice Survey (continued)
AP2
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8
Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings A. Weighted and Nonweighted Mean Satisfaction Ratings for “Clarity of Language Spoken by Provider” (N = 4,086 )
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Clarity of Language Spoken by Provider.” The weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes, while the means for Russian speakers decreased slightly. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown. Not all patients completed all questions in the questionnaire.
B. Weighted and Nonweighted Mean Satisfaction Ratings for “Care Received During Visit” (N = 4,118)
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Care Received During Visit.” The weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes, while the means for Russian speakers decreased slightly. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown. Not all patients completed all questions in the questionnaire. (continued on page AP4)
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
AP3
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8
Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings (continued) C. Weighted and Nonweighted Mean Satisfaction Ratings for “Likelihood of Recommending Practice” (N = 4,086)
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Likelihood of Recommending Practice.” Chinese speakers were significantly less likely to recommend the practice when compared with the overall sample, as well as the state (California) and UHC (https://www.uhc.edu/) means. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown.
AP4
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Impact of Demographic Characteristics on Nonresponse in an Ambulatory Patient Satisfaction Survey Christy K. Boscardin, PhD; Ralph Gonzales, MD, MSPH
P
atient satisfaction surveys are increasingly becoming one of the most important and widely used indicators of health care quality1–3 and are increasingly being used in pay-for-performance schemes. With the US federal mandate for the Physician Compare initiative,4 which will provide information to patients about the quality of care, including patient experience, under Section 10331 of the Patient Protection and Affordable Care Act (Pub L 111148) of 2010,5 the emphasis on patient satisfaction data will be an even more critical component of practice- and systems-based improvement strategies in the future. The reliance on the Consumer Assessment of Healthcare Providers and Systems (CAHPS) data by the federal government to compare health care quality across the health plans serving Medicare beneficiaries is a prime example of the current emphasis on patient satisfaction data as a primary source of health care quality.6 Previous studies have also shown that patient satisfaction is associated with better health outcomes and greater adherence to treatment recommendations and medication.7,8 The positive link between patient satisfaction and compliance has been one of the key arguments for the utility of patient satisfaction surveys. With health care reform and the rise of consumerist ethos in the health care industry, patient satisfaction data will remain an important aspect of health care quality. Despite increasing use of patient satisfaction data for compensation and comparisons across institutions, medical groups, and providers, the low response rate and potential bias of the respondents pose a significant threat to validity of the data. With the response rate for surveys typically ranging from 25% to 50%,9 the potential for response bias in the data is substantial. In a previous study of hospitalized patients, satisfaction scores were significantly associated with certain demographic characteristics such as age, sex, and race (ethnicity).10–13 Despite these concerns, few studies have examined the potential bias and the impact of nonresponse, and studies examining the impact of nonresponse have been limited to hospital settings.3,12,14,15 In this study, we compared the demographic profiles of
Article-at-a-Glance Background: Despite the increasing use of patient satisfac-
tion data for compensation and comparison of performance, low response rate and potential bias of the respondents pose a significant threat to validity of the data. The demographic profiles of respondents and nonrespondents to a patient satisfaction survey in the ambulatory care setting were compared to explore the impact of nonresponse bias. Methods: Patient satisfaction survey data were collected from October through December 2010 for outpatient facilities at a large academic medical center. The association between respondent characteristics and satisfaction ratings on three dimensions of the clinical care process—(1) interpersonal communication (clarity of language), (2) service delivery (overall care during visit), and (3) likelihood of recommending practice to others—were assessed with bivariate and multivariate linear regression. Weighted analyses were performed to examine the impact of nonresponse. Results: Surveys were mailed to 15,549 patients, of which 4,952 (32%) were returned. Respondents had greater proportions of elderly, female, and English speakers. Bivariate analyses showed significant difference in satisfaction ratings by age, language, and insurance type. Multivariate regression analysis showed significant confounding across variables. On the basis of the calculated weighted means, mean satisfaction ratings were discrepant for language and age; however, the overall satisfaction ratings for each dimension were minimally affected. Conclusion: Nonresponse rates and satisfaction ratings differed by age, language, and insurance type. However, if it is assumed that nonrespondents within these demographic groups have similar satisfaction ratings as respondents, then nonresponse levels appear to have minimal effects on overall satisfaction ratings.
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
123
The Joint Commission Journal on Quality and Patient Safety respondents and nonrespondents to a patient satisfaction survey administered in an ambulatory care setting to determine whether systematic differences exist between the two groups.
Methods SURVEY DATA Patient satisfaction survey data were collected by a health care performance improvement vendor for a one-quarter period (October 2010–December 2010) for outpatient facilities at a large academic medical center. These surveys are conducted continuously, and data were aggregated by quarter for monitoring and reporting purposes. The survey was mailed by the vendor to a stratified random sample of 15,549 patients who completed ambulatory visits at a large academic medical center during the observation period. Completed surveys were returned by 4,952 respondents (response rate, 32%). The 35-item survey (Appendix 1, available in online article) includes items on the patient’s satisfaction with registration, facility, test/treatment, personal issues (sensitivity to needs), physician, and overall quality assessment of his or her care. The following demographic information was available for both respondents and nonrespondents: (1) age, (2) sex, (3) language of the survey administered, and (4) insurance type. The demographic data for the analysis were limited to the information collected during registration because these data were available for both groups. Race and ethnicity information were not available for the nonrespondents because this information is collected as part of the survey administration.
the demographic characteristics (age, sex, language, and insurance type) and satisfaction ratings separately. Then we conducted multivariable linear regression analyses to examine the independent effect of the demographic characteristics on the satisfaction ratings for each of the three specific questions. Finally, to examine the impact of nonresponse on reported satisfaction ratings, we performed weighted analyses, weighting by the inverse probability of response. Specifically, we calculated weights through a poststratification approach to adjust the characteristics of the response sample to those of the total sample (including the nonresponse sample) by using the four key demographic variables (language, sex, age, and insurance type). The nonresponse weights were first calculated using the package in Stata version 11 (StataCorp LP, College Station, Texas). We performed subsequent analyses with these weights using Stata’s pweights (probability weights, also known as sampling weights).
Results RESPONDENT AND NONRESPONDENT PROFILES The profiles of respondent and nonrespondents were compared on the four key demographic variables. We found substantial differences between respondents and nonrespondent profiles on the basis of age, sex, language, and insurance type. As shown in Table 1 (page 125), a greater proportion of respondents than nonrespondents were elderly, female, and English speakers (Table 1, p < .001 for all comparisons). A greater proportion of respondents reported Medicare insurance, and a smaller proportion reported Medicaid insurance (p < .001).
ASSOCIATION BETWEEN THE DEMOGRAPHIC CHARACTERISTICS AND SATISFACTION RATINGS Descriptive statistics (relative frequency) were calculated for de- ACROSS THE THREE DIMENSIONS Data Analysis
mographic data. The chi-square test for categorical data between respondents and nonrespondents was used to compare the proportional difference. To examine the association between respondent characteristics and satisfaction ratings, we selected 3 specific questions from among the 35 questions which reflect distinct dimensions of the clinical care process—(1) interpersonal communication (clarity of language), (2) service delivery (overall care during visit), and (3) likelihood of recommending the practice to others. In the survey, patients were asked to rate each item on a 5point scale ranging from very poor (1), poor (2), fair (3), good (4), and very good (5). For consistency with the vendor’s reporting format, we rescaled the 5-point scale to 100 for ease of interpretation: 1 = 0, 2 = 25, 3 = 50, 4 = 75, and 5 = 100. First, we conducted t-tests to examine the relationship between each of 124
To examine whether the differences between respondent and nonrespondent demographic characteristics could potentially bias the respondents’ satisfaction ratings, we examined the association between the demographic characteristics and satisfaction ratings across the three dimensions. 1. Satisfaction with Clarity of Language Spoken by Providers In unadjusted bivariate analyses, language, age, and insurance type were all associated with the satisfaction rating on the clarity of language (Table 2a, page 126). Spanish-speaking patients had a significantly lower satisfaction rating than English-speaking patients. Chinese-speaking patients also reported a significantly lower satisfaction rating on the clarity of the language spoken by the providers. Older patients (65–79 years of age) reported higher satisfaction ratings than younger patients. Medicaid patients had significantly lower satisfaction ratings on the
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Table 1. Proportion of Respondents vs. Nonrespondents by Demographic Characteristics Variable Sex/Gender Male Female Language English Spanish Russian Chinese Age, Years 0–17 18–34 35–49 50–64 65–79 80+ Insurance Medicare Medicaid Insurance
Respondent
Nonrespondent
P Value
42.5 57.5
45 55
< .001
94.5 2.5 2.1 0.9
93.1 3.8 2.1 1.0
< .001
8.3 7.8 16.0 31.3 27.8 8.7
7.7 13.9 22.3 29.5 19.9 6.6
< .001
31.2 8.3 60.5
26.3 11.7 62.0
< .001
clarity of the language spoken by the providers than did privately insured patients. Sex was not associated with the satisfaction rating on the clarity of language. In multivariate linear regression analyses, language, age, and insurance type remained independently associated with the satisfaction ratings on the clarity of language spoken by the provider. Chinese-speaking (adjusted mean, 81.8) and Spanishspeaking (84.7) patients were less satisfied than English-speaking patients (92.0). Patients in the 65–75-year age group (90.1) had a higher satisfaction rating than the other age groups. Medicaid patients (86.6) reported lower satisfaction with clarity of language than privately insured patients (88.5). 2. Satisfaction with Care Received During the Visit In bivariate analyses, all four demographic variables were associated with the satisfaction rating on the care received during the visit (Table 2b, page 126). Chinese-speaking patients reported significantly lower satisfaction ratings than English-speaking patients. Russian speakers, on the other hand, provided higher ratings than English-speaking patients. Older patients (65–79 years of age) were more satisfied than younger patients. In accordance with age effect, Medicare patients were more satisfied than privately insured patients. Male patients also reported higher satisfaction compared to the female patients. In multivariate analyses, the language and age variables remained independently associated with the satisfaction rating on the care the
patients received during the visit. Younger patients (18–35 years of age) were significantly less satisfied (adjusted mean, 86.5) compared to the older patients. Chinese-speaking patients (83.0) were less satisfied than English-speaking patients (89.7). 3. Likelihood of Recommending the Practice to Others In bivariate analyses, patients 18–35 years of age and 36–49 years of age were less likely to recommend than older patients (Table 2c, page 127). Medicare patients were also more likely to recommend the practice than privately insured patients. Male patients reported a significantly higher rate of likelihood of recommending the practice than female patients. In multivariate analyses, sex and age remained significantly associated with the likelihood of recommending the practice to others. Male patients (adjusted mean, 90.1) were more likely to recommend the practice than female patients (88.9). Again, patients 18–35 years of age were less likely to recommend the practice (86.1) than patients 65–79 years of age (92.2).
ADJUSTED MEANS FOR NONRESPONSE Given the significant differences in the sample characteristics between respondents and nonrespondents, we examined the impact of nonrespondents on the reported means for all three dimensions within our patient population by calculating the weighted means, adjusting for the group demographic differences. On the basis of the calculated weighted means, significant differences were found for language and age. As shown in Appendix 2, Figure A (available in online article), the weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes. Yet, the means for Russian speakers decreased slightly when adjusted for nonresponse. As shown in Figure 3, Chinese speakers were significantly less likely to recommend the practice when compared with the overall sample, as well as the state (California) and UHC (https://www.uhc.edu/) means. The satisfaction ratings increased slightly for the 18–35 age group when adjusted for the nonresponse. For all three dimensions, all other calculated weighted means were comparable to the means on the basis of the respondents only.
Discussion Systematic differences between respondents and nonrespondents may produce biased results on patient satisfaction surveys and other measures, possibly threatening the validity of the data. Our findings highlight three important unresolved issues in the use of patient satisfaction data: (1) the high overall proportion of nonresponse, (2) the nonresponse rating bias associated with demographic characteristics, and (3) overestimation of the satisfac-
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
125
The Joint Commission Journal on Quality and Patient Safety Table 2a. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Clarity of Language Spoken by Provider” (N = 4,086)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
91.9 91.1 92.3 92.4 94.6§ 90.8
90.3, 93.4 89.5, 92.7 91.2, 93.4 91.6, 93.2 93.7, 95.4 89.3, 92.4
87.4 86.0 87.2 87.3 90.1‡ 86.6
85.1, 89.6 83.7, 88.4 85.2, 89.2 85.5, 89.2 88.2, 92.0 84.3, 88.8
92.6 92.9
92.0, 93.2 92.2, 93.6
87.3 87.5
85.6, 89.1 85.7, 89.3
93.0 85.0§ 92.1 82.6§
92.5, 93.5 82.0, 87.9 88.8, 95.4 77.6, 87.6
92.0 84.7§ 91.3 81.8§
91.3, 92.7 81.8, 87.6 88.0, 94.6 76.8, 86.8
92.9 89.9§ 93.0
92.4, 93.5 88.3, 91.4 92.2, 93.8
88.5 86.6† 87.2
86.7, 90.3 84.5, 88.7 85.3, 89.2
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
Table 2b. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Care Received During Visit” (N = 4,118)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
90.3 86.5‡ 88.0† 89.7 92.4† 91.1
88.5, 92.1 84.7, 88.4 86.7, 89.3 88.8, 90.7 91.4, 93.4 89.3, 92.9
90.0 86.5‡ 87.9 89.4 91.4 89.9
87.4, 92.5 83.9, 89.2 85.6, 90.1 87.4, 91.5 89.3, 93.5 87.3, 92.5
89.6 90.8†
88.9, 90.3 90.0, 91.6
88.8 89.6
86.8, 90.7 87.6, 91.6
90.0 91.4 94.2† 84.0†
89.5, 90.6 88.1, 94.7 90.6, 97.8 78.5, 89.6
89.7 91.3 92.7 83.0†
88.8, 90.5 87.9, 94.6 89.1, 96.4 77.5, 88.7
89.3 89.0 92.1§
88.6, 89.9 87.2, 90.8 91.1, 93.0
88.9 88.6 90.0
86.9, 91.0 86.2, 91.0 87.8, 92.2
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
126
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Table 2c. Unadjusted and Adjusted Mean Satisfaction Rating Stratified by Patient Demographic Characteristics on “Likelihood of Recommending the Practice” (N = 4,060)* Age, Years 0–17 (R) 18–35 36–49 50–64 65–79 80+ Sex Female (R) Male Language English (R) Spanish Russian Chinese Insurance Private (R) Medicaid Medicare
Unadjusted Mean
95% CI
Adjusted Mean
95% CI
90.7 85.8§ 87.6‡ 90.1 92.9 90.8
88.7, 92.7 83.7, 87.8 86.2, 89.0 89.1, 91.1 91.8, 94.0 88.8, 92.7
90.7 86.1‡ 87.7 90.1 92.2 89.9
87.9, 93.5 83.2, 89.0 85.2, 90.3 87.8, 92.4 89.9, 94.5 87.1, 92.8
89.5 91.2†
88.7, 90.3 90.3, 92.1
88.9 90.1†
86.7, 91.0 87.9, 92.3
90.2 91.8 93.6 86.1
89.6, 90.7 88.2, 95.5 89.5, 97.7 80.0, 92.2
89.5 91.6 92.0 84.9
88.6, 90.3 87.9, 95.2 87.9, 96.0 78.8, 91.0
89.4 88.5 92.3§
88.7, 90.2 86.5, 90.5 91.2, 93.3‡
89.5 88.5 90.4
87.3, 91.8 85.8, 91.1 88.0, 92.9
* Not all patients completed all questions in the questionnaire. CI, confidence interval; R, reference. † p < .05 (chi-square test). ‡ p < .01. § p < .001.
tion due to proportional differences in the demographic profiles of respondents and nonrespondents. Our findings are consistent with previous studies, which also showed that respondents were more likely to be elderly, female, and English speakers than nonrespondents.3,16 Similarly, other studies have found a significant association between ethnicity/ race and nonresponse rates on the CAHPS survey data.17,18 The results indicate that the differences in the proportion of Spanish speakers and the Medicaid patients in the respondent sample could contribute to inflation of the satisfaction ratings. Also, consistent with other studies using CAHPS data, we found significantly lower patient satisfaction ratings by Chinese-speaking patients compared with whites.19 Depending on the distribution of demographic profiles of patients whom an organization serves, the differential response rates among certain demographic groups may bias results in overall satisfaction ratings. Young et al. suggested that the race-associated differences in patient satisfaction scores may reflect both the differences in the expectations and value system associated with care, as well as possibly actual differences in care.3 Also, younger patients’ lower satisfaction ratings might reflect lack of experience and associated unrealistic expectations for the health care system.3 Although such background factors may contribute to overall satisfaction rat-
ings, additional research is needed to determine the nature of interaction between a patient’s background and previous health care experiences, as well as underlying factors contributing to perceptions of care.
LIMITATIONS Our findings should be interpreted in the light of several limitations to the study. First, the demographic variables examined in the study were limited to the demographic information collected during registration. This limited the analysis to those four variables and prevented consideration of other key variables such as race/ethnicity, education, and socioeconomic status. Moreover, any attempts to generalize results of a study sample with response rates below 50% will always be limited.20 In addition, in order to be consistent with the vendor’s reporting and interpretation of the results, we rescaled the outcome using the vendor’s format. Given this rescaling of the responses, we also conducted top-box analyses, in which topbox responses (5) on the 5–point scale were compared against all others (1–4). The two scoring methods indicated overall consistency in the findings. Although we lacked information about race/ethnicity, language variable and insurance type provided some indication of
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
127
The Joint Commission Journal on Quality and Patient Safety the potential influence of ethnicity and socioeconomic status on satisfaction and likelihood of response. Also, the data used in the study came from a single time period from one academic medical center. Given the characteristics of academic centers and their patient population, the results of the findings may not be generalizable to other health care organizations and providers. For one institution, where the goal is to obtain unbiased satisfaction ratings, weighting should suffice; however, for institutional comparisons, case-mix adjustment may be necessary. Finally, the analyses were based on the assumption that the nonresponse patterns would be similar to those of responders. To verify this assumption, further studies on the response patterns of nonresponders will be necessary. The next phase of our study will explicitly test this assumption by collecting patient satisfaction data on the nonresponders using different mode of data collection such as email follow-up and using tablets at the time of care.
Conclusion Although the nonresponse rate in this study did not significantly affect the overall satisfaction score, the effect of nonresponse rates within subgroups, particularly for providers who serve a diverse patient population, will continue to be a challenge. Nonrespondents to an ambulatory care satisfaction survey were more likely to be younger, be non-English-speakers, and have Medicaid insurance. Respondents within these subgroups also had different satisfaction ratings than their counterparts. These findings suggest that overall scores may reflect nonresponse bias, so that comparisons of institutions or providers with patient populations that are significantly different in terms of demographic characteristics may be invalid. Adjustment of satisfaction scores according to patient characteristics should be considered. Future studies of surveys that can achieve higher response rates (for example, > 70%) would be helpful to assess whether overall satisfaction ratings change significantly when a greater proportion of a given patient population is captured in the measurement. J The authors gratefully acknowledge Josh Adler, MD, for his helpful suggestions regarding earlier versions of this manuscript, and Jason Philips, MA, for his assistance with the preparation of the data.
Online-Only Content
8
See the online version of this article for
Appendix 1. Sample Medical Practice Survey Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings
128
Christy K. Boscardin, PhD, is Assistant Professor, Department of Medicine, Office of Medical Education, University of California, San Francisco (UCSF). Ralph Gonzales, MD, MSPH, is Professor of Medicine; Director, Department of Epidemiology and Biostatistics, Program in Implementation and Dissemination Sciences; and Associate Chair for Ambulatory Care and Clinical Innovation, Division of General Internal Medicine, UCSF. Please address correspondence to Christy K. Boscardin, [email protected].
References 1. Carlson MJ, et al. Socioeconomic status and dissatisfaction among HMO enrollees. Med Care. 2000;38(5):508–516. 2. Lasek RJ, et al. An evaluation of the impact of nonresponse bias on patient satisfaction surveys. Med Care. 1997;35(6):646–652. 3. Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: Effects of demographic and institutional characteristics. Med Care. 2000;38(3):325–334. 4. Centers for Medicare & Medicaid Services. Physician Compare Initiative. (Updated: Jan 14, 2013.) Accessed Jan 29, 2013. http://www.cms.gov/Medicare /Quality-Initiatives-Patient-Assessment-Instruments/physician-compare -initiative/index.html. 5. H.R. 3590. The Patient Protection and Affordable Care Act (PPACA) of 2010, Public Law 111–148, Mar 23, 2010. Accessed Jan 29, 2013, http://www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS111hr3590enr.pdf. 6. Darby C, Hays RD, Kletke P. Development and evaluation of the CAHPS hospital survey. Health Serv Res. 2005;40(6 Pt 2):1973–1976. 7. Gary TL, et al. Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Dis Manag. 2005;8(6): 361–371. 8. Beach MC, et al. Do patients treated with dignity report higher satisfaction, adherence, and receipt of preventive care? Ann Fam Med. 2005;3(4):331–338. 9. Chen AY, et al. Differences in CAHPS reports and ratings of health care provided to adults and children. Med Care. 2012 Nov;50(Suppl):S35–39. 10. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry. 1988;25(1):25–36. 11. Sixma HJ, Spreeuwenberg PM, van der Pasch MA. Patient satisfaction with the general practitioner: A two-level analysis. Med Care. 1998;36(2):212–229. 12. van den Akker M, et al. Morbidity in responders and non-responders in a register-based population survey. Fam Pract. 1998;15(3):261–263. 13. Hall JA, Dornan MC. Patient sociodemographic characteristics as predictors of satisfaction with medical care: A meta-analysis. Soc Sci Med. 1990;30(7):811–818. 14. Perneger TV, Chamot E, Bovier PA. Nonresponse bias in a survey of patient perceptions of hospital care. Med Care. 2005;43(4):374–380. 15. Roland M, et al. Reliability of patient responses in pay for performance schemes: Analysis of national General Practitioner Patient Survey Data in England. BMJ. 2009 Sep 29;339:b3851. 16. Hays RD, et al. Hospital quality trends: A short-form patient-based measure. Med Care. 1991;29(7):661–668. 17. Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care. 2002;40(6):485–499. 18. Klein DJ, et al. Understanding nonresponse to the 2007 Medicare CAHPS Survey. Gerontologist. 2011;51(6):843–855. 19. Morales LS, et al. Differences in CAHPS adult survey reports and ratings by race and ethnicity: An analysis of the National CAHPS benchmarking data 1.0. Health Serv Res. 2001;36(3): 595–617. 20. Grady KE, Wallston BS. Research in Health Care Settings. Newbury Park, CA: Sage, 1988.
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8 Appendix 1. Sample Medical Practice Survey
(continued on page AP2)
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
AP1
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8 Appendix 1. Sample Medical Practice Survey (continued)
AP2
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8
Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings A. Weighted and Nonweighted Mean Satisfaction Ratings for “Clarity of Language Spoken by Provider” (N = 4,086 )
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Clarity of Language Spoken by Provider.” The weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes, while the means for Russian speakers decreased slightly. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown. Not all patients completed all questions in the questionnaire.
B. Weighted and Nonweighted Mean Satisfaction Ratings for “Care Received During Visit” (N = 4,118)
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Care Received During Visit.” The weighted means increased slightly for Chinese speakers when adjusted for nonresponse across all three outcomes, while the means for Russian speakers decreased slightly. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown. Not all patients completed all questions in the questionnaire. (continued on page AP4)
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission
AP3
The Joint Commission Journal on Quality and Patient Safety Online-Only Content
8
Appendix 2. Weighted and Nonweighted Mean Satisfaction Ratings (continued) C. Weighted and Nonweighted Mean Satisfaction Ratings for “Likelihood of Recommending Practice” (N = 4,086)
Weighted and the nonweighted (for example, “Chinese”) means are shown for “Likelihood of Recommending Practice.” Chinese speakers were significantly less likely to recommend the practice when compared with the overall sample, as well as the state (California) and UHC (https://www.uhc.edu/) means. Values for the state of California and UHC (https://www.uhc.edu/), provided by the vendor, are also shown.
AP4
March 2013 Volume 39 Number 3 Copyright 2013 © The Joint Commission