o
r
i
g
i
n
a
l
c
o
m
m
u
n
i
c
a
t
i
o
n
Cultural Competency in Health Care: Evaluating the Outcomes of a Cultural Competency Training Among Health Care Professionals Sunil K. Khanna, PhD; Melissa Cheyney, PhD; Molly Engle, PhD
Funding/Support: The authors would like to acknowledge the support of the doctors and staff at The Corvallis Clinic and Good Samaritan Hospital in Corvallis, Oregon. Dr Khanna would like to acknowledge the Department of Anthropology, Oregon State University, for awarding a summer grant in 2008 to complete data analysis and preparation of manuscript. Purpose: The purpose of this research was to examine the effectiveness of a cultural competency training program designed to improve the knowledge and skills of health care providers and administrators engaging in cross-cultural, clinical encounters. Methods: We conducted an evaluation study among 43 health care professionals (health care providers and health administrators) who attended a 4-hour cultural competency workshop. We used a post-then-pre method of self-reported evaluation to answer a key question: Does cultural competency training produce a measurable change in knowledge and skills relating to the care of patients from diverse cultural and ethnic backgrounds? Results: The study findings suggest that there are statistically significant change in participants’ self-report of knowledge and skills related to cultural competency. Conclusions: Our study shows that a cultural competency training program that integrates key topics as recommended by the Institute of Medicine and includes Culturally and Linguistically Appropriate Services in Health Care standards improves the knowledge and skills of health care providers and administrators. Following the training, the participants self-reported not only an enhanced understanding of the health care experiences of patients with diverse backgrounds, but also an improvement in their skills to effectively work in cross-cultural situations. In addition, our study offers a reliable, innovative, and time-efficient strategy—post-thenpre method—for evaluating the outcomes of 4-hour cultural competency training programs. Keywords: cultural competency n health disparities J Natl Med Assoc. 2009;101:886-892
886 JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
Author Affiliations: Department of Anthropology (Dr Khanna, associate professor; Dr Cheyney, assistant professor), Oregon State University, and OSU Extension Service, College of Education (Dr Engle, associate professor and evaluation specialist), Corvallis, Oregon. Corresponding Author: Sunil K. Khanna, PhD, Associate Professor, Department of Anthropology, 238 Waldo Hall, Oregon State University, Corvallis, OR 97331 (
[email protected]).
INTRODUCTION
I
n recent years, scholars and policy makers have paid considerable attention to the topic of cultural competency, especially as it relates to the delivery of health care in the United States. Scholarly research has focused on the potential role of cultural competency training for health care providers in reducing health care disparities.1-3 Researchers suggest that appropriate cultural competency training programs should be implemented to educate care providers about the role of cultural factors, such as ethnomedical beliefs and use of folk medicine, health beliefs and worldview, culturally prescribed values and norms, gender-specific status and roles, and religion, in influencing the outcome of patient-provider encounters.4-6 Although some scholars continue to debate the benefits and limitations of cultural competency training in health care, there is general agreement among researchers that it improves patient-provider communication and that, in the long term, it increases patient satisfaction and compliance.7-9 In this article, we report findings from a pilot study designed to assess the outcomes of a training program aimed at improving knowledge and skills related to the provision of cultural competent care among providers and administrators. There is an increasing level of academic interest in examining the interrelatedness of health and culture, especially as it relates to the delivery of health care.1-6 In light of growing ethnic diversity and associated health disparities in the United States, health policy makers and providers have realized the benefits of cultural competency training. Key stakeholders in the health care arena— for example, providers, administrators, insurers, and policy makers—consider cultural competency training to be germane to providing quality health care to a diverse population of patients. The Health Resources and Services VOL. 101, NO. 9, SEPTEMBER 2009
Cultural competency in health care
Administration (HRSA) strongly advocates for the inclusion of cultural and linguistic competency training and assessment protocols in health care systems. Following up on the seminal definition of cultural competency, HRSA, along with the Office of Minority Health (OMH), has jointly proposed several initiatives to promote culturally competent practices and approaches for improving health care access and utilization by minority populations and for reducing health disparities.10 Most noteworthy among these initiatives is the call for a universal implementation of the National Standards for Culturally and Linguistically Appropriate Services in Health Care (CLAS).11 According to the OMH, the CLAS standards serve as a mechanism for reducing ethnic and racial inequities existing in health care delivery. The CLAS standards are also used as a benchmark for providing culturally and linguistically competent health services to patients from all cultural backgrounds.11 The CLAS standards recognize that regardless of the claims of unprecedented technological innovation and the notion of scientific objectivity in biomedicine, the delivery of medical care always takes place in an interactive, social context. At the core of this social interaction lies a complex set of interpersonal and political factors that influence the patient-provider relationship. The relationship embodies not only the authoritative knowledge and power of health care providers, but also the culturally mediated beliefs of both patient and physician.12,13 Cultural characteristics of patients and health care providers strongly influence perceptions of health and health care seeking behavior as well as access to and utilization of health care services. Several scholars and policy makers have recommended incorporating cultural and linguistic competency in health care delivery on the premise that an inclusive and culturally sensitive health care system will improve the utilization of health care by patients from diverse backgrounds and will eventually reduce racial and ethnic health disparities.14,15 Further, some researchers have suggested that health care organizations should develop a practical cultural competency framework that includes diversifying the workforce, making interpreter services and health promotion literature sensitive to the needs of a diverse patient population, and incorporating cross-cultural training for providers.14 Such recommendations reinforce the sociocultural embeddedness of biomedicine and that of patient-provider interactions.16 Research on health disparities suggests that patients who receive culturally sensitive care are likely to show an increased level of adherence to medical advice and to report satisfaction with their care providers.17,18 As an outcome of ongoing research and policy discussion, an increasing number of health care organizations, physician groups, hospitals systems, and medical schools are integrating cultural competency training into their existing training programs or curricula.19 JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
In this paper, we present and discuss the findings from an evaluative study conducted with 43 health care providers who attended a cultural competency training workshop and completed the workshop evaluation tool. The information presented here addresses a key research question: Does cultural competency training produce a measurable change in the self-assessment of participants’ knowledge and skills related to the care of patients from diverse cultural and ethnic backgrounds? Answering the above question in an evidence-based manner is especially important for researchers and organizations engaged in efforts to reduce cultural and linguistic barriers to health care, as well as for those committed to improving the delivery of health care services and overall health outcomes.
METHODS
Sixty health care providers (nurses and doctors) and health care administrators (unit in-charge and managers) self-selected to participate in a 4-hour-long cultural competency training workshop in 2007. At the time of the training, all participants worked in 1 of the 2 large medical groups in the mid-Willamette Valley region in Oregon. Drawing on the extensive literature on cultural competency, especially as it relates to prescriptions for developing training programs for health care providers,19 we developed an Institute of Medicine (IOM)- and CLAS-based cultural competency training program. The broad topics covered in the training included defining cultural and linguistic competency, ethnic and racial health disparities, the relationship between culture and health beliefs, and the role of cultural competency in facilitating effective communication between patients and providers. Upon completion of this training, participants were able to: • describe the diversity spectrum and define culture; • distinguish among culture, race, and ethnicity; • identify and describe intercultural and intracultural diversity; • distinguish between cultural generalizations and stereotypes; • define cultural competency and examine its individual and institutional underpinnings; • explain the cultural competency continuum and reflect upon their position on the cultural competency continuum; • describe the importance of using explanatory models during patient-provider communication. The workshop offered 4 continuing education units for participation and focused on both knowledge and skills development. We employed a retrospective, postmethod then premethod of evaluation to assess whether the training produced a measurable change in participants’ self-reported VOL. 101, NO. 9, SEPTEMBER 2009 887
Cultural competency in health care
knowledge and skills related to the care of patients from diverse cultural and ethnic backgrounds. At the end of the training session, all participants were given the option to complete a 29-item Cultural Competency Assessment (CCA) tool. Fifty-two participants voluntarily completed the CCA tool. Nine surveys were found to have errors or missing data, while 43 (n = 43) surveys were found to be complete. The CCA tool was developed by the authors. The questions used in the tool reflect the key topics covered in the cultural competency training program. Although the acronym CCA is commonly used among cultural competence education researchers, our tool was independently developed for
the purpose of this study. Our CCA tool was developed to reflect the focus in the training on intercultural and intracultural diversity. The CCA tool included items that were organized into 3 parts—demographic information (5 items), knowledge-related statements (19 items), and skills-related statements (5 items). Responses in the knowledge and skills categories were sought on a 5point Likert-type scale (Table 1). The demographic part of the CCA gathered information on participants’ profession, gender, ethnicity, and language competency. The items in the last 2 parts of the CCA tool focused on changes in knowledge and skills reported by participants in response to the training session. Participants were
Table 1. Survey Instrument Used for Assessing the Cultural Competency Training Outcomes I. Demographic Information 1. Gender: 3. Are you bilingual:
2. Self identified ethnic/racial identity: 4. Profession: 5. Years in profession: After Training
Never
Somewhat often
Very often
Always
Never
Rarely
Somewhat often
Very often
Always
888 JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
Rarely
II. Knowledge 1 Health care systems and providers discriminates against patients based on a. how well they speak English b. their race or ethnic background c. their sexual orientation d. physical disability 2 Culture, race, and ethnicity can be interchangeably used 3 Health differences exist between races/ethnic groups 4 Patients’ health beliefs impact access to health care 5 Patients’ health beliefs impact use of health care 6 Patients’ race/ethnicity influences access to health care 7 Cultural factors influence communication between a patient and a provider 8 Minority patients experience racism and discrimination while seeking health care. 9 Health care providers should a. learn about the cultural and health beliefs of patients b. greet patients in a culturally appropriate manner c. ask patients about their perspectives on illness d. ask patients about the use of folk remedies e. examine patients in a culturally appropriate manner f. develop a culturally sensitive health plan for patients g. acknowledge family members’ roles in patients’ health care decisions h. pay attention to nonverbal cues or gestures when interacting with patients III. Skills 10 I can effectively work with health care interpreters 11 I can successfully deal with cross-cultural adherence problems 13 I can successfully deal with cross-cultural misunderstandings 14 I can effectively deal with patients with limited English proficiency 15 I know it is important to pay attention to cultural expressions of pain, distress, isolation, & disagreement
Before Training
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0
0
0
0
0
0
0
0
0
0
VOL. 101, NO. 9, SEPTEMBER 2009
Cultural competency in health care
asked to answer the questions using the following rating scale—never, rarely, somewhat often, very often, and always. The questions included in the CCA tool were formulated to reflect the learning objectives of the training. Some researchers have suggested that the presentation questionnaire format as used in this research tends to enhance pre to post differences because the “now” and “before” questions are viewed simultaneously.20,21 Before initiating the study, we received approval from the institutional review board of Oregon State University. At the beginning of the workshop, we informed participants about the evaluation study, read aloud the entire informed consent form, answered all questions, handed out the CCA tool, and directed participants to drop the completed survey into a box before exiting the building. We did not provide any verbal instructions for completing the questionnaire.
Retrospective Post-Then-Pre Evaluation Our decision to adopt a retrospective post-then-pre evaluation instrument for assessing the outcomes of the cultural competency training is consistent with the nature and goals of our workshop. Although using a prethen-post questionnaire-based evaluation method is a standard practice in evaluation research, several researchers have recommended the use of the post-then-pre evaluation method, especially for assessing the outcomes of short-term training programs.20,21 In a typical pre-thenpost evaluation, participants are asked a set of questions at the beginning of a training program and then again after its completion. Conversely, in a post-then-pre evaluation, participants are asked a set of questions only once at the end of the training program. The questions are designed to stimulate participants to think about and to answer questions from 2 different frames of reference—after (post) and before (pre) the training program. Questions typically read: “After this training, I think/ will do X. Before this training, I thought/did X.” The post-then-pre evaluation method thus allows a consistent assessment of changes in participants’ knowledge or skills as a result of the training, while reducing response shift bias.22 Response shift bias occurs when respondents’ perspectives on the construct being measured change between the pretest and the posttest as a result of the influence of the training. Post-then-pre evaluation methods have been described by researchers as valid, versatile, and convenient means for controlling response shift bias inherent in the traditional, pre-then-post method because the respondents’ preperceptions and postperceptions are collected at the same time.23-27 Prior research also suggests that by using the postthen-pre method, researchers can limit the problems of over or ideal reporting generally associated with the standard pre-then-post evaluation method.20 Although suitable to the objectives and design of our research, a JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
retrospective post-then-pre evaluation does have some limitations. The major limitations of this evaluation method include recall bias—the inability of participants to accurately recall knowledge and skills before the training program—and social desirability bias where participants respond to fulfill the expectations of the training program.28
Data Analysis We tabulated and analyzed the survey data using descriptive statistics for demographic characteristics. To assess the change in knowledge and skills of participants as a result of the cultural competency training, we pooled and compared individual and mean responses using SPSS 15.0 for Windows (SPSS Inc, Chicago, Illinois). We grouped participants’ responses for all items into 2, “after training” (post) and “before training” (pre) categories, and then compared percentage distributions and mean values for all responses to examine participants’ self-reported change in knowledge and skills. We applied Wilcoxon signed rank tests to compare the mean scores of self-reported “after training” (post) and “before training” (pre) responses. These comparisons allowed us to determine whether the average differences between the 2 variables was significantly different from 0 and whether our training produced a statistically significant change in knowledge and skills related to cultural competency as evaluated by participants. We used the Wilcoxon rank test because the parameters of the population are not known, hence the use of nonparametric tests is more appropriate than that of the parametric t test.
RESULTS
Of the 60 health care providers and administrators who attended the cultural competency training, 43 completed the post-then-pre CCA survey, for a response rate of 72 %. A majority of participants identified themselves as Euro American. More women than men and more Table 2. Demographic Characteristics of the Study Participants (N = 43) Demographic characteristics Gender Female Male Profession Health care provider Health care administrator Bilingual Yes No Self-identified race/ethnicity Euro American Hispanic African American Asian
n (%) 32 (74.4) 11(25.6) 34(79) 9(21) 15(34.9) 28(65.1) 39(90.7) 2(4.7) 1(2.3) 1(2.3)
VOL. 101, NO. 9, SEPTEMBER 2009 889
Cultural competency in health care
health care providers than health care administrators participated in the study. Slightly more than one-third of the study participants described themselves as bilingual (Table 2). Participants had a mean of 14 years of experience (range, 1-35 years) in their respective professions.
Key Findings The purpose of our study was to assess change in participants’ knowledge and skills related to cultural competency in response to our training. The post-thenpre testing revealed that participants experienced a positive shift in their knowledge and skills pertaining to the provision of culturally competent health care. For all items in the questionnaire, except 1, the “after training” (post) mean scores are higher than the “before training” (pre) mean scores (Table 3). For item 2 (“Culture, race, and ethnicity can be interchangeably used”), the “after training” (post) mean value is lower than the “before training” (pre) mean value. This difference suggests that by attending the cultural competency training, participants acquired a more nuanced understanding of the different usages and meanings associated with the 3 terms. This shift is important considering that “after training,” fewer participants reported that the 3 terms can be interchangeably used. The participants self-reported improvements in their knowledge about the presence of systemic
discrimination that patients from diverse racial and ethnic backgrounds face (items 1b and 8), the role of cultural factors in patient-provider communication (item 7), and the importance of nonverbal cues or gestures during the patient-provider interaction (item 9h). The most noticeable shift in participants’ selfreported responses is observed in terms of asking patients about the use of folk remedies (item 9d), examining patients in a culturally appropriate manner (item 9e), and developing a culturally appropriate care plan (item 9f). The positive shift in the differences between “after training” (post) and “before training” (pre) mean scores for the 3 items is 1.9, 1.16, and 1.24, respectively. These changes suggest increased awareness among participants regarding the importance of learning about health beliefs and culturally specific healing practices while collecting case histories, of conducting clinical examination, and of developing a culturally appropriate care plan with patients from diverse cultural backgrounds. These changes reflect an increase in participants’ self-reported knowledge of the key explanatory models discussed during the cultural competency training and in their self-reported skills for soliciting and responding to patients’ explanatory models.
Table 3. Differences Between Participants’ Self-Reported Responses After (Post) and Before (Pre) Attending the Cultural Competency Training (N = 43)
Survey Item II. Knowledge 1a 1b 1c 1d 2 3 4 5 6 7 8 9a 9b 9c 9d 9e 9f 9g 9h III. Skills 10 11 12 13 14 a
After Training (Post) Mean (SD)
Before Training (Pre) Mean (SD)
Mean Difference (Post-Pre)
Wilcoxon Rank Test (p)
3.60 3.49 3.35 3.47 2.30 4.02 4.26 4.37 4.05 4.56 3.84 4.67 4.42 4.51 4.49 4.51 4.47 4.57 4.56
(.73) (.77) (.84) (.94) (1.2) (.89) (.76) (.58) (.72) (.67) (.79) (.47) (.63) (.63) (.55) (.59) (.67) (.59) (.55)
3.28 3.14 2.93 2.93 2.88 3.49 3.95 3.93 3.33 3.86 3.23 3.88 4.09 3.86 3.40 3.35 3.23 3.76 3.91
(.80) (.80) (.86) (.91) (1.1) (.91) (.74) (.86) (.89) (1.0) (.92) (1.1) (.90) (.94) (1.4) (1.3) (1.3) (1.1) (1.0)
0.32 0.35 0.42 0.54 -0.58 0.53 0.31 0.44 0.72 0.70 0.61 0.79 0.33 0.65 1.09 1.16 1.24 0.81 0.65
.010a .030a .011a .001a .016a .001a .042a .002a .000a .000a .001a .000a .025a .000a .000a .000a .000a .000a .000a
3.65 4.19 4.33 4.28 4.44
(.95) (.82) (.64) (.77) (.59)
2.60 2.65 2.91 2.88 3.26
(1.0) (.93) (.87) (.88) (1.1)
1.05 1.54 1.42 1.40 1.18
.000a .000a .000a .000a .000a
Statistically significant.
890 JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
VOL. 101, NO. 9, SEPTEMBER 2009
Cultural competency in health care
DISCUSSION
Cultural competency in health care is generally defined as a set of congruent skills, communication strategies, and policies that come together in a system, agency, or among professionals with the objective of facilitating effective delivery of services in cross-cultural situations.10 Prior research has placed central importance on the role of cultural competency training in improving the self-reported knowledge and skills of health care providers, enabling them to work effectively in cross-cultural situations.29-31 Implicit in many discussions about cultural competency in health care is the assumption that improvements in providers’ self-reported knowledge and skills as a result of cultural competency training will eventually lead to improvement in the delivery of culturally sensitive care and better health outcomes, especially for patients belonging to minority populations. Perhaps not surprisingly, health care organizations, including health management organizations, large hospitals systems, and small clinics and provider groups, are increasingly demanding cultural competency training for their health care professionals.4,5 Several studies suggest that cultural competency training improves participants’ self-reported understanding of knowledge and skills related to cultural competency in health care.29-31 However, innovative approaches to examining outcomes of cultural competency training are scarce and inconsistent. Our findings add to these studies by demonstrating that workshops that integrate key topics as recommended by the IOM and through the CLAS standards significantly improve self-reported knowledge and skills among health care providers and administrators As a result of cultural competency training, participants selfreported not only an enhanced understanding of the health care experiences of patients from diverse backgrounds but also an improvement in the skills necessary to effectively work in cross-cultural situations. In addition, our study offers an innovative strategy, post-then-pre method, for evaluating the outcomes of cultural competency training programs. Using a postthen-pre method offers several advantages over previous work. First and foremost, it is an evaluation method that is efficient in terms of both time and resources. Given the professional time constraints of a majority of health care providers, it may be more effective to evaluate the outcomes of a training workshop by using a post-thenpre method than via a traditional pre-then-post approach. Second, as discussed earlier in the article, evaluation researchers have reported that a post-then-pre method is more suitable than a pre-then-post method for evaluating the outcomes of a short training-based intervention, because it reduces response shift bias among respondents.23 Finally, researchers have established the postthen-pre method as a valid evaluation tool that reduces error related to ideal or overreporting.20 Indeed, cultural competency training has emerged as JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
an important strategy for better informing key health care stakeholders about the sociocultural dynamics inherent in health care–seeking behavior. It can serve, as this study demonstrates, as a means for improving the self-reported knowledge and skills required to effectively care for patients from diverse backgrounds. However, to be successful, cultural competency needs to be understood in the broadest possible sense as a dynamic process rather than as a 1-time, structured training endeavor lending itself to strict, quantitative evaluation. Discourses on cultural competency must encompass issues beyond racial and ethnic difference, expanding to include health issues that relate to sexual orientation, socioeconomic status, health insurance status, and type/timing of care. This demands a radical reexamination of the dynamics of diversity inherent within our society, and, thus, we argue for a larger vision of the role of cultural competency in health care. Keeping in mind the blurring of race/ethnicity, socioeconomic status, place of residence, sexual orientation, and the fact that every patient-provider interaction is a cross-cultural interaction, the scope of cultural competency in health care should be expanded to address multiple markers of difference. We strongly recommend that future research focus on establishing the role of cultural competency training in improving minority patients’ access to health care, adherence with medical advice, and in reducing miscommunication and racial and ethnic health disparities.
REFERENCES
1. Ohtaki S, Ohtaki T, Fetters M. Doctor-patient communication: a comparison of the USA and Japan. Fam Pract. 2002;20:276-282. 2. Ruiz-Moral R, Perez RE, Torres P, et al. Physician-patient communication: a study of observed behaviors of specialty physicians and the ways their patients perceive them. Pat Edu Counsel. 2006;64:242-248. 3. Bischoff A, Hudelson M, Bovier P. Doctor-patient gender concordance and patient satisfaction in interpreter-mediated consultations: an exploratory study. J Trav Med. 2008;12:1-5. 4. Batencourt JR, Green AR, Carrillo JE, et al. Cultural competence and health care disparities: key perspectives and trends. Health Aff. 2005;24:499-505. 5. Betancourt JR, Green AR, Carrillo JE. Cultural competency in health care: Emerging frameworks and practical approaches. New York, NY: The Commonwealth Fund; 2002. 6. Berger JT. Culture and ethnicity in clinical care. Arch Intern Med. 1998;159(12):2085-2090. 7. Welch TR. Culture and the patient-physician relationship: achieving cultural competency in health care. J Pediatr. 2000;136:14-23. 8. Patcher LM. Culture and clinical care: folk illness beliefs and behaviors and their implications for health care delivery. JAMA; 1994;271:690-694. 9. Deason LM, Wallace SL. Does a shared culture between nursing home staff and residents influence quality of life? Int J Div Org Com Nat. 2005;4:235-242. 10 Cross T, Bazron B, Dennis K, et al. Towards a culturally competent system of care: a monograph on effective services for minority children who are severely emotionally disturbed: Volume I, Washington, DC: Georgetown University Child Development Center; 1989. 11. Office of Minority Health. National standards for culturally and linguistically appropriate health care: final report. Washington, DC: US Department of Health and Human Services; 2001. 12. Davis-Floyd RE, Sargent CF, eds. Childbirth and authoritative knowledge: cross-cultural perspectives. Berkeley, CA: University of California Press; 1997.
VOL. 101, NO. 9, SEPTEMBER 2009 891
Cultural competency in health care 13. Coburn D, Willis E. The medical profession: knowledge, power, and autonomy. In: Albrecht GL, Fitzpatrick R, Scrimshaw SC, eds. The handbook of social studies in health & medicine. London, UK: Sage Publications; 2003:377-393. 14.Batencourt JR, Green AR, Carrillo JE, et al. Defining cultural competence: a practical framework for addressing racial/ethnic disparities in health and healthcare. Pub Heal Rep. 2003;118:293-302. 15. Green AR, Carrillo JE, Batencourt JR. Why the disease-based model of medicine fails our patients. West J Med. 2002;176:141-143. 16. Lavizzo-Mourey R, Mackenzie E. Cultural competence: an essential hybrid for delivering high quality care in the 1990s and beyond. Trans Am. Clin Climatol Assoc. 1996;VII:226-238. 17. Chenowethm L, Jeon YH, Goff M, et al. Cultural competency and nursing care: an Australian perspective. Int Nurs Rev. 2006;53:34-40. 18. Cook CT, Omofolasade K, Richard, OB. Satisfaction with and perceived cultural competency of healthcare providers: the minority experience. J Natl Med Assoc. 2005;97:1078-1086. 19. Institute of Medicine. Unequal treatment: confronting racial and ethnic disparities in health care. Washington, DC; 2002. 20. Rockwell SK and Harriet K. Post-Then-Pre Evaluation. J Exten. 1989:27. http://www.joe.org/joe/1989summer/a5.html. 21. Pratt CC, McGuigan WM, Katzev AR. Measuring program outcomes: using retrospective pretest methodology. Am J Eval. 2000;21:341-349.
892 JOURNAL OF THE NATIONAL MEDICAL ASSOCIATION
22. Davis G. Using retrospective pre-post questionnaire to determine program impact. J Exten. 2003:41. 23. Howard, GS. Response-shift bias a problem in evaluation interventions with pre/post self-reports. Eval Res. 1980;4:93-106. 24. Howard GS, Millham J, Slaten S, et al. Influence of subject response-style effects on retrospective measures. Appl Psychol Meas. 1981;5:144-150. 25. Howard GS, Ralph KM, Gulanick NA, et al. Internal invalidity in pre-testpost-test self-report evaluations and a re-examination of retrospective pretests. Appl Psychol Meas. 1979;3:1-23. 26. Hill LG, Betz DL. Revisiting the retrospective pretest. Am J Eval. 2005; 26:501-517. 27. Lam TC, Bengo P. A comparison of three retrospective self-reporting methods of measuring change in instructional practice. Am J Eval. 2003;24:65-80. 28. Colosi L, Dunifon R. What’s the difference? “post then pre” & “pre then post.” Cornell: Cornell Cooperative Extension; 2006.29. Crosson JC, Deng W, Brazeau C, et al. Evaluating the effect of cultural competency training on medical student attitudes. Fam Med. 2004;36:199-203. 30. Schim SM, Doorenbos AZ, Borse NN. Enhancing cultural competency among hospice staff. Am J Hosp Palliat Care. 2006;23:404-411. 31. O’Brien RL, Kosoko-Lasaki O, Cook CT, et al. Self-assessment of cultural attitudes and competence of clinical investigators to enhance recruitment and participation of minority populations in research. J Natl Med Assoc. 2006;98:674-682. n
VOL. 101, NO. 9, SEPTEMBER 2009