- Email: [email protected]
What patients like about their medical care and how often they are asked: A meta-analysis of the satisfaction literature
Sot. Sci. Med. Vol. 27, No. 9. pp. 935-939. 1988
0277-9536188 $3.00+ 0.00 Copyright C 1988Pcrgamon Press plc
Printed in Great Bntain. All rights reserved
WHAT PATIENTS LIKE ABOUT THEIR MEDICAL CARE AND HOW OFTEN THEY ARE ASKED: A META-ANALYSIS OF THE SATISFACTION LITERATURE JUDITH A. HALL’,~* and MICHAEL C. DORNAN’ ‘Institute for Health Research, a joint program of the Harvard Community Health Plan and Harvard University, Boston. MA 02115 and *Department of Psychology, Northeastern University, Boston, MA 02115 and Department of Social Medicine and Health Policy, Harvard Medical School. Boston, MA 02115, U.S.A. Abstract-A meta-analysis of studies on consumer satisfaction with medical care yielded 221 studies that reported relations between satisfaction and other variables, of which 107 reported satisfaction levels for two or more aspects of medical care. A method was developed to derive the relative levels of patient satisfaction with 11aspects of care across these 107 studies. Humaneness and technical quality of medical care were ranked near the top, while the bottom five ranks were occupied by aspects of care that reflected the provider’s attention to other patient needs and the patient’s relation to the system as a whole. In addition, it was demonstrated that different aspects of medical care are measured with extremely uneven frequencies in satisfaction instruments.
Efforts to describe a rank ordering of satisfaction levels for different aspects of care across the available studies meet a methodological problem, however. This problem is that different studies include different aspects in their satisfaction instruments. This precludes applying the method of summed ranks, a simple and commonly used way of deriving one ranking from several sources [lo]. With that method, one would simply rank each aspect within each study in terms of its satisfaction level, sum those ranks over studies, and then rank the sums. But to our knowledge there existed no statistical method appropriate for the satisfaction case, where different aspects appeared in different studies. Therefore a new ranking statistic was developed (by M.C.D.), which we describe in detail below.
In the past 20 years many investigators have reported on consumers’ satisfaction and evaluations regarding the medical care they receive. This literature has been reviewed intermittently [l-6]. Some of these reviews stand out in terms of the use of quantitative methods and comprehensiveness [ 1,2]. Qualitative (that is, narrative) reviews are often insightful but are inefficient when there is a large literature, and they do not allow statistical grounding of relations perceived across studies or of differences perceived between studies. We have conducted a meta-analysis [7-91 of research on satisfaction with medical care. This metaanalysis permits the examination of many important questions. In this article we present some results pertaining to multidimensionality in satisfaction instruments. Other writers have summarized what dimensions (aspects of care) have been inquired about with regard to satisfaction, but no previous research has summarized either the frequency with which different aspects have been measured or the levels of satisfaction obtained for different aspects. By aspects we mean such features of care as access, quality, and cost, among others. One implication of multidimensionality is that differences between consumers in values and experiences across the multiple aspects or dimensions of care can be expressed. Another implication is that, on average, consumers may express more satisfaction with some aspects of care than with others. To ascertain such differences would be a useful goal for health services researchers because of the likely relevance of patient satisfaction to an understanding of the performance of health care systems.
METHOD Criteria for inclusion and search methods
*Please address correspondence to: Judith A. Hall, Department of Psychology, Northeastern University, 360 Huntington Avenue, Boston, MA 02115, U.S.A.
To be included in the meta-analysis, a study had to meet the following criteria: (1) the investigators quantitatively measured satisfaction with medical care; (2) the association between satisfaction and at least one other variable was reported; (3) the study was published in an English-language journal, book, or government document; (4) sample size was 10 or greater. Excluded as measures of satisfaction were patients’ evaluations of mental health care and dentistry, and patients’ stated values and preferences regarding health care, if these were not tied to an evaluation of their actual experiences. Included in the metaanalysis were a few studies of analogue design (e.g. measuring satisfaction with simulated medical care). To locate studies, the following search methods were used: (I) online database searches (Medline and Psychological Abstracts), using the keywords ‘consumer satisfaction’ for the years 1966-1986; (2) review of the bibliographies of other reviews; and (3)
935
936
JUDITH
A. HALL and MICHAELC.
manual search of six key journals (Medical Care, Journal of Health and Social Behavior, Social Science and Medicine, American Journal of Public Health, Journal of Community Health, and Journal of Family Practice).
In compiling the data set, a distinction was drawn between an article and a study. An article, as we use the term, is a published work (journal article, book chapter, book), whereas a study is comprised of all the analyses of a given group of research subjects. An article may contain more than one study. Conversely, a given study may be described in more than one article. When identical results pertaining to the satisfaction data of one study were published in more than one article, only one article from such a set of duplicates was included in the meta-analysis bibliography. When different results pertaining to the satisfaction data of one study were published in more than one article, all such articles were included, but for computing purposes the data were amalgamated to reflect the fact that only one sample of subjects was involved. Coding of aspects of satisfaction
A coding form was developed for identifying which aspects, or dimensions, of satisfaction were inquired about in each study. The following categories were developed after reviewing the methodologies used in a large subset of studies. This list corresponds generally with classifications offered by other writers [I, 111. a. Overall satisfaction. This could be a global item or items, or a composite based on unidentified aspects. b. Satisfaction with access. This included convenience, hours, distance, perceived availability, and ease of getting appointments. c. Satisfaction with cost. d. Satisfaction with overall quality. This category was less inclusive than ‘overall satisfaction’, but more inclusive than satisfaction with ‘humaneness’ and ‘competence’ (see below). Included was the amount of time spent with providers. e. Satisfaction with humaneness. Included were warmth, respect, kindness, willingness to listen, appropriate nonverbal behaviors, and interpersonal skill. f. Satisfaction with competence. This category involved technical performance and competence definable in traditional medical terms. g. Satisfaction with the amount of information supplied by the provider (e.g. explanations of treatments, procedures, or diagnoses). h. Satisfaction with bureaucratic arrangements and members of the bureaucracy, including waiting times at the site. i. Satisfaction with physical facilities, including aesthetic and functional aspects, parking, and adequacy of equipment and laboratories. j. Satisfaction with the provider’s attention to psychosocial problems of the patient.
DORNAN
k. Satisfaction 1. Satisfaction
with continuity of care. with the outcome of care.
Two-hundred and twenty-one studies published in 230 articles were coded for these aspects. A subset of these studies (106) provided mean responses to each satisfaction item or aspect, and these, along with one additional study that ranked the mean satisfaction levels across aspects but did not report means, formed the basis of the analysis of ranks described below. In screening studies for the ranking analysis, we excluded studies if they did not include at least two aspects of care in the satisfaction instrument. The aspect called ‘overall satisfaction’ was not included.* Calculation of overall ranking
For each study, a11satisfaction items were classified by aspect, and the average response to each item was recorded. If results were reported in terms of rating scales (for example, ‘very dissatisfied. . very satiswe fied’, ‘disagree strongly . . agree strongly’), recorded the mean for each such item. If results were reported in terms of the proportion of patients who were ‘satisfied’ (defined consistently across items within a given study), we recorded those proportions. (The proportion of ‘satisfied’ patients can be considered a mean. If ‘satisfied vs not’ is considered as a scale with values 1 and 0, the mean of this variable would be the same as the proportion ‘satisfied’.) If several items within a study represented the same aspect, the means for those items were averaged (weighted by N of patients for each item), to ensure that a given aspect was associated with only one mean in each study. Then a new statistic was applied which yielded a single ranking for the 11 aspects over the 107 studies. The first step in producing the overall ranking involved calculating a ‘position score’ for each aspect across studies. To do this we first identified for each study that included a given aspect (let us call it aspect x, for example satisfaction with the provider’s humaneness) the number of other aspects included and the rank of aspect x within the study. To illustrate the calculation of the position score, assume that there are only 3 studies, and that aspect x ranks first out of 5 aspects in study A, third out of 11 aspects in study B, and last out of 6 aspects in study C. The best aspect x could do is to achieve rank 1 in all studies. Summing across studies, the best score would be (1 + 1 + 1) = 3. The worst aspect x could do is to achieve the lowest rank in all studies, or (5 + 11 + 6) = 22. Its actual place is (1 + 3 + 6) = IO. The calculation of the position score is based on the ratio of the difference between actual and worst to the difference between best and worst. In this case it would be 22-10 12 = G = 0.63. 22 - 3 Stated generally, aspect is:
the position
score for a given
a-b a-c’
*Both bibliographies (TV= 230 and 107) are available from the first author, along with an annotation of which aspects of satisfaction were measured in each study.
where a = sum of worst possible ranks, b = sum of actual ranks achieved by a given aspect, and c = sum
A meta-analysis of the satisfaction literature of best possible ranks (equal to N of studies involving that aspect). The position score can range from 0.00 to 1.00, with 0.00 meaning a given aspect always achieved the worst rank in each study in which it was used, and 1.00 meaning it always achieved the best. Once the position scores of all aspects are calculated, they are ranked to yield an overall indication of patients’ relative satisfaction with each of the 11 aspects, based on all 107 studies. RESULTS
Ranking of aspects across studies
Table 1 presents the overall ranking of satisfaction with I1 aspects of medical care (excluding ‘overall’ satisfaction). The table reveals that the position score of satisfaction with attention to psychosocial problems was extremely low (0.15) and rather close to the minimum possible (0.00). On the other hand, the position score of the highest-ranked aspect, satisfaction with overall quality, was only 0.69, not nearly as close to the maximum possible score of 1.00. This could indicate greater inconsistency across studies in the top-ranked aspects than in the bottom-ranked aspects. After calculating the ranking, we tested for two possible sources of spuriousness. The first concerned the possible effects of the item mix within studies. If an aspect were paired mainly with aspects that received low satisfaction, such an aspect would receive a higher rank than if it were paired predominantly with aspects receiving higher satisfaction. We tested this possibility in the following manner. There were 418 mean satisfaction scores generated from the 107 studies. We divided the aspects into ‘high-ranking’ (aspects d, e, I; 1, i) and ‘low-ranking’ (aspects b, g, c, h, j, k). For each aspect we counted up the total number of comparisons made between it and other aspects, calculated the proportion of these comparisons that involved high-ranking aspects, and ranked the resulting 11 proportions. This ranking was then correlated with the overall ranking shown in Table 1. A significant correlation would indicate that an aspect’s position in the overall ranking is influenced by the kinds of items it was paired with. The rank-order correlation (rho) was 0.18 (df = 9, not significant). Thus, the hypothesis that the rank order of aspects is explained by item mix is not supported. The second possible source of spuriousness concerned the number of aspects represented in a study. Our ranking statistic is, by intention, sensitive to the number of other items an aspect is compared to within a study, so that, for example, achieving a rank of 2 out of a possible 10 would count toward a higher position score than a rank of 2 out of a possible 4. It was possible, therefore, that some aspects could receive low or high position scores solely because they were compared with few or many other aspects, respectively. To test for this artifact we calculated, for each aspect, the mean number of other aspects it was compared with. These means were ranked and correlated with the overall ranking in Table 1. As above, a non-zero correlation would indicate a con-
937
Table I. Overall ranking of satisfaction with medical care (N = 107 studies) Aspects
I I aspects of
Position score
Rank
0.69 0.66 0.63 0.60 0.52 0.48 0.42 0.40 0.28 0.24
2 3 4 5 6 7 8 9 IO
0.15
11
Overall quality Humaneness Competence Outcome Facilities Continuity of care Access Informativeness cost Bureaucracy Attention to psychosocial problems
I
Note: mean number of aspects measured per study was 3.91 (range = 2-9).
founding effect, in this case involving the number of aspects used in given studies. The rank-order correlation (rho) was -0.14 (df = 9, not significant). The hypothesis that the rank order of aspects is explained by the number of items an aspect was compared to is not supported. Frequency with which each aspect was measured
Table 2 presents the percentages and frequencies of studies addressing each aspect of satisfaction. These percentages show considerable variation. The most frequently measured aspects of satisfaction were the provider’s humaneness and informativeness; these were measured in 65% and 50% of studies, respectively. The least frequently measured were psychosocial problem discussion (3%), continuity (4%), and outcome (6%). Though no prediction was made concerning the relation between relative satisfaction levels and the frequency with which different aspects have been measured, the rank-order correlation (rho) between the Tables 1 and 2 results was calculated, using the 11 aspects as cases. This correlation was 0.47, P = 0.12, 2-tailed test. Though intriguing in magnitude, this was not statistically significant. DISCUSSlON
Research on consumer satisfaction with medical care has burgeoned since the early 1970s. We located 221 studies that included correlates of satisfaction. This is only a subset of the literature; many studies
Table 2. Percentages and frequencies of studies in which different aspects of satisfaction were measured (N = 221 studies) Aspects Humaneness Informativeness Overall quality Competence Overall Bureaucracy Access cost Facilities Outcome Continuity Attention to psychosocial problems
Percentage
Frequency
65 50 45 43 43 28 27 18 16 6 4
143 III 100 95 % 61 59 40 36 13 10
3
7
938
JUDITH
A. HALLand MICHAEL C.
that reported on satisfaction without relating satisfaction to any other variables were excluded. Because conventional methods of review are not adequate for a literature of this size, a meta-analysis was undertaken. In this article we have presented an analysis of how satisfied consumers are, relatively, with these aspects based on data from 107 studies, and a description of how often different aspects of satisfaction were measured. We present for the first time a ranking of satisfaction with different aspects of care based on a literature-wide analysis. It is interesting to speculate on what the ranking tells us about patients’ experiences and values. Descriptively, we found humaneness, technical, and overall quality to be ranked near the top. The bottom five ranks were occupied by aspects representing attention to other non-physical patient needs (the need for information and the need to have psychosocial problems addressed), and aspects involving the patient’s relation to the system, not the provider (access, cost, and bureaucracy). One interpretation of this pattern would take the relative satisfaction with different aspects as evidence for the actual performance of the system, such that higher satisfaction reflects better system performance [l 11. Seen in this light, the fact that greater relative satisfaction occurred for technical quality could mean that health care systems emphasize technical performance to the relative neglect of patient needs that fall outside of a biomedical definition of health. The latter would include emotional and cognitive needs, the need to have affordable care, and the need to avoid irritation and delay. Given evidence of important links between physiological health and ‘nonmedical’ factors such as emotional well-being, stress, and compliance, such a resource allocation could of course reflect an overly narrow approach to health and illness. Another interpretation of the ranking is that patients automatically give high ratings to technical quality, either because they feel they cannot judge it very well or because it is threatening to contemplate that the care one chose is not of the highest quality [12]. Though patients may thus defer judgment on technical quality when making satisfaction ratings, giving high ratings by default, they may be discriminating when it comes to the remaining aspects of medical care. Under this interpretation, the satisfaction ranking reflects actual quality only for the non-technical aspects of care. If this interpretation is correct, it would be somewhat ironical in that recent evidence suggests that, up to a point at least, patients cm judge technical quality. Several studies indicate that satisfaction ratings correlate positively with expert-developed indices of technical quality [13-IS]. The aspects of satisfaction we identified were measured with very uneven frequencies. One can only
*Studies asking about global
or overall satisfaction may indeed reflect degrees of satisfaction with this aspect. However, the methodologic trend is away from global questions and toward the development of multidimensional satisfaction scales.
DORNAN
speculate on the reasons for this. One possibility is that the less frequently studied aspects tend to be structural-for example, cost, access, bureaucracy, and facilities. Investigators may assume these are not as ameliorable as are aspects relating to provider behavior. They may also assume that structural aspects are not as important to patients and do not contribute as heavily to overall satisfaction. Regardless of why some aspects are represented more than others, it is important to recognize that these disparities limit our undersanding of what satisfaction is, as operationally defined. As an extreme example, it is obvious that satisfaction, as measured, is nor about psychosocial problem discussion, since only 3% of studies asked about it.* Considering the prevalence of unresolved psychosocial problems [16] and their impact on health care utilization [ 171,this could represent a serious obstacle to our understanding of ‘true’ satisfaction levels and their implications. Though content validity of instruments is increased by a more exhaustive sampling of aspects [l 11,there remains the question of how to assign weights to them when forming composites or total scores. Alternatives include weighting them equally (as on the Client Satisfaction Questionnaire [ 1S]), weighting by each patient’s personal evaluations of the importance of each aspect (as on the Evaluation Ranking Scale [19]), weighting by a normative patient group’s evaluations of the importance of each aspect (as on the satisfaction with nursing care instrument of Abdellah and Levine [20]), weighting by each aspect’s correlation with a global satisfaction item or scale (used in early trials of the Abdellah and Levine scale [20]), and weighting using factor weights derived from factor analysis of an instrument’s items. These different options have different implications for our definition of overall ‘satisfaction’, and there currently exists no consensus on which is preferable. Some authors choose not to combine items from different domains into a total score [ 11,211. The fact that different instruments include different aspects of satisfaction also raises questions about the comparability of ‘satisfaction’ from one study to the next. Factor analytic studies of instruments and other analyses of internal consistency sometimes show strong common factors, reflecting high intercorrelations among aspects [22,23]. This may relieve our concern that instruments are not comparable, i.e. not tapping a common construct. However, it cannot be safely assumed that results of factor analytic research have guided most investigators in their choice of aspects, and furthermore it is not totally clear that satisfaction items are interchangeable. Some writers argue that various aspects, or dimensions, are distinct [I, 111. In conclusion, it is evident that rigorous scrutiny of the satisfaction literature can lead to new insights about the nature and meaning of satisfaction instruments. The present article reveals gaps in the literature regarding the domain of study as well as features of medical care that are relatively more and less pleasing to patients. These results are relevant to directions for future investigations as well as to health policy, especially the evaluation of system performance.
A meta-analysis of the satisfaction literature Acknowledgemenrs-This research was supported by core funds from the Institute for Health Research. Thanks are extended to Michael Feldstein, Jonathan Brown, Howard Frazier, Bert Green, and David Hemenway for their comments.
REFERENCES
1. Pasco G. C. Patient satisfaction in primary health care: a literature review and analysis. .&xl. Prog. Plan. 6, 185, 1983. 2. Ware J. E. Jr, Davies-Avery A. and Stewart A. L. The measurement and meaning of patient satisfaction. Hlth Med. Cure Serv. Rev. 1, 2, 1978. Ley P. Satisfaction, compliance and communication. Br. J. c/in. Psychol. 21, 241, 1982. Lebow J. L. Consumer assessments of the quality of medical care. Med. Cure 12, 328, 1974. Lochman J. E. Factors related to patients’ satisfaction with their medical care. J. Commun. Hfrh 9, 91, 1983. Locker D. and Dunt D. Theoretical and methodological issues in sociological studies of consumer satisfaction with medical care. Sot. Sci. Med. 12, 283, 1978. 7. Green B. F. and Hall J. A. Quantitative methods for literature reviews. A. Rev. Psychol. 35, 37, 1984. Procedures for Social 8. Rosenthal R. Metu-Analytic Research. Sage, Beverly Hills, Calif., 1984. _ 9. Louis T. A. Finebertz H. V. and Mosteller F. Findinns for public health fro& meta-analysis. A. Rev. Publ. Hih 6, 1, 1985. Methods, 2nd edn. 10. Guilford J. P. Psychometric McGraw-Hill, New York, 1954. Il. Ware J. E. Jr, Snyder M. K., Wright W. R. and Davies A. R. Defining and measuring patient satisfaction with medical care. Euul. Proa. Plan. 6, 247, 1983. 12. Aronson E. Dissonancetheory: progress and problems. In Theories of Cognifive Consistency: A Sourcebook
939
(Edited by Abelson R. P. et al.). Rand McNally, Chicago, Ill. 1968. 13. Roter D. L., Hall J. A. and Katz N. R. Relations of physicians’ behaviors to analogue patients’ satisfaction, recall, and impressions. Med. Cure 2S. 437. 1987. 14. Willson P. and McNamara J. R. How.pe&ptions of a simulated physician-patient interaction influence intended satisfaction and compliance. Sot. Sci. Med. 16, 1699, 1982. 15. Linn B. S. Bum patients’ evaluation of emergency department care. Ann. Emerg. Med. 11, 255, 1982. 16. Cohen-Cole S., Bird J., Baker J., Freeman A., Polmer R. and Shugerman A. Psychiatry for internists: a study of needs. J. Opl Psychiut. 13, 100, 1982.
17. Kessler L., Tessler R. and Nycz G. Co-occurrence of psychiatric and medical morbidity in primary care. J. Fum. Pruct. 16, 319, 1983. _ _ 18. Pascoe G. C. Attkisson C. C. and Roberts R. E. Comparison of indirect and direct approaches to measuring patient satisfaction. Euul. Prog. Plan. 6, 359, 1983.
19. Pascoe G. C. and Attkisson C. C. The Evaluation Ranking Scale: a new methodology for assessing satisfaction. Euul. Prog. Plan. 6, 335, 1983. 20. Abdellah F. G. and Levine E. Developing a measure of patient and personnel satisfaction with nursing care. Nurs. Res. 5, 100, 1957. 21. Zyxanski S. J., Hulka B. S. and Cassel J. C. Scale for the measurement of “satisfaction” with medical care: modifications in content, format and scoring. Med. Cure 12, 61 I, 1974.
22. Ware J. E. Jr and Snyder M. K. Dimensions of patient attitudes regarding doctors and medical care services. Med. Cure 13, 669, 1975.
23. DiMatteo M. R. and Havs R. The sienificance of patients’ perceptions of physician conduci: a study of patient satisfaction in a family practice center. J. Commun. Hith 6, 18, 1980.
0277-9536188 $3.00+ 0.00 Copyright C 1988Pcrgamon Press plc
Printed in Great Bntain. All rights reserved
WHAT PATIENTS LIKE ABOUT THEIR MEDICAL CARE AND HOW OFTEN THEY ARE ASKED: A META-ANALYSIS OF THE SATISFACTION LITERATURE JUDITH A. HALL’,~* and MICHAEL C. DORNAN’ ‘Institute for Health Research, a joint program of the Harvard Community Health Plan and Harvard University, Boston. MA 02115 and *Department of Psychology, Northeastern University, Boston, MA 02115 and Department of Social Medicine and Health Policy, Harvard Medical School. Boston, MA 02115, U.S.A. Abstract-A meta-analysis of studies on consumer satisfaction with medical care yielded 221 studies that reported relations between satisfaction and other variables, of which 107 reported satisfaction levels for two or more aspects of medical care. A method was developed to derive the relative levels of patient satisfaction with 11aspects of care across these 107 studies. Humaneness and technical quality of medical care were ranked near the top, while the bottom five ranks were occupied by aspects of care that reflected the provider’s attention to other patient needs and the patient’s relation to the system as a whole. In addition, it was demonstrated that different aspects of medical care are measured with extremely uneven frequencies in satisfaction instruments.
Efforts to describe a rank ordering of satisfaction levels for different aspects of care across the available studies meet a methodological problem, however. This problem is that different studies include different aspects in their satisfaction instruments. This precludes applying the method of summed ranks, a simple and commonly used way of deriving one ranking from several sources [lo]. With that method, one would simply rank each aspect within each study in terms of its satisfaction level, sum those ranks over studies, and then rank the sums. But to our knowledge there existed no statistical method appropriate for the satisfaction case, where different aspects appeared in different studies. Therefore a new ranking statistic was developed (by M.C.D.), which we describe in detail below.
In the past 20 years many investigators have reported on consumers’ satisfaction and evaluations regarding the medical care they receive. This literature has been reviewed intermittently [l-6]. Some of these reviews stand out in terms of the use of quantitative methods and comprehensiveness [ 1,2]. Qualitative (that is, narrative) reviews are often insightful but are inefficient when there is a large literature, and they do not allow statistical grounding of relations perceived across studies or of differences perceived between studies. We have conducted a meta-analysis [7-91 of research on satisfaction with medical care. This metaanalysis permits the examination of many important questions. In this article we present some results pertaining to multidimensionality in satisfaction instruments. Other writers have summarized what dimensions (aspects of care) have been inquired about with regard to satisfaction, but no previous research has summarized either the frequency with which different aspects have been measured or the levels of satisfaction obtained for different aspects. By aspects we mean such features of care as access, quality, and cost, among others. One implication of multidimensionality is that differences between consumers in values and experiences across the multiple aspects or dimensions of care can be expressed. Another implication is that, on average, consumers may express more satisfaction with some aspects of care than with others. To ascertain such differences would be a useful goal for health services researchers because of the likely relevance of patient satisfaction to an understanding of the performance of health care systems.
METHOD Criteria for inclusion and search methods
*Please address correspondence to: Judith A. Hall, Department of Psychology, Northeastern University, 360 Huntington Avenue, Boston, MA 02115, U.S.A.
To be included in the meta-analysis, a study had to meet the following criteria: (1) the investigators quantitatively measured satisfaction with medical care; (2) the association between satisfaction and at least one other variable was reported; (3) the study was published in an English-language journal, book, or government document; (4) sample size was 10 or greater. Excluded as measures of satisfaction were patients’ evaluations of mental health care and dentistry, and patients’ stated values and preferences regarding health care, if these were not tied to an evaluation of their actual experiences. Included in the metaanalysis were a few studies of analogue design (e.g. measuring satisfaction with simulated medical care). To locate studies, the following search methods were used: (I) online database searches (Medline and Psychological Abstracts), using the keywords ‘consumer satisfaction’ for the years 1966-1986; (2) review of the bibliographies of other reviews; and (3)
935
936
JUDITH
A. HALL and MICHAELC.
manual search of six key journals (Medical Care, Journal of Health and Social Behavior, Social Science and Medicine, American Journal of Public Health, Journal of Community Health, and Journal of Family Practice).
In compiling the data set, a distinction was drawn between an article and a study. An article, as we use the term, is a published work (journal article, book chapter, book), whereas a study is comprised of all the analyses of a given group of research subjects. An article may contain more than one study. Conversely, a given study may be described in more than one article. When identical results pertaining to the satisfaction data of one study were published in more than one article, only one article from such a set of duplicates was included in the meta-analysis bibliography. When different results pertaining to the satisfaction data of one study were published in more than one article, all such articles were included, but for computing purposes the data were amalgamated to reflect the fact that only one sample of subjects was involved. Coding of aspects of satisfaction
A coding form was developed for identifying which aspects, or dimensions, of satisfaction were inquired about in each study. The following categories were developed after reviewing the methodologies used in a large subset of studies. This list corresponds generally with classifications offered by other writers [I, 111. a. Overall satisfaction. This could be a global item or items, or a composite based on unidentified aspects. b. Satisfaction with access. This included convenience, hours, distance, perceived availability, and ease of getting appointments. c. Satisfaction with cost. d. Satisfaction with overall quality. This category was less inclusive than ‘overall satisfaction’, but more inclusive than satisfaction with ‘humaneness’ and ‘competence’ (see below). Included was the amount of time spent with providers. e. Satisfaction with humaneness. Included were warmth, respect, kindness, willingness to listen, appropriate nonverbal behaviors, and interpersonal skill. f. Satisfaction with competence. This category involved technical performance and competence definable in traditional medical terms. g. Satisfaction with the amount of information supplied by the provider (e.g. explanations of treatments, procedures, or diagnoses). h. Satisfaction with bureaucratic arrangements and members of the bureaucracy, including waiting times at the site. i. Satisfaction with physical facilities, including aesthetic and functional aspects, parking, and adequacy of equipment and laboratories. j. Satisfaction with the provider’s attention to psychosocial problems of the patient.
DORNAN
k. Satisfaction 1. Satisfaction
with continuity of care. with the outcome of care.
Two-hundred and twenty-one studies published in 230 articles were coded for these aspects. A subset of these studies (106) provided mean responses to each satisfaction item or aspect, and these, along with one additional study that ranked the mean satisfaction levels across aspects but did not report means, formed the basis of the analysis of ranks described below. In screening studies for the ranking analysis, we excluded studies if they did not include at least two aspects of care in the satisfaction instrument. The aspect called ‘overall satisfaction’ was not included.* Calculation of overall ranking
For each study, a11satisfaction items were classified by aspect, and the average response to each item was recorded. If results were reported in terms of rating scales (for example, ‘very dissatisfied. . very satiswe fied’, ‘disagree strongly . . agree strongly’), recorded the mean for each such item. If results were reported in terms of the proportion of patients who were ‘satisfied’ (defined consistently across items within a given study), we recorded those proportions. (The proportion of ‘satisfied’ patients can be considered a mean. If ‘satisfied vs not’ is considered as a scale with values 1 and 0, the mean of this variable would be the same as the proportion ‘satisfied’.) If several items within a study represented the same aspect, the means for those items were averaged (weighted by N of patients for each item), to ensure that a given aspect was associated with only one mean in each study. Then a new statistic was applied which yielded a single ranking for the 11 aspects over the 107 studies. The first step in producing the overall ranking involved calculating a ‘position score’ for each aspect across studies. To do this we first identified for each study that included a given aspect (let us call it aspect x, for example satisfaction with the provider’s humaneness) the number of other aspects included and the rank of aspect x within the study. To illustrate the calculation of the position score, assume that there are only 3 studies, and that aspect x ranks first out of 5 aspects in study A, third out of 11 aspects in study B, and last out of 6 aspects in study C. The best aspect x could do is to achieve rank 1 in all studies. Summing across studies, the best score would be (1 + 1 + 1) = 3. The worst aspect x could do is to achieve the lowest rank in all studies, or (5 + 11 + 6) = 22. Its actual place is (1 + 3 + 6) = IO. The calculation of the position score is based on the ratio of the difference between actual and worst to the difference between best and worst. In this case it would be 22-10 12 = G = 0.63. 22 - 3 Stated generally, aspect is:
the position
score for a given
a-b a-c’
*Both bibliographies (TV= 230 and 107) are available from the first author, along with an annotation of which aspects of satisfaction were measured in each study.
where a = sum of worst possible ranks, b = sum of actual ranks achieved by a given aspect, and c = sum
A meta-analysis of the satisfaction literature of best possible ranks (equal to N of studies involving that aspect). The position score can range from 0.00 to 1.00, with 0.00 meaning a given aspect always achieved the worst rank in each study in which it was used, and 1.00 meaning it always achieved the best. Once the position scores of all aspects are calculated, they are ranked to yield an overall indication of patients’ relative satisfaction with each of the 11 aspects, based on all 107 studies. RESULTS
Ranking of aspects across studies
Table 1 presents the overall ranking of satisfaction with I1 aspects of medical care (excluding ‘overall’ satisfaction). The table reveals that the position score of satisfaction with attention to psychosocial problems was extremely low (0.15) and rather close to the minimum possible (0.00). On the other hand, the position score of the highest-ranked aspect, satisfaction with overall quality, was only 0.69, not nearly as close to the maximum possible score of 1.00. This could indicate greater inconsistency across studies in the top-ranked aspects than in the bottom-ranked aspects. After calculating the ranking, we tested for two possible sources of spuriousness. The first concerned the possible effects of the item mix within studies. If an aspect were paired mainly with aspects that received low satisfaction, such an aspect would receive a higher rank than if it were paired predominantly with aspects receiving higher satisfaction. We tested this possibility in the following manner. There were 418 mean satisfaction scores generated from the 107 studies. We divided the aspects into ‘high-ranking’ (aspects d, e, I; 1, i) and ‘low-ranking’ (aspects b, g, c, h, j, k). For each aspect we counted up the total number of comparisons made between it and other aspects, calculated the proportion of these comparisons that involved high-ranking aspects, and ranked the resulting 11 proportions. This ranking was then correlated with the overall ranking shown in Table 1. A significant correlation would indicate that an aspect’s position in the overall ranking is influenced by the kinds of items it was paired with. The rank-order correlation (rho) was 0.18 (df = 9, not significant). Thus, the hypothesis that the rank order of aspects is explained by item mix is not supported. The second possible source of spuriousness concerned the number of aspects represented in a study. Our ranking statistic is, by intention, sensitive to the number of other items an aspect is compared to within a study, so that, for example, achieving a rank of 2 out of a possible 10 would count toward a higher position score than a rank of 2 out of a possible 4. It was possible, therefore, that some aspects could receive low or high position scores solely because they were compared with few or many other aspects, respectively. To test for this artifact we calculated, for each aspect, the mean number of other aspects it was compared with. These means were ranked and correlated with the overall ranking in Table 1. As above, a non-zero correlation would indicate a con-
937
Table I. Overall ranking of satisfaction with medical care (N = 107 studies) Aspects
I I aspects of
Position score
Rank
0.69 0.66 0.63 0.60 0.52 0.48 0.42 0.40 0.28 0.24
2 3 4 5 6 7 8 9 IO
0.15
11
Overall quality Humaneness Competence Outcome Facilities Continuity of care Access Informativeness cost Bureaucracy Attention to psychosocial problems
I
Note: mean number of aspects measured per study was 3.91 (range = 2-9).
founding effect, in this case involving the number of aspects used in given studies. The rank-order correlation (rho) was -0.14 (df = 9, not significant). The hypothesis that the rank order of aspects is explained by the number of items an aspect was compared to is not supported. Frequency with which each aspect was measured
Table 2 presents the percentages and frequencies of studies addressing each aspect of satisfaction. These percentages show considerable variation. The most frequently measured aspects of satisfaction were the provider’s humaneness and informativeness; these were measured in 65% and 50% of studies, respectively. The least frequently measured were psychosocial problem discussion (3%), continuity (4%), and outcome (6%). Though no prediction was made concerning the relation between relative satisfaction levels and the frequency with which different aspects have been measured, the rank-order correlation (rho) between the Tables 1 and 2 results was calculated, using the 11 aspects as cases. This correlation was 0.47, P = 0.12, 2-tailed test. Though intriguing in magnitude, this was not statistically significant. DISCUSSlON
Research on consumer satisfaction with medical care has burgeoned since the early 1970s. We located 221 studies that included correlates of satisfaction. This is only a subset of the literature; many studies
Table 2. Percentages and frequencies of studies in which different aspects of satisfaction were measured (N = 221 studies) Aspects Humaneness Informativeness Overall quality Competence Overall Bureaucracy Access cost Facilities Outcome Continuity Attention to psychosocial problems
Percentage
Frequency
65 50 45 43 43 28 27 18 16 6 4
143 III 100 95 % 61 59 40 36 13 10
3
7
938
JUDITH
A. HALLand MICHAEL C.
that reported on satisfaction without relating satisfaction to any other variables were excluded. Because conventional methods of review are not adequate for a literature of this size, a meta-analysis was undertaken. In this article we have presented an analysis of how satisfied consumers are, relatively, with these aspects based on data from 107 studies, and a description of how often different aspects of satisfaction were measured. We present for the first time a ranking of satisfaction with different aspects of care based on a literature-wide analysis. It is interesting to speculate on what the ranking tells us about patients’ experiences and values. Descriptively, we found humaneness, technical, and overall quality to be ranked near the top. The bottom five ranks were occupied by aspects representing attention to other non-physical patient needs (the need for information and the need to have psychosocial problems addressed), and aspects involving the patient’s relation to the system, not the provider (access, cost, and bureaucracy). One interpretation of this pattern would take the relative satisfaction with different aspects as evidence for the actual performance of the system, such that higher satisfaction reflects better system performance [l 11. Seen in this light, the fact that greater relative satisfaction occurred for technical quality could mean that health care systems emphasize technical performance to the relative neglect of patient needs that fall outside of a biomedical definition of health. The latter would include emotional and cognitive needs, the need to have affordable care, and the need to avoid irritation and delay. Given evidence of important links between physiological health and ‘nonmedical’ factors such as emotional well-being, stress, and compliance, such a resource allocation could of course reflect an overly narrow approach to health and illness. Another interpretation of the ranking is that patients automatically give high ratings to technical quality, either because they feel they cannot judge it very well or because it is threatening to contemplate that the care one chose is not of the highest quality [12]. Though patients may thus defer judgment on technical quality when making satisfaction ratings, giving high ratings by default, they may be discriminating when it comes to the remaining aspects of medical care. Under this interpretation, the satisfaction ranking reflects actual quality only for the non-technical aspects of care. If this interpretation is correct, it would be somewhat ironical in that recent evidence suggests that, up to a point at least, patients cm judge technical quality. Several studies indicate that satisfaction ratings correlate positively with expert-developed indices of technical quality [13-IS]. The aspects of satisfaction we identified were measured with very uneven frequencies. One can only
*Studies asking about global
or overall satisfaction may indeed reflect degrees of satisfaction with this aspect. However, the methodologic trend is away from global questions and toward the development of multidimensional satisfaction scales.
DORNAN
speculate on the reasons for this. One possibility is that the less frequently studied aspects tend to be structural-for example, cost, access, bureaucracy, and facilities. Investigators may assume these are not as ameliorable as are aspects relating to provider behavior. They may also assume that structural aspects are not as important to patients and do not contribute as heavily to overall satisfaction. Regardless of why some aspects are represented more than others, it is important to recognize that these disparities limit our undersanding of what satisfaction is, as operationally defined. As an extreme example, it is obvious that satisfaction, as measured, is nor about psychosocial problem discussion, since only 3% of studies asked about it.* Considering the prevalence of unresolved psychosocial problems [16] and their impact on health care utilization [ 171,this could represent a serious obstacle to our understanding of ‘true’ satisfaction levels and their implications. Though content validity of instruments is increased by a more exhaustive sampling of aspects [l 11,there remains the question of how to assign weights to them when forming composites or total scores. Alternatives include weighting them equally (as on the Client Satisfaction Questionnaire [ 1S]), weighting by each patient’s personal evaluations of the importance of each aspect (as on the Evaluation Ranking Scale [19]), weighting by a normative patient group’s evaluations of the importance of each aspect (as on the satisfaction with nursing care instrument of Abdellah and Levine [20]), weighting by each aspect’s correlation with a global satisfaction item or scale (used in early trials of the Abdellah and Levine scale [20]), and weighting using factor weights derived from factor analysis of an instrument’s items. These different options have different implications for our definition of overall ‘satisfaction’, and there currently exists no consensus on which is preferable. Some authors choose not to combine items from different domains into a total score [ 11,211. The fact that different instruments include different aspects of satisfaction also raises questions about the comparability of ‘satisfaction’ from one study to the next. Factor analytic studies of instruments and other analyses of internal consistency sometimes show strong common factors, reflecting high intercorrelations among aspects [22,23]. This may relieve our concern that instruments are not comparable, i.e. not tapping a common construct. However, it cannot be safely assumed that results of factor analytic research have guided most investigators in their choice of aspects, and furthermore it is not totally clear that satisfaction items are interchangeable. Some writers argue that various aspects, or dimensions, are distinct [I, 111. In conclusion, it is evident that rigorous scrutiny of the satisfaction literature can lead to new insights about the nature and meaning of satisfaction instruments. The present article reveals gaps in the literature regarding the domain of study as well as features of medical care that are relatively more and less pleasing to patients. These results are relevant to directions for future investigations as well as to health policy, especially the evaluation of system performance.
A meta-analysis of the satisfaction literature Acknowledgemenrs-This research was supported by core funds from the Institute for Health Research. Thanks are extended to Michael Feldstein, Jonathan Brown, Howard Frazier, Bert Green, and David Hemenway for their comments.
REFERENCES
1. Pasco G. C. Patient satisfaction in primary health care: a literature review and analysis. .&xl. Prog. Plan. 6, 185, 1983. 2. Ware J. E. Jr, Davies-Avery A. and Stewart A. L. The measurement and meaning of patient satisfaction. Hlth Med. Cure Serv. Rev. 1, 2, 1978. Ley P. Satisfaction, compliance and communication. Br. J. c/in. Psychol. 21, 241, 1982. Lebow J. L. Consumer assessments of the quality of medical care. Med. Cure 12, 328, 1974. Lochman J. E. Factors related to patients’ satisfaction with their medical care. J. Commun. Hfrh 9, 91, 1983. Locker D. and Dunt D. Theoretical and methodological issues in sociological studies of consumer satisfaction with medical care. Sot. Sci. Med. 12, 283, 1978. 7. Green B. F. and Hall J. A. Quantitative methods for literature reviews. A. Rev. Psychol. 35, 37, 1984. Procedures for Social 8. Rosenthal R. Metu-Analytic Research. Sage, Beverly Hills, Calif., 1984. _ 9. Louis T. A. Finebertz H. V. and Mosteller F. Findinns for public health fro& meta-analysis. A. Rev. Publ. Hih 6, 1, 1985. Methods, 2nd edn. 10. Guilford J. P. Psychometric McGraw-Hill, New York, 1954. Il. Ware J. E. Jr, Snyder M. K., Wright W. R. and Davies A. R. Defining and measuring patient satisfaction with medical care. Euul. Proa. Plan. 6, 247, 1983. 12. Aronson E. Dissonancetheory: progress and problems. In Theories of Cognifive Consistency: A Sourcebook
939
(Edited by Abelson R. P. et al.). Rand McNally, Chicago, Ill. 1968. 13. Roter D. L., Hall J. A. and Katz N. R. Relations of physicians’ behaviors to analogue patients’ satisfaction, recall, and impressions. Med. Cure 2S. 437. 1987. 14. Willson P. and McNamara J. R. How.pe&ptions of a simulated physician-patient interaction influence intended satisfaction and compliance. Sot. Sci. Med. 16, 1699, 1982. 15. Linn B. S. Bum patients’ evaluation of emergency department care. Ann. Emerg. Med. 11, 255, 1982. 16. Cohen-Cole S., Bird J., Baker J., Freeman A., Polmer R. and Shugerman A. Psychiatry for internists: a study of needs. J. Opl Psychiut. 13, 100, 1982.
17. Kessler L., Tessler R. and Nycz G. Co-occurrence of psychiatric and medical morbidity in primary care. J. Fum. Pruct. 16, 319, 1983. _ _ 18. Pascoe G. C. Attkisson C. C. and Roberts R. E. Comparison of indirect and direct approaches to measuring patient satisfaction. Euul. Prog. Plan. 6, 359, 1983.
19. Pascoe G. C. and Attkisson C. C. The Evaluation Ranking Scale: a new methodology for assessing satisfaction. Euul. Prog. Plan. 6, 335, 1983. 20. Abdellah F. G. and Levine E. Developing a measure of patient and personnel satisfaction with nursing care. Nurs. Res. 5, 100, 1957. 21. Zyxanski S. J., Hulka B. S. and Cassel J. C. Scale for the measurement of “satisfaction” with medical care: modifications in content, format and scoring. Med. Cure 12, 61 I, 1974.
22. Ware J. E. Jr and Snyder M. K. Dimensions of patient attitudes regarding doctors and medical care services. Med. Cure 13, 669, 1975.
23. DiMatteo M. R. and Havs R. The sienificance of patients’ perceptions of physician conduci: a study of patient satisfaction in a family practice center. J. Commun. Hith 6, 18, 1980.