- Email: [email protected]
Patient satisfaction and visiting the doctor: A self-regulating system
0277-9536/83 $3.00 + 0.00 Copyright f(’ 1983 Pergamon Press Ltd
Sot. Ser. Med. Vol. 17. No. 18, pp. 1353-1361. 1983 Printed in Great Bntam. All nghts reserved
PATIENT SATISFACTION AND VISITING THE DOCTOR: A SELF-REGULATING SYSTEM* JOHN MIROWSKY Department
of Sociology
and College
and CATHERINE E. Ross
of Medicine, University IL 61801, U.S.A.
of Illinois at Urbana-Champaign,
Urbana,
Abstract-In 1974, Aday and Anderson proposed that client satisfaction may affect utilization of physician services, and that utilization, in turn, may affect satisfaction. Since that time, a number of researchers have investigated the issue, adopting increasingly sophisticated approaches. To date, however, the statistical models used to test the hypothesized feedback have not been completely appropriate. We develop and test alternative hypotheses on the reciprocal effects of satisfaction and utilization. Using methods that allow us to specify models with feedback effects, we examine whether satisfaction with the doctor and visits to the doctor form a self-regulating or a self-amplifying system. In both, satisfaction is expected to result in more visits to the doctor, but in a self-regulating system, visits are hypothesized to lead to lower satisfaction while in the self-amplifying system. visits are hypothesized to produce greater satisfaction. We specify and test the feedback model in two data sets: one based on a cross-sectional survey of pediatric practices in New Haven. Connecticut and the other based on Gray’s panel survey of the Federal Employees Health Benefits Program. In both cases the results indicate that satisfaction and visits form a self-regulating system
A number of researchers interested in explaining individual differences in the use of physician services have proposed that patients are more likely to visit their doctor if they have positive attitudes toward their doctor [l-3]. Looking at things from the other point of view, researchers interested in explaining differences among clients in satisfaction with their doctors’ services have proposed that visiting the doctor affects the patient’s attitudes toward the doctor [4-61. Both are probably correct. Therefore, we suggest that a reciprocal effect exists between the patient’s attitudes toward the doctor and the frequency with which the patient visits the doctor. This suggestion is not new. In 1974 Aday and Andersen proposed, as part of their framework for studying access to medical care, that “the utilization of services is apt to influence a consumer’s satisfaction with the system, and, in turn, the satisfaction or dissatisfaction he experiences from this encounter influences his subsequent use of services” [7]. Although Aday and Andersen did not state whether satisfaction and visits would form a selfregulating or self-amplifying system. their explicit recognition that a reciprocal effect may exist placed the issue on the research agenda. In later work, other researchers also recognized the possibility and adopted increasingly sophisticated approaches to the problem. In 1976 Wolinsky elaborated the ideas of Aday and Andersen, again suggesting that satisfaction and utilization affect one another [8]. However, in his data analysis he only looked at the effect of utilization on satisfaction. making the causal-order assumption *Data coliection for the New Haven Pediatric Practice survey was supported by a grant from the Robert Wood Johnson Foundation. Princeton. NJ; Raymond S. Duff. principal investigator. 1353
that any nonspurious covariance between the two was due to the effect of the former on the latter. He did not explicitly include the possible impact of satisfaction on the use of services in his statistical model and data analyses. Wolinsky found that the frequency of adult visits to the doctor had no effect on satisfaction and that the frequency of child visits had a very small negative effect. However. if there really is a reciprocal effect between visiting the doctor and satisfaction with the doctor’s services then Wolinsky’s estimates are biased, possibly underestimating the impact of visits on satisfaction. In 1979. Roghmann, Hengst and Zastowny also argued that utilization and satisfaction reciprocally affect one another [9]. They attempted to account for the reciprocal effects in their data analysis in the following way: first Roghmann and his colleagues regressed the number of visits on satisfaction and a set of other variables. Then they regressed satisfaction on the number of visits and the same set of other variables. This method of estimation is based on two conflicting causal-order assumptions. In the first analysis it is assumed that any nonspurious covariance between the two variables of interest is due to the effect of the visits on satisfaction and not vice versa. In the second analysis the opposite assumption is made: that any nonspurious covariance is due to the effect of satisfaction on visits. While the formal consequences of making these incompatible assumptions are complex, the substantive consequence is that the estimated coefficients do not accurately reflect the true reciprocal effects [lo]. In 1980 Gray reported the results of a study in which she used a research design that made it possible to estimate the effect of satisfaction on the frequency of visits and the effect of visits on satisfaction without making conflicting assumptions [1 11. First she measured the patient’s satisfaction and other charac-
1354
JOHN MIROWSKY
and
teristics at one point in time. Then she recorded the number of visits to the doctor made by each patient over the course of the following year and remeasured each patient’s satisfaction at the end of that year. She found that the more visits a patient made over the course of the year the more satisfaction decreased. However, she also found the peculiar result that the greater a patient’s satisfaction was at the beginning of the year the fewer visits he or she made during the course of the year (controlling for the patient’s health at the beginning). An examination of Gray’s statistical model shows that it assumes there are no correlations among the residuals of the equations predieting satisfaction at the beginning of the year, the number of visits during the year, and satisfaction at the end of the year. It may not be correct to assume that there are no correlations among the residuals, particularly since satisfaction is measured at two points in time and the measurement error is likely to be correlated over time. It is possible that Gray’s estimates of the reciprocal effects of visits and satisfaction are biased as a result of mis-specifying the relationships among the residuals in the model. In particular, her unexpected finding that satisfaction leads to fewer visits may be incorrect. Research on the possibility of a reciprocal effect between satisfaction with the doctor and the frequency of visits to the doctor has evolved rapidly since Aday and Andersen explicitly recognized the possibility. An early study that mentions the problem was soon followed by a study that alternately made one causal-direction assumption and then the other. This was in turn followed by a survey explicitly designed to address the problem by measuring satisfaction at two points in time. Understanding of the issues and sophistication of the methods used to resolve them have improved rapidly. It is now possible to state explicit hypotheses and to test explicit statistical models of the reciprocal relationship between satisfaction and visits [12].
HYPOTHESES: SELF-REGULATING SELF-AMPLIFYING SYSTEM
VS
In a self-regulating system a tendency in one direction leads to consequences that produce a dampening counter-tendency, whereas in a self-amplifying system a tendency in one direction leads to consequences that reinforce and magnify the tendency. Attitudes and behavior may be linked in a cycle of self-regulating reappraisal in which favorable attitudes lead to actions with consequences that make a person less favorably inclined in the future, or they may be linked in a cycle of habit-forming satisfaction in which the consequences make a person more favorably inclined. We investigate whether satisfaction with the doctor and visits to the doctor are reciprocally related, and if so whether the two factors form a self-regulating or a self-amplifying system. A self-regulating relationship might result if high satisfaction is coupled with high expectations that are likely to be disappointed as the frequency of visits increases. In contrast, a self-amplifying relationship might result if patients bolster their confidence and reaffirm their decision to visit the doctor by sub-
CATHERINE E. Ross
sequently increasing their satisfaction. We examine these issues by testing three hypotheses. H,: Controlling for other variables that influence the frequency of visits (such as the type of practice and the patient’s health), the more satisfied the client is with his or her doctor. the more frequently the client will visit the doctor. This is a straightforward assertion that a person’s attitudes affect their behavior. We expect that a patient’s favorable judgment of his or her physician will tend to result in more frequent contact with the physician. Hz&: Controlling for other variables that influence satisfaction, the more frequently a patient visits the doctor the less satisfied the patient will tend to become. In combination with H, , H,, amounts to an assertion that satisfaction and the frequency of visits form a self-regulating system or. more precisely, a system of self-correcting expectations. A client who is pleased with his or her physician may expect more than is realistic. Most often, patients come to the doctor with minor problems that are easy to care for, especially if the physician has some understanding of the patient and family. Sensitive care for these simple problems will lead to satisfaction on the part of the client. However, the more frequently someone visits the doctor. the more likely it will be that one of the visits will be for a major problem, a chronic recurring problem. or a problem the physician can do nothing about. In the case of a serious or chronic problem, it may take some time before it becomes apparent to the client that the doctor is not solving the problem. At that point of awareness, satisfaction will decrease. Satisfaction will also decrease if the patient comes in with a problem for which there is no simple solution. In addition. on a probability basis alone, the more often someone visits the doctor, the more likely it is that the doctor will make an incorrect diagnosis, be rude or uncaring, or prescribe an ineffective treatment. If a doctor smiles at patients, listens to patients or make correct diagnoses 9 visits out of 10 (all else being equal), then the more often one visits the doctor the more likely one is to encounter the doctor on a visit in which he is gruff. inattentive or inept. So, the more often a client visits the doctor, the more likely he or she is to eventually come in with a real problem for which there is no easy solution, the more likely the physician is to make a mistake and the more likely the client is to lose confidence in the physician’s ability to heal. In sum, the relationship between satisfaction and visits produces a negative feedback loop. HZb: Controlling for other variables that influence satisfaction, the more often a client visits the doctor the more satisfied he or she tends to become with the doctor. In combination with H, , H,, amounts to an assertion that satisfaction and visiting the doctor form a self-amplifying system. Such a system might exist if patients tend to infer attitudes toward the physician from the frequency of their contact with the physician [13]. The simple fact of visiting the doctor may increase satisfaction because, as Aronson says, when a person does something, chooses something, or makes a decision. the person will convince himself or
Patient satisfaction and visiting the doctor herself (and others) that it was a logical, reasonable thing to do [14]. Thus, it is dissonance-reducing and consistency-producing to have a positive attitude toward the doctor if one has visited the doctor. It is possible that, in effect, people tell themselves, “If I hadn’t thought well of this physician, I would not have visited him”. The consequences of a self-amplifying system are quite different from those of a self-regulating one. If visiting the doctor leads to greater satisfaction rather than less, then there will be a tendency for some patients to be pulled into more and more contact with their physicians while others will tend to drift out of contact. The drift may be set in motion by variation in needs as well as in factors such as the type of practice the patient goes to. Once set in motion, going to the doctor becomes more and more of a habit for some patients while not going to the doctor becomes a habit for others. In contrast, a self-regulating system would tend to hold the frequency of visits to a level appropriate to the external factors such as the patient’s health and the type of practice the patient attends.
1355
incorporate all three checks. We will examine the mutually causal effects of client satisfaction and the utilization of physician services in two data sets and we will specify the model differently in the two samples. In addition, one of the models will be based on overtime data and the other on cross-sectional data. If the results are substantively similar, we will have greater confidence in the reliability and generality of the findings. For each data set we will develop a model that will allow us to estimate both the effect of satisfaction on visits and the effect of visits on satisfaction. Each model will use the variables that are available in the data set. Each model will be specified by placing reasonable restrictions on the structural relations among the variables. The restrictions will be based on prior research and theory. The parameters of each model, and the ability of each to reproduce the variances and covariances of the data, will be estimated using the LISREL program [18, 191. More detailed descriptions of the models and the testing procedure are given in the following sections. SAMPLES
MODELING
AND VIRTUAL
REPLICATION
In order to find out whether satisfaction and visiting the doctor form a self-regulating or selfamplifying system we will develop and test two models. The first will be an instantaneous reciprocaleffects model that will be tested on cross-sectional data from a survey of pediatric practices in New Haven, Connecticut. The second will be an overtime model that will be tested on data from the Federal Employees Health Benefits Program Utilization Study [ll]. The two surveys differ in a number of ways, including the reference population, location, nature of the practices (child care vs adult care), time frame, sampling procedures and measurements. However, both studies measure the frequency of visits and the satisfaction of the client with the doctor’s service (in pediatric practice the parent is considered the client, since parents usually make the decisions for their children). This crucial similarity will allow us to carry out a strategy of virtual replication [ 151. If the results are the same in essence, even though the models and the data sets on which they are based differ in many particular ways, then confidence in the generality of the findings will be much greater than would be the case based on either model alone. When reciprocal causal relationships are being modelled, a check on the replicability of the results is especially important. There is always the possibility that the results in a nonrecursive model may be due to the particular specification of the instrumental variables [16, 171. Confidence in the results is undermined if they differ depending on the researcher’s choice among equally-plausible instruments and identifying restrictions. To cope with this problem, a number of researchers have recommended what amounts to a strategy of virtual replication: (1) compare estimates from different samples; (2) use alternative combinations of instrumental variables and evaluate the similarity of results; and (3) compare the results of static (cross-sectional analyses) with dynamic (overtime) models [16,17]. Our analysis will
Nen> Haven pediatric practices The initial sampling units in this study were pediatricians. In the New Haven, Connecticut area, 71 physicians were found to be spending most of their professional time providing primary outpatient pediatric care. Each of these physicians was asked to participate in the study and 61 (86%) agreed. The nonparticipants were significantly more likely to be in solo practice than the participating physicians. Eighteen percent of the physicians in this study were in solo practice, 20% in partnerships and 36% in small groups of three to five physicians. These small practices were all fee.-for-service. Twenty-six percent of the physicians in this study practiced in one of three large prepaid multispecialty groups. The data collection consisted of three parts. One of three researcher-physicians interviewed the physician and observed the physician in practice, and a trained interviewer later interviewed selected families that had been sampled during the observations. The days on which observers visited the physicians were randomly selected. In the pediatrician’s office the observer requested permission of all families to be observed during their visit. All the families who were asked to be observed in the doctor’s office agreed. This produced a total of 1221 observed visits, a mean of 20 visits per physician. In no instance was more than one visit observed for the same child. Of the observed visits, 442 were of children in 376 families that were interviewed later in their homes. Every visit that was observed in which there were between two and four children in the family was asked to be interviewed in their homes. Every eligible family was asked until seven families per pediatrician were selected for an interview. The 376 home interviews represent an 86”/, response rate. since 436 interviews were requested. The only known variable that differentiated participating from nonparticipating families was the age of the child. More specifically, families with older children more often refused to be interviewed. In each home interview the
JOHN MIROWSKY and
1356
mother was asked to be the respondent. The analyses reported here are based on 376 cases. See Ross et al. [20] for more detail on the sample and data collection. Federal Employees Health Benefits Program The data on the Federal Employees Health Benefits Program (FEHBP) is from Gray [l 11. ACcording to Gray, the respondents are members of two plans of the FEHBP: a prepaid group plan called the Group Health Association, Inc., or a Blue Cross-Blue Shield fee-for-service plan. The survey population consisted of ‘all nonannuitant, high-option contract holders and their families’ enrolled in the two plans [ 1 I]. A sample of 1124 contract holders was taken. Of the original sample, 10% did not complete the first questionnaire and 17% did not complete the study for other reasons (e.g. moved, left government service, switched plans, etc.). The response rate is therefore 737:. For more detail on the survey, see Gray [I 11.
MEASURES
Nei+ Haven survey The two endogenous variables are client satisfaction and the number of visits made to a physician in the past year. The number of visits comes from the patient’s medical record. Satisfaction is the mother’s satisfaction with her child’s medical care. We call this measure either client satisfaction or patient satisfaction, since the mother and child are often considered the ‘patient’ in pediatric practice. Answers to the following questions were combined to produce a measure of overall satisfaction: if a close friend just had her first baby and asked you for your advice about pediatricians would you recommend your pediatrician? Have you even considered changing pediatricians? Is your pediatrician available to you when you need him/her? What is your opinion of your pediatrician? Have you ever found his/her advice confusing or harmful? Do you usually follow his/her advice? And, how does your pediatrician compare with the ideal pediatrician? The response to each question was coded on a three-point scale (I = dissatisfied, 2 = ambivalent, 3 = satisfied). The composite measure is an average of the scores on the six items. The exogenous variables include the child’s health, the mother’s age, the type of practice the child attends, the quality of the technical illness care the child received and the quality of the psychosocial ‘caring’ aspects of the doctor-client interaction. These variables were included because there was theoretical reason to expect them to be related to either client satisfaction or utilization, or because previous analysis had shown them to be related [20,21]. The child’s health status during the past year is a global measure of health as evaluated by the reseacher-physicians. The evaluation is based on interviews with the child’s physician, including examination of the medical record; observations of the child in the visit; and interviews with the child’s mother. It is not the child’s own subjective assessment
CATHERINE E. Ross
of his or her health, but an outside evaluation. It is coded poor to good. The type of practice is coded 1 = large prepaid multispecialty group, 2 = small feefor-service group of up to five physicians, 3 = feefor-service partnership, and 4 = fee-for-service solo practice [22]. The quality of the physician’s technical illness care and the quality of the ‘caring’ psychosocial aspects of the doctor-child and doctor-mother interactions were arrived at by observing visits and interviewing the physician. During the observations and interviews, physicians were rated on a number of variables. Each physician was then scored on illness care and psychosocial care for the child in question in terms of a composite of these variables by three researcher-physicians. The variables used to evaluate the quality of the technical illness care include history-taking, physical examination, laboratory utilization, diagnosis, treatment, prescription-writing and follow-up. The psychosocial care measure indicates how well the physician deals with the psychological-developmental aspects of care; how well the physician knows the family; and whether the physician listens to the client, respects his or her wishes and fully explains the illness and the treatment to the client. (For more details on the measures of illness and psychosocial care, see Ross er al. [20].) It is important to reiterate that the two components of the doctor-client interaction-illness care and psychosocial care-are not client perceptions of care. since client perceptions of care are obviously not independent of client satisfaction. The measures are derived from outside observers. Federal employees survey We use eight variables from Gray’s correlation matrix [I I]. The three endogenous variables are satisfaction at the beginning of the 12-month study period (called Satisfaction I), satisfaction at the end of the period (Satisfaction 2) and the number of visits in between. Satisfaction is an index composed of the unweighted sum of the respondent’s judgments of the physician’s performance on each of the seven items: (1) the quality of the medical care; (2) the amount of time the doctor spends; (3) the amount of the doctor’s information; (4) the doctor’s courtesy; (5) the doctor’s explanation of home care; (6) the doctor’s follow-up care; and (7) the doctor’s personal interest. The possible index values range from 7 to 21. The number of visits during the period is measured as a count of all visits for illness, injury or preventive service, excluding doctor-initiated return visits. The number of visits is coded from 0 to 4 or more. We use five exogenous variables from Gray’s data. The first indicates whether (1) or not (0) the individual has a personal physician. The second indicates whether the respondent’s plan is a large prepaid group (0) or fee-for-service (I). The third exogenous variable is the respondent’s self-reported health status at the beginning of the study period (1 = poor, 2 = fair, 3 = good, and 4 = very good). The fourth is the respondent’s race (white = I. other = 0) and the fifth is the respondent’s sex (1 = female, 0 = male). The matrix of correlations among the variables and more details on measurement are given in Gray [I I].
Patient
satisfaction
and visitmg
and testing qf models
For each of the two data sets we began by specifying a structural model that was identified and that would allow us to estimate the reciprocal effects between satisfaction and visits without assuming, a priori, that the residuals in the equations were uncorrelated. In each case we specified a model that required the smallest number of reasonable restrictions in order to be identified. Using the LISREL program [l9]. we estimated the parameters and overall fit of each model, analyzing the variance-covariance matrix in each case. Using the initial models as a baseline, we placed additional restrictions on each following a procedure outlined by Jiireskog and Sorbom [19]. We continued this procedure until none of the remaining hierarchical restrictions indicated by prior research or theory produced a simpler model that reproduced the data as well. In the following paragraphs we will describe the restrictions of the initial models. Nen, Haven survey: initial model There are two equations in the New Haven model: one predicting satisfaction from the number of visits and the exogenous variables, and one predicting the number of visits from satisfaction and the exogenous variables. In order for each equation to be identified it must exclude, by assumption, at least one exogenous variable that has a nonzero effect in the other equation [23]. Prior research has shown that older children visit the pediatrician less frequently [24]. In our initial New Haven model we assumed that the child’s age had no direct impact on the mother’s satisfaction with the pediatrician. The assumption of no direct effect of child‘s age on satisfaction (i.e. fixing the parameter for child’s age to zero in the satisfaction equation) is equivalent to choosing child’s age as an instrumental variable for visits. According to our assumption. if the child’s age had any impact on the mother’s satisfaction with the pediatrician. it was because older children visited the pediatrician less frequently and the frequency of visits in turn influenced the mother’s satisfaction. We also assumed that the pediatrician’s psychosocial care had no direct impact on the frequency of visits-any influence of the doctor’s psychosocial care on the frequency of visits is explained by the fact that good psychosocial care increases the mother’s satisfaction Table
I. Correlations*
among 1
I. 2. 3. 4. 5. 6. i.
Frequency of visits Satisfaction Child’s age Child’s health Type of practice+ Doctor’s illness care Doctor’s psychosoctal
Means SD
care
1357
[20], which in turn influences visits. Thus, psychosocial care is the instrument for satisfaction. The initial model was exactly identified.
METHODS
SpeciJcation
the doctor
Federal emplovees
survey: initial model
There are three equations in the Federal employee model: one predicting satisfaction at the beginning of the study period, one predicting the number of visits during the period, and one predicting satisfaction at the end. We placed the following restrictions on our initial model: (I) satisfaction at the beginning of the study period is not affected by the number of subsequent visits or by satisfaction at the end of the period; (2) the impact of having a personal physician on the number of visits during the period is entirely mediated by the amount of satisfaction it produces at the beginning and satisfaction at the end of the period has no impact on the frequency of prior visits; and (3) any impact of the exogenous variables on satisfaction at the end of the study period is entirely mediated by satisfaction at the beginning and the number of visits during the period. We did not assume (as Gray had) that the residuals of the three equations were uncorrelated. The initial model was identified and had three degrees of freedom (proof that the initial and final models are identified is available from the authors on request). RESULTS
New Haven pediatric practice The final New Haven model has five restrictions in addition to those in the initial model. The Chi-square statistic for the overall fit, which measures the difference between the observed variances and covariances and those predicted by the model, is 0.978 with five degrees of freedom. This is 0.196 Chi-squares per degree of freedom. The probability that the differences between the observed and predicted variances and covariances could occur by chance alone is 0.964. which means that the model reproduces the data very well [25]. The unstandardized coefficients and their standard errors are given in Table 2. Figure I illustrates the structure of the final New Haven model, along with its standardized coefficients. (The absence of a statistically significant correlation between the two residuals is a result, and not an assumption of our analysis: correlations, variances, and covariances for the exogenous variable are not given in either Table 2 or Fig. I, since they were fixed variables 2
in the New Haven 3
4
survey 5
6
7
1.ooo - 0.058 -0.254 -0.139 -0.054 -0.043 0.081
1.000 0.053 0.109 -0.066 0.176 0.347
1.000 - 0.002 - 0.099 0.061 -0.096
1.000 -0.025 0.179 0.215
1.000 -0.126 -0.316
1.000 0.404
1.000
3.697 4.647
2.609 0.559
2.757 1.796
2.607 0.599
1.442 1.073
3.602 0.661
1.984 1.048
*The correlations. means and standard deviations in this table are based on 376 cases. iType of practice is coded in the followinS way: 1 = large group: 2 = small group: 3 = partnership; 4 = solo practice.
1358
JOHN MIROWSKY
Table 2. LISREL
estimates* of the coefficients in the final New Haven
Independent
and CATHERINE E. Ross
modelt
Effect coefficients: Dependent variable Visits
-
Visits Satisfaction Child’s age Child’s health Mother’s age Type of practice// Illness care Psychosocial care Residuals$:
223 -0.847 - 1.285 0.336 -0.337 -0.467 -
(1.047) (0.177) (0.417) (0.241) (0.229) (0.397) -
variance
Equation
Visits
Visits Satisfaction
21.393
variable Satisfaction -0.039 0.199
(0.014) (0.027)
matrix Equation Satisfaction
(2.227)
0.289
(0.27)
*Estimates are based on data from a sample of 376 respondents. Khi-square = 0.978: d.f. = 5: P = 0.964. $This table gives unstandardized coefficients (with their standard errors in parentheses). PCoefficients with values tixed to one or zero are noted by a dash. IlCoded I = large group; 2 = small group; 3 = partnership: 4 = solo practice.
values that are given in, or can be easily estimated from, Table 1.) The final model, in Fig. 1, shows that the child’s age and health, the mother’s age, the type of practice and the quality of the doctor’s illness care only influence satisfaction indirectly through visits. Older children and healthly children visit less, older mothers take their children to the pediatrician more, equal
to the sample
I
CHILD’S HEALTH
children whose mothers take them to small, fee-forservice practices visit less often and children whose doctors provide high-quality illness care visit less often. The model also illustrates our assumption that the quality of a doctor’s psychosocial care only affects the number of visits through its influence on the mother’s satisfaction. The estimated parameters show that the better a pediatrician’s psychosocial care, the more satisfied the mother is with the doctor’s services. The results that bear directly on our hypotheses are described below. H,: Satisfaction leads to more frequent visits. Figure 1 shows that the more satisfied a mother is with her pediatrician, the more frequently she visits the pediatrician. The standardized effect of satisfaction on visits is 0.272. Thus, the model is consistent with H, H?, vs H,,: Visits decrease vs increase satisfaction. Figure 1 shows that the frequency of a mother’s visits to her pediatrician has a negative impact on her satisfaction. The standardized coefficient is -0.320. The results support the hypothesis that visits and satisfaction form a self-regulating system, not a selfamplifying system. Thus Hla is supported while H’b is not. Federol employees’ benejit program The final model for the Federal employee data has three restrictions in addition to those in the initial model. The Chi-square statistic for the overall fit is 7.433, with six degrees of freedom (1.239 Chisquares per degree of freedom). The probability that the differences between the observed and predicted variances and covariances could occur by chance alone is 0.283, which means that the model reproduces the data moderately well. The unstandardized coefficients
I
0 990
-
“v
MOTHER’S AGE 0272
320 TYPE OF PRACTICE I
T I
ElSATISFACTION
0 927
“S
Fig. I. Reciprocal effects of the frequency of visits and satisfaction with the doctor’s care in New Haven P = 0.964. All coefficients are pediatric practices. Chi square = 0.978, with 5 degrees of freedom. standardized, Estimates are based on data from a sample of 376 respondents.
Patient satisfaction and visiting the doctor
tent with H, and replicates the findings for the New Haven pediatric practices. H,, vs H,,: Visits decrease versus increase satisfaction Figure 2 shows that the more frequently a Federal employee visited the doctor during the study period the less satisfied he or she was likely to become at the end of the period (net of satisfaction at the beginning). The standardized coefficient is -0.163. Hla is therefore supported while H,, is not. The final model supports the hypothesis that visits and satisfaction form a self-regulating system and not a selfamplifying one. This replicates the results of the analysis for New Haven pediatric practices.
and their standard errors are given in Table 3. Figure 2 illustrates the structure of the final Federal employees’ model, along with its standardized coefficients. (For visual clarity correlations among the exogenous variables are not shown in Fig. 2. The correlations can be calculated easily from the variances and covariances in Table 3.) The final model, in Fig. 2, shows that having a personal physician and feeling healthy only influence a patient’s number of visits indirectly through their impact on the patient’s satisfaction at the beginning of the study period. People who have a personal physician and believe they are healthy are more satisfied with their physicians. The type of practice directly influences both satisfaction and visits-feefor-service patients are more satisfied but visit less. This is in partial contrast to the results for the New Haven pediatric practices, where the type of practice has a direct influence on the frequency of visits but not on satisfaction. Figure 2 also shows that, among the Federal employees studied, race and sex only influence satisfaction through their impact on the frequency of visits. Whites and females visit more frequently than blacks and males. The results that bear directly on our hypotheses, are described below. H,: Satisfaction leads to more frequent visits. Figure 2 shows, that a Federal employee’s satisfaction with his or her health care increases the employee’s expected frequency of visits. The standardized coefficient representing the effect of satisfaction at the beginning of the study period on visits during the period is 0.179. Thus, the model is consis-
Table
3. LISREL
1359
DISCUSSION
To date the statistical models used to test the hypothesized reciprocal effect between client satisfaction and utilization of physician services have not been completely appropriate. Early studies that assumed one causal order or the other [8,9] were followed by an overtime data analysis, that, while more sophisticated than the earlier research, made the assumption of no correlations among the residuals and found the unexpected result that client satisfaction leads to fewer visits [l 11. In our models we do not assume that there are no correlations among residuals. We find that satisfaction with the doctor increases the frequency of visiting the doctor which, in turn, decreases satisfaction. The selfregulating reciprocal effects between satisfaction and
of the coefficients in the final Federal employees
estimates*
modelt Effect coefficients:
Independent
variable
Satisfaction +
Visits Satisfaction 1 Personal physician Subjective health Fee-for-service White Female
1.718 0.273 0.447
Exogenous 1. 2. 3. 4. 5.
Personal physician Subjective health Fee-for-service White Female
variables: 1
Satisfaction Visits Satisfaction
(0.262) (0.156) (0.232)
1 2
-
_
0.068 -0.677 0.595 0.609
(0.060) (0.111) (0.097) (0.095)
variance
variance-covariance Satisfaction
Equation
Dependent variable Visits Satisfaction 2
-
0.230 -0.019 0.079 0.060 0.007 Residuals:
1
12.377 - 1.342 - 7.273
1
(0.611) (0.764) (1.828)
matrix
(0.261) (0.141) -
I/ 4
0.250 0.080 -0.012
0.250 -0.038
matrix Equation Visits 1.806 1.402
-0.411 0.903 -
(0.185) (0.686)
5
0.230
Satisfaction
14.278
2
(2.203)
*Estimates are based on data from a sample of 821 respondents. tChi-square = 7.433: d.f. = 6: P = 0.283. :This table gives unstandardized coefficients (with their standard errors in parentheses). ljcoefficients with values fixed to one or zero are noted by a dash. /‘No standard errors are given because the model assumes that the variances and covariances of the exogenous variables are fixed to the sample values..
1360
JOHN Y~IRUWSKY and CATHERINE E. Ross
SUBJECTIVE HEALTH
FEE-FORSERVICE
WHITE I
r-l FEMALE
L
_I
Fig. 2. Reciprocal &ects of the frequency of visits and satisfaction with the doctor’s care in the Federal Employees’ Benefit Program. Chi square = 7.433 with 6 degrees of freedom. P = 0.283. All coefficients are standardized. Estimates are based on data from a sample of 821 respondents.
visits to the doctor are found amohg both the clients of New Haven pediatricians and the members of the Federal Employees Health Benefits Program. The similarity of the results. given the differences between the study populations, medical specialties, research designs, and measurement procedures, underscores the robustness of our finding. The self-regulating relationship between satisfaction and visits does not appear to be a quirk of medical practice in a particular place in the United States or of a particular type of medical practice that was studied in a particular way. Our results support the hypothesis that satisfaction with the doctor and the frequency of visiting the doctor form a self-regulating system. The more that people are satisfied with their doctors the more frequently they will visit the doctor, but as the frequency of contact increases, so does the probability that the patients’ high expectations will be unmet. As the frequency of visits increases so does the likelihood of an encounter in which the physician is gruff, inattentive, or inept. Eventually, the client may come in with a problem for which the physician has no easy solution. In the case of a chronic or recurring problem the patient may realize that the doctor cannot solve the problem quickly, easily and finally. As awareness of the doctor’s limitations increases, the patient’s satisfaction decreases. Since the less-satisfied patient tends to visit the doctor less frequently. the
result is a self-regulating reciprocal relationship between satisfaction and visits. We do not find support in our results for the idea that patients become satisfied with their doctors as a result of visits. We find no evidence that patients adapt psychologically to frequent visits by increasing their satisfaction. If such an adjustment is occurring it is outweighed by the disappointment that results from unmet expectations. Even though unintended, the self-regulating feedback between satisfaction and the use of services has a number of desirable consequences for the patient, the physician and society. Because more visits lead to lower satisfaction and lower satisfaction leads to fewer visits, the patient avoids excessive dependency on the physician. By the same token, the physician avoids excessive demands. Society gains because physician services are distributed more evenly across patients and because persons, and their resources, are less likely to be entrapped by accelerating dependency. REFERENCES
I. Andersen Health
R. A Behaaiorui Model cjf Fumilies Use of Services. University of Chicago Center for
Health Administration Studies, Chicago, 1968. 2. Rosenstock 1. M. Why people use health services. Milhonk
meml.
Fund Q. 44, 94. 1966.
Patient
satisfaction
and visiting
E. A. Sociomedical variations among ethnic 3. Suchman groups. Am .I. Social. IO, 319, 1964. 4. Deisher R. W., Engle W., Spieholz R. and Standfast S. J. Mothers’ opinions of their pediatric care. Pediatrics 35, 82. 1965. 5. Larsen D. E. and Rootman I. Physician role performance and patient satisfaction. Sot. Sci. Med. 10,29. 1976. of 6. Linn L. S. Factors associated with patient evaluation health care. Milbank meml. Fund. Q. 53, 531, 1975. I. Aday L. A. and Andersen R. A framework for the study of access to medical care. Hlth Serv. Res. 9, 208. 1974. 8. Wolinsky F. D. Health service utilization and attitudes toward health maintenance organizations: a theoretical and methodological discussion. J. Hllh sot. Eehac. 17, 221, 1976. 9. Roghmann K. J., Hengst A. and Zastowny T. R. Satisfaction with medical care: its measurement and relation to utilization. Med. Care 17, 461. 1979. IO. Hanushek E. A. and Jackson J. E. Statisrical Melhods for Social Scientisrs. Academic Press, New York, 1977. II. Gray L. C. Consumer satisfaction with physician provided services: a panel study. Sot. Sci. Med. 14, 65, 1980. 12. Erlanger H. and Winsboro H. The subculture of violence thesis: an example of a simultaneous equation model in sociology. Sot. Meth. Res. 5, 231. 1976, state that although a number of theories in sociology explicitly or implicitly propose reciprocal causal relationships, few researchers model these relationships statistically. This is especially true in medical sociology. Therefore, in this paper, we limit our focus to the reciprocal relationship of client satisfaction and utilization. We take a selective rather than a general approach since the separate issues of client satisfaction and of utilization have been addressed more broadly elsewhere. Rather than restating the extensive theory and research on satisfaction and, especially, on utilization here, we refer readers to reviews and syntheses: Aday L. A. and Eichhorn R. The Urili:arion of Health Services: Indices and Correlates. National Center for Health Services Research and Development, Rockville, MD, 1972: Andersen R.. Kravits J. and Andersen 0. W. Equity in Health Sciences: Empirical Anal.vsis in Social Policy. Balhnger. Cambridge, MA, 1975; Ben-Sira Z. The function of the professionals’ affective behavior in client satisfaction: a revised approach to social interaction theory. J. Hlth sot. Behav. 17, 3, 1976; Dutton D. B. Explaining the low use of health services by the poor: costs, attitudes or delivery systems? Am. Social. Rev. 43, 348. 1978: Freidson E. Patients’ Views of Medical Practice. Russell Sage, New York. 1961: Mechanic D. The Growth qf Bureaucratic Medicine. Wiley. New York. 1976; McKinlay J. B. Some approaches and problems in the study of the use of services. J. Hlfh sot.
13.
14. 15.
16.
17. 18.
19.
20.
21.
22.
23. 24. 25.
the doctor
1361
Eehav. 13. 115. 1972: Shortell S. M.. Richardson W. C., LoGerto J. P., Diehr P., Weaver B. and Green K. E. The relationships among dimensions of health services in two provider systems: a causal model approach. J. Hlth sot. Behav. 18, 139. 1977. Bern D. J. Self-perception theory. In Advances in Experimenral Social Psycho/og_v (Edited by Berkowitz L.), Vol. 6. Academic Press, New York, 1972. Aronson E. The Social Animal. Freeman, San Francisco, 1972. Finifter B. M. The generation of confidence: evaluating research findings by random subsample replication. In Socioloaical Methodoloav (Edited bv Costner H. L.). PP. I 12-l 75. Josey-Bass, San Francisco, 1972. . Alexander K. L., Reilly T. W. and Fennessey J. Issues in instrumental variables analysis. Am. Socioi. Rev. 46, 937, 1981. Cramer J. C. Fertility and female employment: problems of causal direction. Am. social. Rev. 45, 167. 1980. Jiireskog K. G. and Sorbom D. Advances in Factor Analysis and Srructural Equation Models. Abt Books, Cambridge, MA, 1979. Joreskog K. G. and Sorbom D. LISREL IV: Analysis of Linear Swucrural Relationships by [he Method of Maximum Likelihood (User’s Guide). International Education Services, Chicago, 1980. Ross C. E., Wheaton B. and Dull’ R. S. Client satisfaction and the organization of medical care: why time counts. J. Hlrh sot. Behat?. 22, 243. 1981. Ross C. E. and Duff R. S. Returning to the doctor: the effect of client characteristics. type of practice and experiences with care. J. Hlth sot. Eehav. 23, 119, 1982. Because the large multispecialty groups are prepaid, and the small practices are fee-for-service, we unfortunately cannot disentangle organizational versus payment effects of type of practice on utilization or satisfaction. However, since this is not a major focus of the present analysis, we hope future research will examine these effects separately. Asher H. B. Causal Modeling. Sage Publications, Beverly Hills, CA. 1976. Tessler R. Birth order, family size and children’s use of physician services. Hlrh Sem. Res. 15, 55. 1980. According to Joreskog and Sorbom, in the Chi-square test the null hypothesis is that there is no difference between the observed variance-covariance matrix and the one reproduced by the model. The alternative hypothesis is that there is a difference between them that cannot be attributed to chance alone. Since we would like any differences to be due to chance. and not to errors in the model, we would like the probability value associated with the Chi-square test to be as large as possible. The probability levels associated with the Chi-square statistic in textbook examples typically range from 0.05, to 0.40.
Sot. Ser. Med. Vol. 17. No. 18, pp. 1353-1361. 1983 Printed in Great Bntam. All nghts reserved
PATIENT SATISFACTION AND VISITING THE DOCTOR: A SELF-REGULATING SYSTEM* JOHN MIROWSKY Department
of Sociology
and College
and CATHERINE E. Ross
of Medicine, University IL 61801, U.S.A.
of Illinois at Urbana-Champaign,
Urbana,
Abstract-In 1974, Aday and Anderson proposed that client satisfaction may affect utilization of physician services, and that utilization, in turn, may affect satisfaction. Since that time, a number of researchers have investigated the issue, adopting increasingly sophisticated approaches. To date, however, the statistical models used to test the hypothesized feedback have not been completely appropriate. We develop and test alternative hypotheses on the reciprocal effects of satisfaction and utilization. Using methods that allow us to specify models with feedback effects, we examine whether satisfaction with the doctor and visits to the doctor form a self-regulating or a self-amplifying system. In both, satisfaction is expected to result in more visits to the doctor, but in a self-regulating system, visits are hypothesized to lead to lower satisfaction while in the self-amplifying system. visits are hypothesized to produce greater satisfaction. We specify and test the feedback model in two data sets: one based on a cross-sectional survey of pediatric practices in New Haven. Connecticut and the other based on Gray’s panel survey of the Federal Employees Health Benefits Program. In both cases the results indicate that satisfaction and visits form a self-regulating system
A number of researchers interested in explaining individual differences in the use of physician services have proposed that patients are more likely to visit their doctor if they have positive attitudes toward their doctor [l-3]. Looking at things from the other point of view, researchers interested in explaining differences among clients in satisfaction with their doctors’ services have proposed that visiting the doctor affects the patient’s attitudes toward the doctor [4-61. Both are probably correct. Therefore, we suggest that a reciprocal effect exists between the patient’s attitudes toward the doctor and the frequency with which the patient visits the doctor. This suggestion is not new. In 1974 Aday and Andersen proposed, as part of their framework for studying access to medical care, that “the utilization of services is apt to influence a consumer’s satisfaction with the system, and, in turn, the satisfaction or dissatisfaction he experiences from this encounter influences his subsequent use of services” [7]. Although Aday and Andersen did not state whether satisfaction and visits would form a selfregulating or self-amplifying system. their explicit recognition that a reciprocal effect may exist placed the issue on the research agenda. In later work, other researchers also recognized the possibility and adopted increasingly sophisticated approaches to the problem. In 1976 Wolinsky elaborated the ideas of Aday and Andersen, again suggesting that satisfaction and utilization affect one another [8]. However, in his data analysis he only looked at the effect of utilization on satisfaction. making the causal-order assumption *Data coliection for the New Haven Pediatric Practice survey was supported by a grant from the Robert Wood Johnson Foundation. Princeton. NJ; Raymond S. Duff. principal investigator. 1353
that any nonspurious covariance between the two was due to the effect of the former on the latter. He did not explicitly include the possible impact of satisfaction on the use of services in his statistical model and data analyses. Wolinsky found that the frequency of adult visits to the doctor had no effect on satisfaction and that the frequency of child visits had a very small negative effect. However. if there really is a reciprocal effect between visiting the doctor and satisfaction with the doctor’s services then Wolinsky’s estimates are biased, possibly underestimating the impact of visits on satisfaction. In 1979. Roghmann, Hengst and Zastowny also argued that utilization and satisfaction reciprocally affect one another [9]. They attempted to account for the reciprocal effects in their data analysis in the following way: first Roghmann and his colleagues regressed the number of visits on satisfaction and a set of other variables. Then they regressed satisfaction on the number of visits and the same set of other variables. This method of estimation is based on two conflicting causal-order assumptions. In the first analysis it is assumed that any nonspurious covariance between the two variables of interest is due to the effect of the visits on satisfaction and not vice versa. In the second analysis the opposite assumption is made: that any nonspurious covariance is due to the effect of satisfaction on visits. While the formal consequences of making these incompatible assumptions are complex, the substantive consequence is that the estimated coefficients do not accurately reflect the true reciprocal effects [lo]. In 1980 Gray reported the results of a study in which she used a research design that made it possible to estimate the effect of satisfaction on the frequency of visits and the effect of visits on satisfaction without making conflicting assumptions [1 11. First she measured the patient’s satisfaction and other charac-
1354
JOHN MIROWSKY
and
teristics at one point in time. Then she recorded the number of visits to the doctor made by each patient over the course of the following year and remeasured each patient’s satisfaction at the end of that year. She found that the more visits a patient made over the course of the year the more satisfaction decreased. However, she also found the peculiar result that the greater a patient’s satisfaction was at the beginning of the year the fewer visits he or she made during the course of the year (controlling for the patient’s health at the beginning). An examination of Gray’s statistical model shows that it assumes there are no correlations among the residuals of the equations predieting satisfaction at the beginning of the year, the number of visits during the year, and satisfaction at the end of the year. It may not be correct to assume that there are no correlations among the residuals, particularly since satisfaction is measured at two points in time and the measurement error is likely to be correlated over time. It is possible that Gray’s estimates of the reciprocal effects of visits and satisfaction are biased as a result of mis-specifying the relationships among the residuals in the model. In particular, her unexpected finding that satisfaction leads to fewer visits may be incorrect. Research on the possibility of a reciprocal effect between satisfaction with the doctor and the frequency of visits to the doctor has evolved rapidly since Aday and Andersen explicitly recognized the possibility. An early study that mentions the problem was soon followed by a study that alternately made one causal-direction assumption and then the other. This was in turn followed by a survey explicitly designed to address the problem by measuring satisfaction at two points in time. Understanding of the issues and sophistication of the methods used to resolve them have improved rapidly. It is now possible to state explicit hypotheses and to test explicit statistical models of the reciprocal relationship between satisfaction and visits [12].
HYPOTHESES: SELF-REGULATING SELF-AMPLIFYING SYSTEM
VS
In a self-regulating system a tendency in one direction leads to consequences that produce a dampening counter-tendency, whereas in a self-amplifying system a tendency in one direction leads to consequences that reinforce and magnify the tendency. Attitudes and behavior may be linked in a cycle of self-regulating reappraisal in which favorable attitudes lead to actions with consequences that make a person less favorably inclined in the future, or they may be linked in a cycle of habit-forming satisfaction in which the consequences make a person more favorably inclined. We investigate whether satisfaction with the doctor and visits to the doctor are reciprocally related, and if so whether the two factors form a self-regulating or a self-amplifying system. A self-regulating relationship might result if high satisfaction is coupled with high expectations that are likely to be disappointed as the frequency of visits increases. In contrast, a self-amplifying relationship might result if patients bolster their confidence and reaffirm their decision to visit the doctor by sub-
CATHERINE E. Ross
sequently increasing their satisfaction. We examine these issues by testing three hypotheses. H,: Controlling for other variables that influence the frequency of visits (such as the type of practice and the patient’s health), the more satisfied the client is with his or her doctor. the more frequently the client will visit the doctor. This is a straightforward assertion that a person’s attitudes affect their behavior. We expect that a patient’s favorable judgment of his or her physician will tend to result in more frequent contact with the physician. Hz&: Controlling for other variables that influence satisfaction, the more frequently a patient visits the doctor the less satisfied the patient will tend to become. In combination with H, , H,, amounts to an assertion that satisfaction and the frequency of visits form a self-regulating system or. more precisely, a system of self-correcting expectations. A client who is pleased with his or her physician may expect more than is realistic. Most often, patients come to the doctor with minor problems that are easy to care for, especially if the physician has some understanding of the patient and family. Sensitive care for these simple problems will lead to satisfaction on the part of the client. However, the more frequently someone visits the doctor. the more likely it will be that one of the visits will be for a major problem, a chronic recurring problem. or a problem the physician can do nothing about. In the case of a serious or chronic problem, it may take some time before it becomes apparent to the client that the doctor is not solving the problem. At that point of awareness, satisfaction will decrease. Satisfaction will also decrease if the patient comes in with a problem for which there is no simple solution. In addition. on a probability basis alone, the more often someone visits the doctor, the more likely it is that the doctor will make an incorrect diagnosis, be rude or uncaring, or prescribe an ineffective treatment. If a doctor smiles at patients, listens to patients or make correct diagnoses 9 visits out of 10 (all else being equal), then the more often one visits the doctor the more likely one is to encounter the doctor on a visit in which he is gruff. inattentive or inept. So, the more often a client visits the doctor, the more likely he or she is to eventually come in with a real problem for which there is no easy solution, the more likely the physician is to make a mistake and the more likely the client is to lose confidence in the physician’s ability to heal. In sum, the relationship between satisfaction and visits produces a negative feedback loop. HZb: Controlling for other variables that influence satisfaction, the more often a client visits the doctor the more satisfied he or she tends to become with the doctor. In combination with H, , H,, amounts to an assertion that satisfaction and visiting the doctor form a self-amplifying system. Such a system might exist if patients tend to infer attitudes toward the physician from the frequency of their contact with the physician [13]. The simple fact of visiting the doctor may increase satisfaction because, as Aronson says, when a person does something, chooses something, or makes a decision. the person will convince himself or
Patient satisfaction and visiting the doctor herself (and others) that it was a logical, reasonable thing to do [14]. Thus, it is dissonance-reducing and consistency-producing to have a positive attitude toward the doctor if one has visited the doctor. It is possible that, in effect, people tell themselves, “If I hadn’t thought well of this physician, I would not have visited him”. The consequences of a self-amplifying system are quite different from those of a self-regulating one. If visiting the doctor leads to greater satisfaction rather than less, then there will be a tendency for some patients to be pulled into more and more contact with their physicians while others will tend to drift out of contact. The drift may be set in motion by variation in needs as well as in factors such as the type of practice the patient goes to. Once set in motion, going to the doctor becomes more and more of a habit for some patients while not going to the doctor becomes a habit for others. In contrast, a self-regulating system would tend to hold the frequency of visits to a level appropriate to the external factors such as the patient’s health and the type of practice the patient attends.
1355
incorporate all three checks. We will examine the mutually causal effects of client satisfaction and the utilization of physician services in two data sets and we will specify the model differently in the two samples. In addition, one of the models will be based on overtime data and the other on cross-sectional data. If the results are substantively similar, we will have greater confidence in the reliability and generality of the findings. For each data set we will develop a model that will allow us to estimate both the effect of satisfaction on visits and the effect of visits on satisfaction. Each model will use the variables that are available in the data set. Each model will be specified by placing reasonable restrictions on the structural relations among the variables. The restrictions will be based on prior research and theory. The parameters of each model, and the ability of each to reproduce the variances and covariances of the data, will be estimated using the LISREL program [18, 191. More detailed descriptions of the models and the testing procedure are given in the following sections. SAMPLES
MODELING
AND VIRTUAL
REPLICATION
In order to find out whether satisfaction and visiting the doctor form a self-regulating or selfamplifying system we will develop and test two models. The first will be an instantaneous reciprocaleffects model that will be tested on cross-sectional data from a survey of pediatric practices in New Haven, Connecticut. The second will be an overtime model that will be tested on data from the Federal Employees Health Benefits Program Utilization Study [ll]. The two surveys differ in a number of ways, including the reference population, location, nature of the practices (child care vs adult care), time frame, sampling procedures and measurements. However, both studies measure the frequency of visits and the satisfaction of the client with the doctor’s service (in pediatric practice the parent is considered the client, since parents usually make the decisions for their children). This crucial similarity will allow us to carry out a strategy of virtual replication [ 151. If the results are the same in essence, even though the models and the data sets on which they are based differ in many particular ways, then confidence in the generality of the findings will be much greater than would be the case based on either model alone. When reciprocal causal relationships are being modelled, a check on the replicability of the results is especially important. There is always the possibility that the results in a nonrecursive model may be due to the particular specification of the instrumental variables [16, 171. Confidence in the results is undermined if they differ depending on the researcher’s choice among equally-plausible instruments and identifying restrictions. To cope with this problem, a number of researchers have recommended what amounts to a strategy of virtual replication: (1) compare estimates from different samples; (2) use alternative combinations of instrumental variables and evaluate the similarity of results; and (3) compare the results of static (cross-sectional analyses) with dynamic (overtime) models [16,17]. Our analysis will
Nen> Haven pediatric practices The initial sampling units in this study were pediatricians. In the New Haven, Connecticut area, 71 physicians were found to be spending most of their professional time providing primary outpatient pediatric care. Each of these physicians was asked to participate in the study and 61 (86%) agreed. The nonparticipants were significantly more likely to be in solo practice than the participating physicians. Eighteen percent of the physicians in this study were in solo practice, 20% in partnerships and 36% in small groups of three to five physicians. These small practices were all fee.-for-service. Twenty-six percent of the physicians in this study practiced in one of three large prepaid multispecialty groups. The data collection consisted of three parts. One of three researcher-physicians interviewed the physician and observed the physician in practice, and a trained interviewer later interviewed selected families that had been sampled during the observations. The days on which observers visited the physicians were randomly selected. In the pediatrician’s office the observer requested permission of all families to be observed during their visit. All the families who were asked to be observed in the doctor’s office agreed. This produced a total of 1221 observed visits, a mean of 20 visits per physician. In no instance was more than one visit observed for the same child. Of the observed visits, 442 were of children in 376 families that were interviewed later in their homes. Every visit that was observed in which there were between two and four children in the family was asked to be interviewed in their homes. Every eligible family was asked until seven families per pediatrician were selected for an interview. The 376 home interviews represent an 86”/, response rate. since 436 interviews were requested. The only known variable that differentiated participating from nonparticipating families was the age of the child. More specifically, families with older children more often refused to be interviewed. In each home interview the
JOHN MIROWSKY and
1356
mother was asked to be the respondent. The analyses reported here are based on 376 cases. See Ross et al. [20] for more detail on the sample and data collection. Federal Employees Health Benefits Program The data on the Federal Employees Health Benefits Program (FEHBP) is from Gray [l 11. ACcording to Gray, the respondents are members of two plans of the FEHBP: a prepaid group plan called the Group Health Association, Inc., or a Blue Cross-Blue Shield fee-for-service plan. The survey population consisted of ‘all nonannuitant, high-option contract holders and their families’ enrolled in the two plans [ 1 I]. A sample of 1124 contract holders was taken. Of the original sample, 10% did not complete the first questionnaire and 17% did not complete the study for other reasons (e.g. moved, left government service, switched plans, etc.). The response rate is therefore 737:. For more detail on the survey, see Gray [I 11.
MEASURES
Nei+ Haven survey The two endogenous variables are client satisfaction and the number of visits made to a physician in the past year. The number of visits comes from the patient’s medical record. Satisfaction is the mother’s satisfaction with her child’s medical care. We call this measure either client satisfaction or patient satisfaction, since the mother and child are often considered the ‘patient’ in pediatric practice. Answers to the following questions were combined to produce a measure of overall satisfaction: if a close friend just had her first baby and asked you for your advice about pediatricians would you recommend your pediatrician? Have you even considered changing pediatricians? Is your pediatrician available to you when you need him/her? What is your opinion of your pediatrician? Have you ever found his/her advice confusing or harmful? Do you usually follow his/her advice? And, how does your pediatrician compare with the ideal pediatrician? The response to each question was coded on a three-point scale (I = dissatisfied, 2 = ambivalent, 3 = satisfied). The composite measure is an average of the scores on the six items. The exogenous variables include the child’s health, the mother’s age, the type of practice the child attends, the quality of the technical illness care the child received and the quality of the psychosocial ‘caring’ aspects of the doctor-client interaction. These variables were included because there was theoretical reason to expect them to be related to either client satisfaction or utilization, or because previous analysis had shown them to be related [20,21]. The child’s health status during the past year is a global measure of health as evaluated by the reseacher-physicians. The evaluation is based on interviews with the child’s physician, including examination of the medical record; observations of the child in the visit; and interviews with the child’s mother. It is not the child’s own subjective assessment
CATHERINE E. Ross
of his or her health, but an outside evaluation. It is coded poor to good. The type of practice is coded 1 = large prepaid multispecialty group, 2 = small feefor-service group of up to five physicians, 3 = feefor-service partnership, and 4 = fee-for-service solo practice [22]. The quality of the physician’s technical illness care and the quality of the ‘caring’ psychosocial aspects of the doctor-child and doctor-mother interactions were arrived at by observing visits and interviewing the physician. During the observations and interviews, physicians were rated on a number of variables. Each physician was then scored on illness care and psychosocial care for the child in question in terms of a composite of these variables by three researcher-physicians. The variables used to evaluate the quality of the technical illness care include history-taking, physical examination, laboratory utilization, diagnosis, treatment, prescription-writing and follow-up. The psychosocial care measure indicates how well the physician deals with the psychological-developmental aspects of care; how well the physician knows the family; and whether the physician listens to the client, respects his or her wishes and fully explains the illness and the treatment to the client. (For more details on the measures of illness and psychosocial care, see Ross er al. [20].) It is important to reiterate that the two components of the doctor-client interaction-illness care and psychosocial care-are not client perceptions of care. since client perceptions of care are obviously not independent of client satisfaction. The measures are derived from outside observers. Federal employees survey We use eight variables from Gray’s correlation matrix [I I]. The three endogenous variables are satisfaction at the beginning of the 12-month study period (called Satisfaction I), satisfaction at the end of the period (Satisfaction 2) and the number of visits in between. Satisfaction is an index composed of the unweighted sum of the respondent’s judgments of the physician’s performance on each of the seven items: (1) the quality of the medical care; (2) the amount of time the doctor spends; (3) the amount of the doctor’s information; (4) the doctor’s courtesy; (5) the doctor’s explanation of home care; (6) the doctor’s follow-up care; and (7) the doctor’s personal interest. The possible index values range from 7 to 21. The number of visits during the period is measured as a count of all visits for illness, injury or preventive service, excluding doctor-initiated return visits. The number of visits is coded from 0 to 4 or more. We use five exogenous variables from Gray’s data. The first indicates whether (1) or not (0) the individual has a personal physician. The second indicates whether the respondent’s plan is a large prepaid group (0) or fee-for-service (I). The third exogenous variable is the respondent’s self-reported health status at the beginning of the study period (1 = poor, 2 = fair, 3 = good, and 4 = very good). The fourth is the respondent’s race (white = I. other = 0) and the fifth is the respondent’s sex (1 = female, 0 = male). The matrix of correlations among the variables and more details on measurement are given in Gray [I I].
Patient
satisfaction
and visitmg
and testing qf models
For each of the two data sets we began by specifying a structural model that was identified and that would allow us to estimate the reciprocal effects between satisfaction and visits without assuming, a priori, that the residuals in the equations were uncorrelated. In each case we specified a model that required the smallest number of reasonable restrictions in order to be identified. Using the LISREL program [l9]. we estimated the parameters and overall fit of each model, analyzing the variance-covariance matrix in each case. Using the initial models as a baseline, we placed additional restrictions on each following a procedure outlined by Jiireskog and Sorbom [19]. We continued this procedure until none of the remaining hierarchical restrictions indicated by prior research or theory produced a simpler model that reproduced the data as well. In the following paragraphs we will describe the restrictions of the initial models. Nen, Haven survey: initial model There are two equations in the New Haven model: one predicting satisfaction from the number of visits and the exogenous variables, and one predicting the number of visits from satisfaction and the exogenous variables. In order for each equation to be identified it must exclude, by assumption, at least one exogenous variable that has a nonzero effect in the other equation [23]. Prior research has shown that older children visit the pediatrician less frequently [24]. In our initial New Haven model we assumed that the child’s age had no direct impact on the mother’s satisfaction with the pediatrician. The assumption of no direct effect of child‘s age on satisfaction (i.e. fixing the parameter for child’s age to zero in the satisfaction equation) is equivalent to choosing child’s age as an instrumental variable for visits. According to our assumption. if the child’s age had any impact on the mother’s satisfaction with the pediatrician. it was because older children visited the pediatrician less frequently and the frequency of visits in turn influenced the mother’s satisfaction. We also assumed that the pediatrician’s psychosocial care had no direct impact on the frequency of visits-any influence of the doctor’s psychosocial care on the frequency of visits is explained by the fact that good psychosocial care increases the mother’s satisfaction Table
I. Correlations*
among 1
I. 2. 3. 4. 5. 6. i.
Frequency of visits Satisfaction Child’s age Child’s health Type of practice+ Doctor’s illness care Doctor’s psychosoctal
Means SD
care
1357
[20], which in turn influences visits. Thus, psychosocial care is the instrument for satisfaction. The initial model was exactly identified.
METHODS
SpeciJcation
the doctor
Federal emplovees
survey: initial model
There are three equations in the Federal employee model: one predicting satisfaction at the beginning of the study period, one predicting the number of visits during the period, and one predicting satisfaction at the end. We placed the following restrictions on our initial model: (I) satisfaction at the beginning of the study period is not affected by the number of subsequent visits or by satisfaction at the end of the period; (2) the impact of having a personal physician on the number of visits during the period is entirely mediated by the amount of satisfaction it produces at the beginning and satisfaction at the end of the period has no impact on the frequency of prior visits; and (3) any impact of the exogenous variables on satisfaction at the end of the study period is entirely mediated by satisfaction at the beginning and the number of visits during the period. We did not assume (as Gray had) that the residuals of the three equations were uncorrelated. The initial model was identified and had three degrees of freedom (proof that the initial and final models are identified is available from the authors on request). RESULTS
New Haven pediatric practice The final New Haven model has five restrictions in addition to those in the initial model. The Chi-square statistic for the overall fit, which measures the difference between the observed variances and covariances and those predicted by the model, is 0.978 with five degrees of freedom. This is 0.196 Chi-squares per degree of freedom. The probability that the differences between the observed and predicted variances and covariances could occur by chance alone is 0.964. which means that the model reproduces the data very well [25]. The unstandardized coefficients and their standard errors are given in Table 2. Figure I illustrates the structure of the final New Haven model, along with its standardized coefficients. (The absence of a statistically significant correlation between the two residuals is a result, and not an assumption of our analysis: correlations, variances, and covariances for the exogenous variable are not given in either Table 2 or Fig. I, since they were fixed variables 2
in the New Haven 3
4
survey 5
6
7
1.ooo - 0.058 -0.254 -0.139 -0.054 -0.043 0.081
1.000 0.053 0.109 -0.066 0.176 0.347
1.000 - 0.002 - 0.099 0.061 -0.096
1.000 -0.025 0.179 0.215
1.000 -0.126 -0.316
1.000 0.404
1.000
3.697 4.647
2.609 0.559
2.757 1.796
2.607 0.599
1.442 1.073
3.602 0.661
1.984 1.048
*The correlations. means and standard deviations in this table are based on 376 cases. iType of practice is coded in the followinS way: 1 = large group: 2 = small group: 3 = partnership; 4 = solo practice.
1358
JOHN MIROWSKY
Table 2. LISREL
estimates* of the coefficients in the final New Haven
Independent
and CATHERINE E. Ross
modelt
Effect coefficients: Dependent variable Visits
-
Visits Satisfaction Child’s age Child’s health Mother’s age Type of practice// Illness care Psychosocial care Residuals$:
223 -0.847 - 1.285 0.336 -0.337 -0.467 -
(1.047) (0.177) (0.417) (0.241) (0.229) (0.397) -
variance
Equation
Visits
Visits Satisfaction
21.393
variable Satisfaction -0.039 0.199
(0.014) (0.027)
matrix Equation Satisfaction
(2.227)
0.289
(0.27)
*Estimates are based on data from a sample of 376 respondents. Khi-square = 0.978: d.f. = 5: P = 0.964. $This table gives unstandardized coefficients (with their standard errors in parentheses). PCoefficients with values tixed to one or zero are noted by a dash. IlCoded I = large group; 2 = small group; 3 = partnership: 4 = solo practice.
values that are given in, or can be easily estimated from, Table 1.) The final model, in Fig. 1, shows that the child’s age and health, the mother’s age, the type of practice and the quality of the doctor’s illness care only influence satisfaction indirectly through visits. Older children and healthly children visit less, older mothers take their children to the pediatrician more, equal
to the sample
I
CHILD’S HEALTH
children whose mothers take them to small, fee-forservice practices visit less often and children whose doctors provide high-quality illness care visit less often. The model also illustrates our assumption that the quality of a doctor’s psychosocial care only affects the number of visits through its influence on the mother’s satisfaction. The estimated parameters show that the better a pediatrician’s psychosocial care, the more satisfied the mother is with the doctor’s services. The results that bear directly on our hypotheses are described below. H,: Satisfaction leads to more frequent visits. Figure 1 shows that the more satisfied a mother is with her pediatrician, the more frequently she visits the pediatrician. The standardized effect of satisfaction on visits is 0.272. Thus, the model is consistent with H, H?, vs H,,: Visits decrease vs increase satisfaction. Figure 1 shows that the frequency of a mother’s visits to her pediatrician has a negative impact on her satisfaction. The standardized coefficient is -0.320. The results support the hypothesis that visits and satisfaction form a self-regulating system, not a selfamplifying system. Thus Hla is supported while H’b is not. Federol employees’ benejit program The final model for the Federal employee data has three restrictions in addition to those in the initial model. The Chi-square statistic for the overall fit is 7.433, with six degrees of freedom (1.239 Chisquares per degree of freedom). The probability that the differences between the observed and predicted variances and covariances could occur by chance alone is 0.283, which means that the model reproduces the data moderately well. The unstandardized coefficients
I
0 990
-
“v
MOTHER’S AGE 0272
320 TYPE OF PRACTICE I
T I
ElSATISFACTION
0 927
“S
Fig. I. Reciprocal effects of the frequency of visits and satisfaction with the doctor’s care in New Haven P = 0.964. All coefficients are pediatric practices. Chi square = 0.978, with 5 degrees of freedom. standardized, Estimates are based on data from a sample of 376 respondents.
Patient satisfaction and visiting the doctor
tent with H, and replicates the findings for the New Haven pediatric practices. H,, vs H,,: Visits decrease versus increase satisfaction Figure 2 shows that the more frequently a Federal employee visited the doctor during the study period the less satisfied he or she was likely to become at the end of the period (net of satisfaction at the beginning). The standardized coefficient is -0.163. Hla is therefore supported while H,, is not. The final model supports the hypothesis that visits and satisfaction form a self-regulating system and not a selfamplifying one. This replicates the results of the analysis for New Haven pediatric practices.
and their standard errors are given in Table 3. Figure 2 illustrates the structure of the final Federal employees’ model, along with its standardized coefficients. (For visual clarity correlations among the exogenous variables are not shown in Fig. 2. The correlations can be calculated easily from the variances and covariances in Table 3.) The final model, in Fig. 2, shows that having a personal physician and feeling healthy only influence a patient’s number of visits indirectly through their impact on the patient’s satisfaction at the beginning of the study period. People who have a personal physician and believe they are healthy are more satisfied with their physicians. The type of practice directly influences both satisfaction and visits-feefor-service patients are more satisfied but visit less. This is in partial contrast to the results for the New Haven pediatric practices, where the type of practice has a direct influence on the frequency of visits but not on satisfaction. Figure 2 also shows that, among the Federal employees studied, race and sex only influence satisfaction through their impact on the frequency of visits. Whites and females visit more frequently than blacks and males. The results that bear directly on our hypotheses, are described below. H,: Satisfaction leads to more frequent visits. Figure 2 shows, that a Federal employee’s satisfaction with his or her health care increases the employee’s expected frequency of visits. The standardized coefficient representing the effect of satisfaction at the beginning of the study period on visits during the period is 0.179. Thus, the model is consis-
Table
3. LISREL
1359
DISCUSSION
To date the statistical models used to test the hypothesized reciprocal effect between client satisfaction and utilization of physician services have not been completely appropriate. Early studies that assumed one causal order or the other [8,9] were followed by an overtime data analysis, that, while more sophisticated than the earlier research, made the assumption of no correlations among the residuals and found the unexpected result that client satisfaction leads to fewer visits [l 11. In our models we do not assume that there are no correlations among residuals. We find that satisfaction with the doctor increases the frequency of visiting the doctor which, in turn, decreases satisfaction. The selfregulating reciprocal effects between satisfaction and
of the coefficients in the final Federal employees
estimates*
modelt Effect coefficients:
Independent
variable
Satisfaction +
Visits Satisfaction 1 Personal physician Subjective health Fee-for-service White Female
1.718 0.273 0.447
Exogenous 1. 2. 3. 4. 5.
Personal physician Subjective health Fee-for-service White Female
variables: 1
Satisfaction Visits Satisfaction
(0.262) (0.156) (0.232)
1 2
-
_
0.068 -0.677 0.595 0.609
(0.060) (0.111) (0.097) (0.095)
variance
variance-covariance Satisfaction
Equation
Dependent variable Visits Satisfaction 2
-
0.230 -0.019 0.079 0.060 0.007 Residuals:
1
12.377 - 1.342 - 7.273
1
(0.611) (0.764) (1.828)
matrix
(0.261) (0.141) -
I/ 4
0.250 0.080 -0.012
0.250 -0.038
matrix Equation Visits 1.806 1.402
-0.411 0.903 -
(0.185) (0.686)
5
0.230
Satisfaction
14.278
2
(2.203)
*Estimates are based on data from a sample of 821 respondents. tChi-square = 7.433: d.f. = 6: P = 0.283. :This table gives unstandardized coefficients (with their standard errors in parentheses). ljcoefficients with values fixed to one or zero are noted by a dash. /‘No standard errors are given because the model assumes that the variances and covariances of the exogenous variables are fixed to the sample values..
1360
JOHN Y~IRUWSKY and CATHERINE E. Ross
SUBJECTIVE HEALTH
FEE-FORSERVICE
WHITE I
r-l FEMALE
L
_I
Fig. 2. Reciprocal &ects of the frequency of visits and satisfaction with the doctor’s care in the Federal Employees’ Benefit Program. Chi square = 7.433 with 6 degrees of freedom. P = 0.283. All coefficients are standardized. Estimates are based on data from a sample of 821 respondents.
visits to the doctor are found amohg both the clients of New Haven pediatricians and the members of the Federal Employees Health Benefits Program. The similarity of the results. given the differences between the study populations, medical specialties, research designs, and measurement procedures, underscores the robustness of our finding. The self-regulating relationship between satisfaction and visits does not appear to be a quirk of medical practice in a particular place in the United States or of a particular type of medical practice that was studied in a particular way. Our results support the hypothesis that satisfaction with the doctor and the frequency of visiting the doctor form a self-regulating system. The more that people are satisfied with their doctors the more frequently they will visit the doctor, but as the frequency of contact increases, so does the probability that the patients’ high expectations will be unmet. As the frequency of visits increases so does the likelihood of an encounter in which the physician is gruff, inattentive, or inept. Eventually, the client may come in with a problem for which the physician has no easy solution. In the case of a chronic or recurring problem the patient may realize that the doctor cannot solve the problem quickly, easily and finally. As awareness of the doctor’s limitations increases, the patient’s satisfaction decreases. Since the less-satisfied patient tends to visit the doctor less frequently. the
result is a self-regulating reciprocal relationship between satisfaction and visits. We do not find support in our results for the idea that patients become satisfied with their doctors as a result of visits. We find no evidence that patients adapt psychologically to frequent visits by increasing their satisfaction. If such an adjustment is occurring it is outweighed by the disappointment that results from unmet expectations. Even though unintended, the self-regulating feedback between satisfaction and the use of services has a number of desirable consequences for the patient, the physician and society. Because more visits lead to lower satisfaction and lower satisfaction leads to fewer visits, the patient avoids excessive dependency on the physician. By the same token, the physician avoids excessive demands. Society gains because physician services are distributed more evenly across patients and because persons, and their resources, are less likely to be entrapped by accelerating dependency. REFERENCES
I. Andersen Health
R. A Behaaiorui Model cjf Fumilies Use of Services. University of Chicago Center for
Health Administration Studies, Chicago, 1968. 2. Rosenstock 1. M. Why people use health services. Milhonk
meml.
Fund Q. 44, 94. 1966.
Patient
satisfaction
and visiting
E. A. Sociomedical variations among ethnic 3. Suchman groups. Am .I. Social. IO, 319, 1964. 4. Deisher R. W., Engle W., Spieholz R. and Standfast S. J. Mothers’ opinions of their pediatric care. Pediatrics 35, 82. 1965. 5. Larsen D. E. and Rootman I. Physician role performance and patient satisfaction. Sot. Sci. Med. 10,29. 1976. of 6. Linn L. S. Factors associated with patient evaluation health care. Milbank meml. Fund. Q. 53, 531, 1975. I. Aday L. A. and Andersen R. A framework for the study of access to medical care. Hlth Serv. Res. 9, 208. 1974. 8. Wolinsky F. D. Health service utilization and attitudes toward health maintenance organizations: a theoretical and methodological discussion. J. Hllh sot. Eehac. 17, 221, 1976. 9. Roghmann K. J., Hengst A. and Zastowny T. R. Satisfaction with medical care: its measurement and relation to utilization. Med. Care 17, 461. 1979. IO. Hanushek E. A. and Jackson J. E. Statisrical Melhods for Social Scientisrs. Academic Press, New York, 1977. II. Gray L. C. Consumer satisfaction with physician provided services: a panel study. Sot. Sci. Med. 14, 65, 1980. 12. Erlanger H. and Winsboro H. The subculture of violence thesis: an example of a simultaneous equation model in sociology. Sot. Meth. Res. 5, 231. 1976, state that although a number of theories in sociology explicitly or implicitly propose reciprocal causal relationships, few researchers model these relationships statistically. This is especially true in medical sociology. Therefore, in this paper, we limit our focus to the reciprocal relationship of client satisfaction and utilization. We take a selective rather than a general approach since the separate issues of client satisfaction and of utilization have been addressed more broadly elsewhere. Rather than restating the extensive theory and research on satisfaction and, especially, on utilization here, we refer readers to reviews and syntheses: Aday L. A. and Eichhorn R. The Urili:arion of Health Services: Indices and Correlates. National Center for Health Services Research and Development, Rockville, MD, 1972: Andersen R.. Kravits J. and Andersen 0. W. Equity in Health Sciences: Empirical Anal.vsis in Social Policy. Balhnger. Cambridge, MA, 1975; Ben-Sira Z. The function of the professionals’ affective behavior in client satisfaction: a revised approach to social interaction theory. J. Hlth sot. Behav. 17, 3, 1976; Dutton D. B. Explaining the low use of health services by the poor: costs, attitudes or delivery systems? Am. Social. Rev. 43, 348. 1978: Freidson E. Patients’ Views of Medical Practice. Russell Sage, New York. 1961: Mechanic D. The Growth qf Bureaucratic Medicine. Wiley. New York. 1976; McKinlay J. B. Some approaches and problems in the study of the use of services. J. Hlfh sot.
13.
14. 15.
16.
17. 18.
19.
20.
21.
22.
23. 24. 25.
the doctor
1361
Eehav. 13. 115. 1972: Shortell S. M.. Richardson W. C., LoGerto J. P., Diehr P., Weaver B. and Green K. E. The relationships among dimensions of health services in two provider systems: a causal model approach. J. Hlth sot. Behav. 18, 139. 1977. Bern D. J. Self-perception theory. In Advances in Experimenral Social Psycho/og_v (Edited by Berkowitz L.), Vol. 6. Academic Press, New York, 1972. Aronson E. The Social Animal. Freeman, San Francisco, 1972. Finifter B. M. The generation of confidence: evaluating research findings by random subsample replication. In Socioloaical Methodoloav (Edited bv Costner H. L.). PP. I 12-l 75. Josey-Bass, San Francisco, 1972. . Alexander K. L., Reilly T. W. and Fennessey J. Issues in instrumental variables analysis. Am. Socioi. Rev. 46, 937, 1981. Cramer J. C. Fertility and female employment: problems of causal direction. Am. social. Rev. 45, 167. 1980. Jiireskog K. G. and Sorbom D. Advances in Factor Analysis and Srructural Equation Models. Abt Books, Cambridge, MA, 1979. Joreskog K. G. and Sorbom D. LISREL IV: Analysis of Linear Swucrural Relationships by [he Method of Maximum Likelihood (User’s Guide). International Education Services, Chicago, 1980. Ross C. E., Wheaton B. and Dull’ R. S. Client satisfaction and the organization of medical care: why time counts. J. Hlrh sot. Behat?. 22, 243. 1981. Ross C. E. and Duff R. S. Returning to the doctor: the effect of client characteristics. type of practice and experiences with care. J. Hlth sot. Eehav. 23, 119, 1982. Because the large multispecialty groups are prepaid, and the small practices are fee-for-service, we unfortunately cannot disentangle organizational versus payment effects of type of practice on utilization or satisfaction. However, since this is not a major focus of the present analysis, we hope future research will examine these effects separately. Asher H. B. Causal Modeling. Sage Publications, Beverly Hills, CA. 1976. Tessler R. Birth order, family size and children’s use of physician services. Hlrh Sem. Res. 15, 55. 1980. According to Joreskog and Sorbom, in the Chi-square test the null hypothesis is that there is no difference between the observed variance-covariance matrix and the one reproduced by the model. The alternative hypothesis is that there is a difference between them that cannot be attributed to chance alone. Since we would like any differences to be due to chance. and not to errors in the model, we would like the probability value associated with the Chi-square test to be as large as possible. The probability levels associated with the Chi-square statistic in textbook examples typically range from 0.05, to 0.40.