- Email: [email protected]
Client satisfaction and social desirability in psychotherapy
Evahration and Program Planning, Vol.
0149-7189/92
15,pp. 227-231, 1992
$5.00 + .OO
Copyright (in 1992 Pergamon Press Lrd.
Printed in the USA. All rights reserved.
CLIENT SATISFACTION AND SOCIAL DESIRABILITY IN PSYCHOTHERAPY
LOUISE GASTON Department of Psychiatry, McGill University
STBPHANE SABOURIN Department of Psychology, University of Montreal
ABSTRACT In this study, the authors examined whether reports of client satisfaction in psychotherapy are biased by client tendency to provide socially desirable answers. This question was explored in the context of dynamic, eclectic, and cognitive/behavioral modalities of psychotherapy, which were provided in private practice settings. The Client Satisfaction Questionnaire-8 (CSQ-8) was employed to assess satisfaction, and the Marlowe-Crowne Social Desirability Scale (MCSD) was used to measure social desirability. The relation of CSQ-8 scores was also examined with respect to sociodemographic variables, symptomatology, and treatment characteristics. A very weak correlation between CSQ-8 and MCSD scores was obtained (r(146) = -.OS, p > .05), which was not found to be significant. This association also was not found to be moderated by client’s and therap~sf ~ gender, therapy modaiity, or freatrneni length. These results co&d be only partially explained by the resfricfed variance associated with CSQ-8 scores. it thus appears that client sai~sfacfion represents a ~egif~tnafe oufrome dimension of psychoihera~y.
studies examined the plausibility of the social desirability critique. In one study, a significant but weak association was obtained between satisfaction scores and estimates of impression management, and a nonsignificant association was observed with estimates of selfdeception (Sabourin, Laferrittre, et al., 1989). The other study yielded somewhat higher coefficients, along with a wider range of satisfaction scores (Sabourin, Bourgeois, Gendreau, & Morval, 1989) (see Table 1). As suggested by Sabourin et al. (1989), the present study aims at examining whether client satisfaction reports are biased by a tendency to provide socially desirable answers, using the Marlowe-Crowne Social Desirability Scale (MCSD). An additional unique aspect of this study is the examination of this plausible bias across diverse types of psychotherapy, that is, dynamic, eclectic (humanistic-dynamic), and cognitive/ behavioral.
While there is a persistent belief that client reports of satisfaction in psychotherapy do not represent a valid outcome variable, satisfaction recently has been proposed as a legitimate outcome dimension of psychotherapy (Kazdin, 1986; Lambert, Shapiro, & Bergin, 1986). The two major arguments in favor of this persistent belief are that clients tend to provide positively skewed evaluations of psychotherapy and that satisfaction reports are biased by socially desirable responding (e.g., Lebow, 1983). When empirical strategies have addressed these arguments, only partial support was found, however. Although it is true that clients usually report being quite satisfied with psychotherapy (e.g., Attkisson & Zwick, 1982), client satisfaction has been consistently found to be associated with dropout rates (e.g. Attkisson & Zwick, 1982; Kokotovic & Tracey, 1987; Sabourin, Gendreau, & FrCnette, 1987), indicating that clients do also report their dissatisfaction (see Table 1). Two
Requests for reprints should be sent to Dr. Louise Caston, Department of Psychiatry, McGill University, 1033 Pine Avenue West, MontrPal (QuCbec), Canada H3A 1A 1.
227
LOUISE
228
GASTON
and STl?PHANE
SABOURIN
METHOD Subjects One-hundred forty-seven subjects participated in this study. The subjects were patients consulting in private practice for individual psychotherapy services. All subjects were Caucasian; 3 1% were men and 69% were women. Their mean age was 35.33 years (SD = 7.85), and they attended school, college, or university for an average of 15.88 years (SD = 3.10). Fifty-four percent were single, 29% were married or living with a partner, and 20% were either divorced or separated. They earned yearly an average of 30,660 dollars (SD = 18,360). Their professional status was as follows: professionals (47%), specialized workers (25%), nonspecialized workers (16070), students (7%), unemployed (3%), and housewife (2%). Subjects were described by their therapists as mostly presenting symptoms of depression, anxiety, and/or interpersonal difficulties. When they completed the study questionnaires, the subjects had spent an average of 87.33 weeks in therapy (SD = 104.12; range l-600), and most patients attended therapy once a week (M = 1.09). Only four subjects (3%) had previous psychiatric hospitalizations. Measures Satisfaction. Client satisfaction in psychotherapy was measured by the Client Satisfaction Questionnaire (CSQ-8; Larsen, Attkisson, Hargreaves, & Nguyen, 1979), which is an eight-item self-report questionnaire assessing consumer satisfaction with mental health services. The CSQ-8 evaluates various dimensions of consumer satisfaction: physical surroundings; kind/type of treatment; treatment staff; quality of service; amount, length, or quantity of service; outcome of service; general satisfaction; and procedures. For each item, scores vary from 0 to 4, with higher scores reflecting greater
TABLE 1 PREVIOUS
ASSOCIATIONS
AND CRITERION-RELATED
Study
p <
,001
Attkisson
p <
.OOi
Kokotovic
& Tracey
t = 6.99, p < ,001 r = .20, p < .05
Sabourin, Sabourin,
et al. (1987) Laferrihre. et al.
r =
Sabourin.
Dropout Dropout
rate rate
r =
Dropout
rate
.57,
t = 3.82,
management Impression
& Zwick
(1982)
p <
.Ol
Bourgeois,
et al.
(1989) r =
.08,
p > .05
Sabourin,
Laferrihre,
et al.
(1989) Self-deception
r =
.l 9.
p<
.05
Sabourin,
Bourgeois,
et al.
(1989) CSQ-8
= Client Satisfaction
Questionaire-8
Social Desirability. The Marlowe-Crowne Social Desirability Scale (MCSD; Crowne & Marlowe, 1960) was used to assess the subjects’ tendency to report socially desirable responses. It contains 33 items describing behaviors that are culturally sanctioned and approved, but for which occurrence is improbable. For each item, the respondent indicates whether the statement describes himself or herself by circling either “True” or “False.” The MCSD assesses a trait or an individual difference dimension, rather than a response bias because high scorers generally believe what they are reporting, and they attempt to behave accordingly (Crowne & Marlowe, 1964). In an outpatient population, the mean score was found to be 13.72 (SD = 5.78) (Crowne & Marlowe, 1960). The internal consistency of the MCSD was found to equal to .88. The MCSD correlated with MMPI validity scales but was unrelated to most psychopathology ones. The test-retest correlation for clinic outpatients was found to be .68 for a 5-month period, even though they participated in 20-25 sessions of psychotherapy (Strickland & Crowne, 1963). This scale is widely used to measure people’s tendencies to present and perceive themselves as behaving in a socially desirable manner, as well as to tap subjects’ levels of defensiveness (Tanaka-Matsumi & Kameoka, 1986). In the present sample, the mean MCSD score was 14.04 (SD = 5.27).
(1987)
(1989) .27,
management Self-deception
CSQ-8
SCORES
Result
Criterion
Impression
BETWEEN
satisfaction. In a sample of 3,120 respondents, the mean was 27.09 (SD = 4.01). Internal consistency of the CSQ-8 was found to be high, with alpha coefficients ranging from .84 to .93 (Larsen et al., 1979). Under factor analysis, the CSQ-8 has repeatedly yielded only one factor (e.g., Nguyen, Attkisson, 8z Stegner, 1983). CSQ-8 scores were found to be marginally to moderately correlated with symptomatic outcome and dropout rate (Sabourin et al., 1987). The CSQ-8 thus consists of a measure that is acceptably reliable and valid, tapping the consumer satisfaction aspect of treatment outcome. In this study, the CSQ-8 items were reworded to be adapted to the context of psychotherapy services delivered in private practice settings. The terms “our program” were replaced by “this service” whenever they were present; that is, in items 3, 4, and 8.
item (Larsen et al.. 1979).
Syrnptomatology. The symptomatic status of the subjects was assessed by the Symptoms Checklist-10 (Nguyen et al., 1983), which was derived from the SCL90 (Derogatis, Lipman, & Covi, 1973). The SCL-10 is a lo-item scale measuring global symptomatology. Based on a factor analysis conducted by Hoffamn and Overall (1978), the 10 items which had the highest loadings on the three most important factors of the SCL-10 were selected; six items reflect depression, two items
Client Satisfaction and Desirability represent somatization, and two items describe anxiety symptoms. The SCL-10 scores can range from 0 to 4, with higher scores reflecting greater symptomatology. In a sample of 3,120 respondents, the SCL-10 mean score was found to be 1.45 (SD = .92). The internal consistency of the SCL-10 was found to equal to .88, and its test-retest reliability was .59 after 4 months. A principal component factor analysis yielded only one factor. In an outpatient community sample, the SCL-10 was found to be highly correlated with the Psychiatric Symptom Inventory, a 29-item scale designed to tap psychological distress (Sabourin, Veronneau, & Gendreau, 1988). The SCL-10 thus represents a reliable and valid measure of psychological distress that offers the advantage of being shorter than the SCL-90. Representing mean constituent item score, the SCL-10 can range from 0 to 4. In this sample, the subjects presented almost-absent to severe psychological distress on the SCL-10 (M = 1.49; SD = .75; range 0.1-3.5). Psychotherapy Alliance. The alliance in psychotherapy
was assessed by the California Psychotherapy Alliance Scales-Patient Version (CALPAS-P; Gaston, 1991). This is a 24-item version filled out by patients after completion of a therapy session. A total score can be derived, as well as four scales scores. An internal consistency coefficient of .83 was found for the total CALPAS-P scale, the score to be reported in this study. Patients’, therapists’, and treatment characteristics were not found to be related to the CALPAS-P, although it was related to interpersonal difficulties and symptomatology. Finally, CALPAS-P was not found to be associated with patients’ tendencies to report socially acceptable answers (Gaston, 1991). A CALPAS-P score ranges from 1 to 7, representing the mean of item scores. In this study, the total CALPAS-P score was found to equal 5.96 (SD = .57; range 3.88-6.88), indicating relatively good alliances. Therapists
and Treatment
Of the 21 therapists who participated in the study, 19 were psychologists, 1 was a social worker, and 1 was a psychiatrist/psychoanalyst. Their mean age was 36.4 years (SD = 5.7). There were 15 female therapists (75%)
229
and 5 male therapists (25%). They have practiced individual psychotherapy for an average of 9.6 years (SD = 4.8), and they have been supervised for an average of 5.2 years (SD = 1.9). At the time of the study, they were seeing 4-25 patients in psychotherapy. When asked to describe their theoretical and clinical orientation, eight (41%) said that their main orientation was psychodynamic, eight (41%) characterized it as eclectic (humanistic and psychodynamic), three (15%) described it as behavioral, and one (4070)reported to use a systemic approach. As a result, the following numbers of subjects were treated in each treatment modality: 48 in dynamic psychotherapy; 44 in humanistic/dynamic psychotherapy; and 23 in behavioral/cognitive therapy. Procedure
Psychotherapists in private practice were recruited to participate in this study, which involves an examination of the relation between client satisfaction and social desirability, along with the validity of an alliance measure. The psychotherapists were told that their participation consisted of asking each of their patients if they would be interested in answering questionnaires about their psychotherapy. Of the 25 psychotherapists who were contacted, 21 agreed to participate in the study. They communicated to their patients that the information would be totally confidential since the patient names would not appear on the questionnaires. Patients who agreed to participate in the study were provided with an envelope containing questionnaires as well as a prestamped return envelope. Along with the questionnaires, a letter was provided to the patients that explained the purpose of the study, the time required to complete the questionnaires (15-30 minutes), and the confidentiality procedure. Patients were asked explicitly not to write their names on the questionnaires. A total of 225 questionnaires were distributed to the participating therapists. Since 147 patients participated in the study, the response rate equals 65%, which is comparable to the return rate of 68% reported by Tracey and Kokotovic (1989) and of 66% obtained by Sabourin et al. (1989). The reasons motivating the nonresponders, however, remain unknown, leaving the biases introduced by this recruitment strategy masked.
RESULTS In this study, the mean CSQ-8 score was 28.74 (SD = 3.61), with a range of 16 to 32. Cronbach’s coefficient of internal consistency equaled .93, with correlations of item with total (of remaining items) varying from .68 to .81. A principal component factor analysis of the CSQ-8 items yielded one factor associated with an eigen value of 5.39, which accounted for 67.3% of the variance. The loadings of the eight items ranged from .75 to .87,
indicating that all items contributed to this single factor. The factor associated with the next highest eigen value (.64) accounted for 7.9% of the variance only, suggesting that it did not contribute substantially to explain the variance of CSQ-8 scores. The association of CSQ-8 scores with criterion variables was performed using Pearson correlation or analysis of variance (including t-test) whenever appropriate. Given the number of analyses performed, the alpha
230
LOUISE GASTON
and
level of .05 was reduced to .005 by the Bonferroni adjustment criterion to counter the family-wise error rate. As shown in Table 2, no significant correlation was observed between CSQ-8 scores and patients’ age, number of years of education, and yearly income. No significant level of difference was revealed between CSQ-8 scores reported by male or female clients, as well as between CSQ-8 scores reported with respect to male or female therapists. In addition, the three psychotherapy modalities were not found to yield significant CSQ-8 level differences. A significant correlation was observed, however, between CSQ-8 scores of the number of weeks in psychotherapy, with only 5% of the variance of the criterion variable accounted for. The association between CSQ-8 scores and SCL-10 scores was marginally significant, with a shared variance of 5%. In contrast, a markedly significant relation was observed between CSQ-8 scores and CALPAS-P alliance scores (see Table 2). The CSQ-8 scores were not found to be significantly related to the MCSD scores, revealing that client satisfaction was not substantially related to estimates of social desirability (see Table 2). Although no direct association between client satisfaction and social desirability was observed, their relation could be moderated by predictors such as sex of patient or therapist, treatment length, or treatment modality. To test the contribution of these potentially moderating variables, four hierarchical regression analyses were conducted with CSQ-8 scores as the criterion. Each analysis was performed with the predictor entered at the first step, social desirability scores at the second step, and with their
ASSOCIATION
TABLE 2 OF CSQ-8 AND CRITERION-RELATED
Criterion
Test
Age
r(140)
Education Income Psychotherapy length Distress (XL- 10) Social desfrability (MCSD) Alliance (CALPAS-P) Subject gender Therapist gender Psychotherapy modality
r(137)
p Value
r(146)
03 02 = .07 = .24 = -.22
r(146)
= -
r(146)
=
r(130) r(143)
SCORES
=
= -
05 .65
f(141) = -98 t(l29) = 1.02 F(2,l 12) = -81
.76
a0 45 ,004 006 52 .ooo 33 .31 45
CSQ-8 = Clrent Satisfaction Questionaire-8 Item (Larsen, et al.. 1979); SCL-IO = Symptoms ChecklIst-10 Item (Nguyen et al.. 1983), MSCD = Marlow-Crowne Social Desirability Scale (Crowne & Marlowe, 1960); CALPAS-P = California Psychotherapy Alltance Scales (Gaston, 1991)
interaction terms at the third step. Because four analyses were conducted, the alpha level of .05 was reduced to .0125 using the Bonferroni adjustment criteria. Neither main effects nor interaction terms were found to be statistically significant, In the interest of comparing these results with those associated with symptomatology, we computed the correlation between SCL-10 and MCSD scores. A significant but weak association was obtained (r( 146) = - .21, p < .Ol), indicating that a greater tendency toward socially desirable responding yielded self-reports of lower symptomatology.
DISCUSSION As expected, CSQ-8 scores were not found to be related to estimates of social desirability, either directly or through the mediation of selected variables. These results are especially striking given that, in contrast, symptomatology self-reports, a sine qua non outcome variable of psychotherapy, were found to be marginally but significantly associated with socially desirable responding. Furthermore, no significant association was observed between CSQ-8 scores and clients’ sociodemographic variables. A weak but significant relation was obtained between satisfaction and the number of weeks of psychotherapy, while treatment length did not interact with estimates of social desirability to predict satisfaction. The very small correlation observed between satisfaction scores and social desirability estimates, -.05, was lower than those reported by Sabourin et al. (1989), .27 and .19. This variety of findings could be due to the different measures of socia1 desirability employed: Marlowe Crowne Social Desirability Scale in this study; and Self-Deception Questionnaire and Other-Deception
Questionnaires in the Sabourin et al.‘s study. The lack of significant correlation between satisfaction and social desirability in this study also could be attributed to restricted variance associated with CSQ-8 scores. The standard deviation of CSQ-8 scores of the present sample equaled 3.6, while that obtained by Sabourin et al. (1989) was 5.9. Using the test for difference between variances of two independent samples (Bruning & Kintz, 1968), the variance of this study was found to be statistically smaller (F’(80, 146) = 2.64, p < .Ol) than that reported by Sabourin et al. (1989). Therefore, the correlation between CSQ-8 and MCSD scores could be due in part to the restricted variance associated with CSQ-8 scores. In contrast with this argument, CSQ-8 scores have been found to be highly related to a convergentvalidity criterion, that is, a measure of alliance in psychotherapy. Although a restricted range necessarily lowers the maximum I’, these CSQ-8 scores varied sufficiently to yield substantial associations with selected criterion variables. Moreover, scores of both satisfaction and social desirability were drawn from the client’s
Client Satisfaction and Desirability perspective, a strategy which was likely to increase the correlation between variables (Orlinsky & Howard, 1986). No difference in levels of CSQ-8 scores was found across clients treated in dynamic, eclectic (humanistic and dynamic) or cognitive/behavioral therapy. This observed lack of significant difference suggests that the measurement of client satisfaction could be useful to assess the outcome of diverse psychotherapy approaches, a methodological strategy suggested by Kazdin (1986). The present findings thus provide some support to the
231
discriminant validity of client satisfaction measures, which could be viewed as relatively independent from the tendency to provide socially desirable impressions. Client satisfaction also appears to be a nonredundant outcome variable of psychotherapy because a small and marginally significant association was observed between satisfaction and symptomatology scores. The CSQ-8 could thus be used to assess client satisfaction in an outpatient psychotherapy setting in which clients are mostly adults with at least a moderate level of education and an average yearly income.
REFERENCES ATTKISSON, Questionnaire: psychotherapy 233-237. BRUNING, of statistics.
C.C., & ZWICK, R. (1982). The Client Satisfaction Properties and correlations with service utilization and outcome. Evaluation and Program Planning, 5,
J.L., & KINTZ, B.L. (1968). Compufational Chicago: Scott, Foreman and Company.
handbook
CROWNE, D.P., & MARLOWE, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354.
LEBOW, J.L. (1983). Research assessing consumer satisfaction with mental health treatment: A review of findings. Evaluation and Program Planning, 6, 21 l-236. NGUYEN, T.D., ATTKISSON, C.C., & STEGNER, B.L. (1983). Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6, 299-3 14.
motive:
ORLINSKY, D.E., &HOWARD, K.I. (1986). Process and outcome in psychotherapy. In S.L. Garfield and A.E. Bergin (Eds.), Handbook of Psychotherapy and Behavior Change (pp. 3 1 l-384). New York: Wiley.
DEROGATIS, L., LIPMAN, R., & COVI, L. (1973). The SCL-90: An outpatient rating scale (Preliminary report). Psychopharmacology Bulletin, 9, 13-28.
SABOURIN, S., BOURGEOIS, L., GENDREAU, P., & MORVAL, M. (1989). Self-deception, impression management, and consumer satisfaction with mental health treatment. Psychological Assessment: A Journal of Consulting and Clinical Psychology, I, 126- 129.
CROWNE, D.P., & MARLOWE, Studies in evaluative dependence.
D. (1964). The approval New York: Wiley.
GASTON. L. (1991). Reliability and criterion-related validity of the California Psychotherapy Alliance Scales-Patient version. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 68-74. HOFFMAN, N.G., &OVERALL, P.B. (1978). Factor structure of the SCL-90 in a psychiatric population. Journal of Personality and Social Psychology, 37, 147-169. KAZDIN, A.E. (1986). Comparative outcome studies of psychotherapy: Methodological issues and strategies. Journal of Consultingand Clinical Psychology, 54, 95-105. KOKOTOVIC, A.M., & TRACEY, T.J. (1987). Premature termination at a university counseling center. Journal of CounselingPsychology, 34, 80-82. LAMBERT, M.J., SHAPIRO, D.M., & BERGIN, A.E. (1986). The effects of psychotherapy. In S.L. Garfield and A.E. Bergin (Eds.), Handbook of psychotherapy and behavior change (3rd ed., pp. 157212). New York: Wiley. LARSEN, D.L., ATTKISSON, C.C., HARGREAVES, W.A., & NGUYEN, T.D. (1979). Assessment of client/patient satisfaction: Development of a general scale. Evaluation and Program Planning, 2, 197-207.
SABOURIN, S., GENDREAU, P., & FRENETTE, L. (1987). Le niveau de satisfaction des cas d’abandon dans un service universitaire de psychologie [Client satisfaction and dropout cases in a university psychology clinic]. Revue Canadienne des Sciences du Comportement, 19, 314-323. SABOURIN, S., LAFERRIERE, N., SICURO, F., COALLIER, J.C., COURNOYER, L.G., & GENDREAU, P. (1989). Social desirability, psychological distress, and consumer satisfaction with mental health treatment. Journal of Counseling Psychology, 36, 352-356. SABOURIN, psychometric
S., VERONNEAU, F., & GENDREAU, P. (1988). The qualities of the SCL-10. Unpublished data.
STRICKLAND, B.R., & CROWNE, D.P. (1963). Need for approval and premature termination of psychotherapy. Journal of Consulting Psychology, 27, 95-101. TANAKA-MATSUMI, J., & KAMEOKA, V.A. (1986). Reliabilities and concurrent validities of popular self-report measures of depression, anxiety, and social desirability. Journal of Consulting and Clinical Psychology, 54, 328-333. TRACEY, T.J., & KOKOTOVIC, A.M. (1989). Factor structure of the Working Alliance Inventory. PsychologicalAssessment: A Journal of Consulting and Clinical Psychology, I, 207-210.
0149-7189/92
15,pp. 227-231, 1992
$5.00 + .OO
Copyright (in 1992 Pergamon Press Lrd.
Printed in the USA. All rights reserved.
CLIENT SATISFACTION AND SOCIAL DESIRABILITY IN PSYCHOTHERAPY
LOUISE GASTON Department of Psychiatry, McGill University
STBPHANE SABOURIN Department of Psychology, University of Montreal
ABSTRACT In this study, the authors examined whether reports of client satisfaction in psychotherapy are biased by client tendency to provide socially desirable answers. This question was explored in the context of dynamic, eclectic, and cognitive/behavioral modalities of psychotherapy, which were provided in private practice settings. The Client Satisfaction Questionnaire-8 (CSQ-8) was employed to assess satisfaction, and the Marlowe-Crowne Social Desirability Scale (MCSD) was used to measure social desirability. The relation of CSQ-8 scores was also examined with respect to sociodemographic variables, symptomatology, and treatment characteristics. A very weak correlation between CSQ-8 and MCSD scores was obtained (r(146) = -.OS, p > .05), which was not found to be significant. This association also was not found to be moderated by client’s and therap~sf ~ gender, therapy modaiity, or freatrneni length. These results co&d be only partially explained by the resfricfed variance associated with CSQ-8 scores. it thus appears that client sai~sfacfion represents a ~egif~tnafe oufrome dimension of psychoihera~y.
studies examined the plausibility of the social desirability critique. In one study, a significant but weak association was obtained between satisfaction scores and estimates of impression management, and a nonsignificant association was observed with estimates of selfdeception (Sabourin, Laferrittre, et al., 1989). The other study yielded somewhat higher coefficients, along with a wider range of satisfaction scores (Sabourin, Bourgeois, Gendreau, & Morval, 1989) (see Table 1). As suggested by Sabourin et al. (1989), the present study aims at examining whether client satisfaction reports are biased by a tendency to provide socially desirable answers, using the Marlowe-Crowne Social Desirability Scale (MCSD). An additional unique aspect of this study is the examination of this plausible bias across diverse types of psychotherapy, that is, dynamic, eclectic (humanistic-dynamic), and cognitive/ behavioral.
While there is a persistent belief that client reports of satisfaction in psychotherapy do not represent a valid outcome variable, satisfaction recently has been proposed as a legitimate outcome dimension of psychotherapy (Kazdin, 1986; Lambert, Shapiro, & Bergin, 1986). The two major arguments in favor of this persistent belief are that clients tend to provide positively skewed evaluations of psychotherapy and that satisfaction reports are biased by socially desirable responding (e.g., Lebow, 1983). When empirical strategies have addressed these arguments, only partial support was found, however. Although it is true that clients usually report being quite satisfied with psychotherapy (e.g., Attkisson & Zwick, 1982), client satisfaction has been consistently found to be associated with dropout rates (e.g. Attkisson & Zwick, 1982; Kokotovic & Tracey, 1987; Sabourin, Gendreau, & FrCnette, 1987), indicating that clients do also report their dissatisfaction (see Table 1). Two
Requests for reprints should be sent to Dr. Louise Caston, Department of Psychiatry, McGill University, 1033 Pine Avenue West, MontrPal (QuCbec), Canada H3A 1A 1.
227
LOUISE
228
GASTON
and STl?PHANE
SABOURIN
METHOD Subjects One-hundred forty-seven subjects participated in this study. The subjects were patients consulting in private practice for individual psychotherapy services. All subjects were Caucasian; 3 1% were men and 69% were women. Their mean age was 35.33 years (SD = 7.85), and they attended school, college, or university for an average of 15.88 years (SD = 3.10). Fifty-four percent were single, 29% were married or living with a partner, and 20% were either divorced or separated. They earned yearly an average of 30,660 dollars (SD = 18,360). Their professional status was as follows: professionals (47%), specialized workers (25%), nonspecialized workers (16070), students (7%), unemployed (3%), and housewife (2%). Subjects were described by their therapists as mostly presenting symptoms of depression, anxiety, and/or interpersonal difficulties. When they completed the study questionnaires, the subjects had spent an average of 87.33 weeks in therapy (SD = 104.12; range l-600), and most patients attended therapy once a week (M = 1.09). Only four subjects (3%) had previous psychiatric hospitalizations. Measures Satisfaction. Client satisfaction in psychotherapy was measured by the Client Satisfaction Questionnaire (CSQ-8; Larsen, Attkisson, Hargreaves, & Nguyen, 1979), which is an eight-item self-report questionnaire assessing consumer satisfaction with mental health services. The CSQ-8 evaluates various dimensions of consumer satisfaction: physical surroundings; kind/type of treatment; treatment staff; quality of service; amount, length, or quantity of service; outcome of service; general satisfaction; and procedures. For each item, scores vary from 0 to 4, with higher scores reflecting greater
TABLE 1 PREVIOUS
ASSOCIATIONS
AND CRITERION-RELATED
Study
p <
,001
Attkisson
p <
.OOi
Kokotovic
& Tracey
t = 6.99, p < ,001 r = .20, p < .05
Sabourin, Sabourin,
et al. (1987) Laferrihre. et al.
r =
Sabourin.
Dropout Dropout
rate rate
r =
Dropout
rate
.57,
t = 3.82,
management Impression
& Zwick
(1982)
p <
.Ol
Bourgeois,
et al.
(1989) r =
.08,
p > .05
Sabourin,
Laferrihre,
et al.
(1989) Self-deception
r =
.l 9.
p<
.05
Sabourin,
Bourgeois,
et al.
(1989) CSQ-8
= Client Satisfaction
Questionaire-8
Social Desirability. The Marlowe-Crowne Social Desirability Scale (MCSD; Crowne & Marlowe, 1960) was used to assess the subjects’ tendency to report socially desirable responses. It contains 33 items describing behaviors that are culturally sanctioned and approved, but for which occurrence is improbable. For each item, the respondent indicates whether the statement describes himself or herself by circling either “True” or “False.” The MCSD assesses a trait or an individual difference dimension, rather than a response bias because high scorers generally believe what they are reporting, and they attempt to behave accordingly (Crowne & Marlowe, 1964). In an outpatient population, the mean score was found to be 13.72 (SD = 5.78) (Crowne & Marlowe, 1960). The internal consistency of the MCSD was found to equal to .88. The MCSD correlated with MMPI validity scales but was unrelated to most psychopathology ones. The test-retest correlation for clinic outpatients was found to be .68 for a 5-month period, even though they participated in 20-25 sessions of psychotherapy (Strickland & Crowne, 1963). This scale is widely used to measure people’s tendencies to present and perceive themselves as behaving in a socially desirable manner, as well as to tap subjects’ levels of defensiveness (Tanaka-Matsumi & Kameoka, 1986). In the present sample, the mean MCSD score was 14.04 (SD = 5.27).
(1987)
(1989) .27,
management Self-deception
CSQ-8
SCORES
Result
Criterion
Impression
BETWEEN
satisfaction. In a sample of 3,120 respondents, the mean was 27.09 (SD = 4.01). Internal consistency of the CSQ-8 was found to be high, with alpha coefficients ranging from .84 to .93 (Larsen et al., 1979). Under factor analysis, the CSQ-8 has repeatedly yielded only one factor (e.g., Nguyen, Attkisson, 8z Stegner, 1983). CSQ-8 scores were found to be marginally to moderately correlated with symptomatic outcome and dropout rate (Sabourin et al., 1987). The CSQ-8 thus consists of a measure that is acceptably reliable and valid, tapping the consumer satisfaction aspect of treatment outcome. In this study, the CSQ-8 items were reworded to be adapted to the context of psychotherapy services delivered in private practice settings. The terms “our program” were replaced by “this service” whenever they were present; that is, in items 3, 4, and 8.
item (Larsen et al.. 1979).
Syrnptomatology. The symptomatic status of the subjects was assessed by the Symptoms Checklist-10 (Nguyen et al., 1983), which was derived from the SCL90 (Derogatis, Lipman, & Covi, 1973). The SCL-10 is a lo-item scale measuring global symptomatology. Based on a factor analysis conducted by Hoffamn and Overall (1978), the 10 items which had the highest loadings on the three most important factors of the SCL-10 were selected; six items reflect depression, two items
Client Satisfaction and Desirability represent somatization, and two items describe anxiety symptoms. The SCL-10 scores can range from 0 to 4, with higher scores reflecting greater symptomatology. In a sample of 3,120 respondents, the SCL-10 mean score was found to be 1.45 (SD = .92). The internal consistency of the SCL-10 was found to equal to .88, and its test-retest reliability was .59 after 4 months. A principal component factor analysis yielded only one factor. In an outpatient community sample, the SCL-10 was found to be highly correlated with the Psychiatric Symptom Inventory, a 29-item scale designed to tap psychological distress (Sabourin, Veronneau, & Gendreau, 1988). The SCL-10 thus represents a reliable and valid measure of psychological distress that offers the advantage of being shorter than the SCL-90. Representing mean constituent item score, the SCL-10 can range from 0 to 4. In this sample, the subjects presented almost-absent to severe psychological distress on the SCL-10 (M = 1.49; SD = .75; range 0.1-3.5). Psychotherapy Alliance. The alliance in psychotherapy
was assessed by the California Psychotherapy Alliance Scales-Patient Version (CALPAS-P; Gaston, 1991). This is a 24-item version filled out by patients after completion of a therapy session. A total score can be derived, as well as four scales scores. An internal consistency coefficient of .83 was found for the total CALPAS-P scale, the score to be reported in this study. Patients’, therapists’, and treatment characteristics were not found to be related to the CALPAS-P, although it was related to interpersonal difficulties and symptomatology. Finally, CALPAS-P was not found to be associated with patients’ tendencies to report socially acceptable answers (Gaston, 1991). A CALPAS-P score ranges from 1 to 7, representing the mean of item scores. In this study, the total CALPAS-P score was found to equal 5.96 (SD = .57; range 3.88-6.88), indicating relatively good alliances. Therapists
and Treatment
Of the 21 therapists who participated in the study, 19 were psychologists, 1 was a social worker, and 1 was a psychiatrist/psychoanalyst. Their mean age was 36.4 years (SD = 5.7). There were 15 female therapists (75%)
229
and 5 male therapists (25%). They have practiced individual psychotherapy for an average of 9.6 years (SD = 4.8), and they have been supervised for an average of 5.2 years (SD = 1.9). At the time of the study, they were seeing 4-25 patients in psychotherapy. When asked to describe their theoretical and clinical orientation, eight (41%) said that their main orientation was psychodynamic, eight (41%) characterized it as eclectic (humanistic and psychodynamic), three (15%) described it as behavioral, and one (4070)reported to use a systemic approach. As a result, the following numbers of subjects were treated in each treatment modality: 48 in dynamic psychotherapy; 44 in humanistic/dynamic psychotherapy; and 23 in behavioral/cognitive therapy. Procedure
Psychotherapists in private practice were recruited to participate in this study, which involves an examination of the relation between client satisfaction and social desirability, along with the validity of an alliance measure. The psychotherapists were told that their participation consisted of asking each of their patients if they would be interested in answering questionnaires about their psychotherapy. Of the 25 psychotherapists who were contacted, 21 agreed to participate in the study. They communicated to their patients that the information would be totally confidential since the patient names would not appear on the questionnaires. Patients who agreed to participate in the study were provided with an envelope containing questionnaires as well as a prestamped return envelope. Along with the questionnaires, a letter was provided to the patients that explained the purpose of the study, the time required to complete the questionnaires (15-30 minutes), and the confidentiality procedure. Patients were asked explicitly not to write their names on the questionnaires. A total of 225 questionnaires were distributed to the participating therapists. Since 147 patients participated in the study, the response rate equals 65%, which is comparable to the return rate of 68% reported by Tracey and Kokotovic (1989) and of 66% obtained by Sabourin et al. (1989). The reasons motivating the nonresponders, however, remain unknown, leaving the biases introduced by this recruitment strategy masked.
RESULTS In this study, the mean CSQ-8 score was 28.74 (SD = 3.61), with a range of 16 to 32. Cronbach’s coefficient of internal consistency equaled .93, with correlations of item with total (of remaining items) varying from .68 to .81. A principal component factor analysis of the CSQ-8 items yielded one factor associated with an eigen value of 5.39, which accounted for 67.3% of the variance. The loadings of the eight items ranged from .75 to .87,
indicating that all items contributed to this single factor. The factor associated with the next highest eigen value (.64) accounted for 7.9% of the variance only, suggesting that it did not contribute substantially to explain the variance of CSQ-8 scores. The association of CSQ-8 scores with criterion variables was performed using Pearson correlation or analysis of variance (including t-test) whenever appropriate. Given the number of analyses performed, the alpha
230
LOUISE GASTON
and
level of .05 was reduced to .005 by the Bonferroni adjustment criterion to counter the family-wise error rate. As shown in Table 2, no significant correlation was observed between CSQ-8 scores and patients’ age, number of years of education, and yearly income. No significant level of difference was revealed between CSQ-8 scores reported by male or female clients, as well as between CSQ-8 scores reported with respect to male or female therapists. In addition, the three psychotherapy modalities were not found to yield significant CSQ-8 level differences. A significant correlation was observed, however, between CSQ-8 scores of the number of weeks in psychotherapy, with only 5% of the variance of the criterion variable accounted for. The association between CSQ-8 scores and SCL-10 scores was marginally significant, with a shared variance of 5%. In contrast, a markedly significant relation was observed between CSQ-8 scores and CALPAS-P alliance scores (see Table 2). The CSQ-8 scores were not found to be significantly related to the MCSD scores, revealing that client satisfaction was not substantially related to estimates of social desirability (see Table 2). Although no direct association between client satisfaction and social desirability was observed, their relation could be moderated by predictors such as sex of patient or therapist, treatment length, or treatment modality. To test the contribution of these potentially moderating variables, four hierarchical regression analyses were conducted with CSQ-8 scores as the criterion. Each analysis was performed with the predictor entered at the first step, social desirability scores at the second step, and with their
ASSOCIATION
TABLE 2 OF CSQ-8 AND CRITERION-RELATED
Criterion
Test
Age
r(140)
Education Income Psychotherapy length Distress (XL- 10) Social desfrability (MCSD) Alliance (CALPAS-P) Subject gender Therapist gender Psychotherapy modality
r(137)
p Value
r(146)
03 02 = .07 = .24 = -.22
r(146)
= -
r(146)
=
r(130) r(143)
SCORES
=
= -
05 .65
f(141) = -98 t(l29) = 1.02 F(2,l 12) = -81
.76
a0 45 ,004 006 52 .ooo 33 .31 45
CSQ-8 = Clrent Satisfaction Questionaire-8 Item (Larsen, et al.. 1979); SCL-IO = Symptoms ChecklIst-10 Item (Nguyen et al.. 1983), MSCD = Marlow-Crowne Social Desirability Scale (Crowne & Marlowe, 1960); CALPAS-P = California Psychotherapy Alltance Scales (Gaston, 1991)
interaction terms at the third step. Because four analyses were conducted, the alpha level of .05 was reduced to .0125 using the Bonferroni adjustment criteria. Neither main effects nor interaction terms were found to be statistically significant, In the interest of comparing these results with those associated with symptomatology, we computed the correlation between SCL-10 and MCSD scores. A significant but weak association was obtained (r( 146) = - .21, p < .Ol), indicating that a greater tendency toward socially desirable responding yielded self-reports of lower symptomatology.
DISCUSSION As expected, CSQ-8 scores were not found to be related to estimates of social desirability, either directly or through the mediation of selected variables. These results are especially striking given that, in contrast, symptomatology self-reports, a sine qua non outcome variable of psychotherapy, were found to be marginally but significantly associated with socially desirable responding. Furthermore, no significant association was observed between CSQ-8 scores and clients’ sociodemographic variables. A weak but significant relation was obtained between satisfaction and the number of weeks of psychotherapy, while treatment length did not interact with estimates of social desirability to predict satisfaction. The very small correlation observed between satisfaction scores and social desirability estimates, -.05, was lower than those reported by Sabourin et al. (1989), .27 and .19. This variety of findings could be due to the different measures of socia1 desirability employed: Marlowe Crowne Social Desirability Scale in this study; and Self-Deception Questionnaire and Other-Deception
Questionnaires in the Sabourin et al.‘s study. The lack of significant correlation between satisfaction and social desirability in this study also could be attributed to restricted variance associated with CSQ-8 scores. The standard deviation of CSQ-8 scores of the present sample equaled 3.6, while that obtained by Sabourin et al. (1989) was 5.9. Using the test for difference between variances of two independent samples (Bruning & Kintz, 1968), the variance of this study was found to be statistically smaller (F’(80, 146) = 2.64, p < .Ol) than that reported by Sabourin et al. (1989). Therefore, the correlation between CSQ-8 and MCSD scores could be due in part to the restricted variance associated with CSQ-8 scores. In contrast with this argument, CSQ-8 scores have been found to be highly related to a convergentvalidity criterion, that is, a measure of alliance in psychotherapy. Although a restricted range necessarily lowers the maximum I’, these CSQ-8 scores varied sufficiently to yield substantial associations with selected criterion variables. Moreover, scores of both satisfaction and social desirability were drawn from the client’s
Client Satisfaction and Desirability perspective, a strategy which was likely to increase the correlation between variables (Orlinsky & Howard, 1986). No difference in levels of CSQ-8 scores was found across clients treated in dynamic, eclectic (humanistic and dynamic) or cognitive/behavioral therapy. This observed lack of significant difference suggests that the measurement of client satisfaction could be useful to assess the outcome of diverse psychotherapy approaches, a methodological strategy suggested by Kazdin (1986). The present findings thus provide some support to the
231
discriminant validity of client satisfaction measures, which could be viewed as relatively independent from the tendency to provide socially desirable impressions. Client satisfaction also appears to be a nonredundant outcome variable of psychotherapy because a small and marginally significant association was observed between satisfaction and symptomatology scores. The CSQ-8 could thus be used to assess client satisfaction in an outpatient psychotherapy setting in which clients are mostly adults with at least a moderate level of education and an average yearly income.
REFERENCES ATTKISSON, Questionnaire: psychotherapy 233-237. BRUNING, of statistics.
C.C., & ZWICK, R. (1982). The Client Satisfaction Properties and correlations with service utilization and outcome. Evaluation and Program Planning, 5,
J.L., & KINTZ, B.L. (1968). Compufational Chicago: Scott, Foreman and Company.
handbook
CROWNE, D.P., & MARLOWE, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354.
LEBOW, J.L. (1983). Research assessing consumer satisfaction with mental health treatment: A review of findings. Evaluation and Program Planning, 6, 21 l-236. NGUYEN, T.D., ATTKISSON, C.C., & STEGNER, B.L. (1983). Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6, 299-3 14.
motive:
ORLINSKY, D.E., &HOWARD, K.I. (1986). Process and outcome in psychotherapy. In S.L. Garfield and A.E. Bergin (Eds.), Handbook of Psychotherapy and Behavior Change (pp. 3 1 l-384). New York: Wiley.
DEROGATIS, L., LIPMAN, R., & COVI, L. (1973). The SCL-90: An outpatient rating scale (Preliminary report). Psychopharmacology Bulletin, 9, 13-28.
SABOURIN, S., BOURGEOIS, L., GENDREAU, P., & MORVAL, M. (1989). Self-deception, impression management, and consumer satisfaction with mental health treatment. Psychological Assessment: A Journal of Consulting and Clinical Psychology, I, 126- 129.
CROWNE, D.P., & MARLOWE, Studies in evaluative dependence.
D. (1964). The approval New York: Wiley.
GASTON. L. (1991). Reliability and criterion-related validity of the California Psychotherapy Alliance Scales-Patient version. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 68-74. HOFFMAN, N.G., &OVERALL, P.B. (1978). Factor structure of the SCL-90 in a psychiatric population. Journal of Personality and Social Psychology, 37, 147-169. KAZDIN, A.E. (1986). Comparative outcome studies of psychotherapy: Methodological issues and strategies. Journal of Consultingand Clinical Psychology, 54, 95-105. KOKOTOVIC, A.M., & TRACEY, T.J. (1987). Premature termination at a university counseling center. Journal of CounselingPsychology, 34, 80-82. LAMBERT, M.J., SHAPIRO, D.M., & BERGIN, A.E. (1986). The effects of psychotherapy. In S.L. Garfield and A.E. Bergin (Eds.), Handbook of psychotherapy and behavior change (3rd ed., pp. 157212). New York: Wiley. LARSEN, D.L., ATTKISSON, C.C., HARGREAVES, W.A., & NGUYEN, T.D. (1979). Assessment of client/patient satisfaction: Development of a general scale. Evaluation and Program Planning, 2, 197-207.
SABOURIN, S., GENDREAU, P., & FRENETTE, L. (1987). Le niveau de satisfaction des cas d’abandon dans un service universitaire de psychologie [Client satisfaction and dropout cases in a university psychology clinic]. Revue Canadienne des Sciences du Comportement, 19, 314-323. SABOURIN, S., LAFERRIERE, N., SICURO, F., COALLIER, J.C., COURNOYER, L.G., & GENDREAU, P. (1989). Social desirability, psychological distress, and consumer satisfaction with mental health treatment. Journal of Counseling Psychology, 36, 352-356. SABOURIN, psychometric
S., VERONNEAU, F., & GENDREAU, P. (1988). The qualities of the SCL-10. Unpublished data.
STRICKLAND, B.R., & CROWNE, D.P. (1963). Need for approval and premature termination of psychotherapy. Journal of Consulting Psychology, 27, 95-101. TANAKA-MATSUMI, J., & KAMEOKA, V.A. (1986). Reliabilities and concurrent validities of popular self-report measures of depression, anxiety, and social desirability. Journal of Consulting and Clinical Psychology, 54, 328-333. TRACEY, T.J., & KOKOTOVIC, A.M. (1989). Factor structure of the Working Alliance Inventory. PsychologicalAssessment: A Journal of Consulting and Clinical Psychology, I, 207-210.