Psychometric evaluation of the symptom checklist (SCL-90) in psychiatric inpatients

Person. indiuid. DI@ Vol. 14, No. 6, pp. 751-756, 1993

0191~8869/93

Printedin Great Britain. All rights reserved

$6.00 + 0.00

Copyright 0 1993 PergamonPress Ltd

PSYCHOMETRIC EVALUATION OF THE SYMPTOM CHECKLIST (SCL-90) IN PSYCHIATRIC INPATIENTS A. Sinai Centre,

Jewish

Mental

HAFKENSCHEID

Health

Services,

Amersfoort,

The Netherlands

(Received 29 July 1992) Summary-The factorial validity, temporal stability and discriminatory power of the Symptom Checklist (SCL-90) were investigated in a heterogeneous sample of short-stay psychiatric inpatients (n = 437). Results from a confirmatory factorial approach suggested a reduced dimensionality for the SCL-90 rather than supporting the 10 or 9 o priori symptom dimensions. From exploratory factor analyses 4 clinically interpretable and moderately intercorrelated dimensions emerged: depression, hostile-suspiciousness, somatization and phobic anxiety. Internal consistency and temporal stability coefficients of these 4 subscales and of the global scale were satisfactory. SCL-90 discriminated poorly between different clinical groups. However, the scale quite successfully discriminated patients from healthy subjects from the general population. It is concluded that SCL-90 might be a valuable instrument for the detection of potential psychiatric cases in the general population.

INTRODUCTION

Self-report symptom inventories have been recognized by both clinicians and researchers as convenient, economical methods of gathering information about patients’ mental states. Rather than clinician-based rating scales, such inventories directly reflect patients’ unique experiences of their distress. Pragmatically put, the patient is generally the most available source of information. In the case of psychiatric inpatients, self-report symptom inventories may describe patients’ target complaints at the time of admission to the hospital, also, repeated administration of the inventory may inform clinicians or researchers about symptom change over the course of hospitalization as experienced by patients themselves. The Symptom Checklist (SCL-90; Derogatis, Lipman & Covi, 1973; Derogatis, Rickels & Rock, 1976; Derogatis & Cleary 1977a,b) is a go-item self-report inventory that originated in the U.S. in the early 1970s. Since then, the SCL-90 has been incorporated for classification or evaluation purposes in many clinical studies (e.g. Turner, McGovern & Sandrock, 1983; O’Donnel, DeSoto & Reynolds, 1984; Wegner, Rabiner & Kane, 1985; Frost, Sher & Geen, 1986; Rosenbaum, Horowitz & Wilner, 1986; Perse, Greist, Jefferson, Rosenfeld & Dar, 1987; Turner & McGovern, 1987; Dormaar, Dijkman & DeVries, 1988; Frank, Carpenter & Kuper, 1988; Boleloucky & Polach, 1989). The frequent use of SCL-90 has undoubtedly been advanced by its convenient checklist format and by its purported multidimensionality. Nine (Derogatis et al., 1973) or ten (Lipman, Covi & Shapiro, 1979) clinically identifiable dimensions (subscales) of symptomatology have been postulated: somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, psychoticism and sleep difficulty. The hypothesized dimensional structure of SCL-90 has been confirmed to a large extent by Derogatis and Cleary (1977a) and by Lipman et al. (1979) in samples of psychiatric outpatients. However, other factor analytic investigations (for a review see: Cyr, McKenna-Foley & Peacock, 1985) cast doubt on the stability of SCL-90 factors across samples. Only a few of the postulated dimensions were found common to different factor analytic studies of SCL-90 in psychiatric samples (e.g. Hoffmann & Overall, 1978; Clark & Friedman, 1983; Holcomb, Adams & Ponder, 1983; Arrindell & Ettema, 1986; Mazmanian, Mendonca, Holden & Dufton, 1987; Brophy, Norvell 8z Kiluk, 1988; Cyr, Doxey & Vigna, 1988). The poor match of SCL-90 factors across studies may partly be attributed to the emphasis on exploratory principal component factoring methods rather than confirmatory approaches. Since principal component analysis maximizes explained variance for a given sample data set idiosyncrasies are expressed within the factors, therefore reducing the likelihood of replicating a given 751

152

A. HAFKENSCHEID

factor structure (Shutty, Degood & Schwartz, 1986). It should be pointed out that most factor analytic studies of SCL-90 in psychiatric samples used outpatients. Not only the severity but also the clustering of symptoms may be quite different for patients in need of hospitalization contrasted to patients treated in outpatient settings. In the present study SCL-90 is evaluated psychometrically in a psychiatric inpatient sample. A confirmatory factorial approach is pursued to verify the ten (nine) original SCL-90 dimensions. Moreover, temporal stability and discriminatory power are assessed. MATERIALS

AND

METHODS

Subjects The Ss were 437 short-stay psychiatric inpatients; 382 of whom completed the SCL-90 as part of the admission procedure, 46 during the 1st month, 5 during the 2nd month and 4 in the 3rd month of their stay in the hospital. Diagnoses according to ICD-9 were: organic psychoses (ICD 290-294), 9; schizophrenic psychoses (295) 25; affective psychoses (296), 67; paranoid state (297) 5, other non-organic psychoses (298), 17; neurotic disorders (300), 246; personality disorders (301), 11; alcohol dependence syndrome (303), 12; depressive disorder (31 l), 10; other (305, 1; 307, 3; 309, 4; 312, 2), 10; missing data, 25. Sex: 36% males, 59% females, 5% missing data. Age: M = 38.7, SD = 14.1 (range: 14-38). All patients were considered testable by their treating psychiatrist or clinical psychologist. A control group of 269 psychiatric outpatients, completing SCL-90 as part of the intake screening for (behaviour-oriented) psychotherapy (50%), social-psychiatric support (37%) or partial day hospitalization (2 days a week: 13%). Sex: 43.5% males, 56.5% females. Age: M = 35.7, SD = 10.1 (range: 16-72). Inpatient and outpatient samples are comparable in sex distribution (x2 = 2.18 c( = 0.10, df = 1). As a group, outpatients were younger, (t = 3.09, P = 0.002, two-tailed, df = 642) a difference of relatively small effect size (Cohen, 1977). Statistical analyses As a confirmatory description-oriented (in contrast to statistically-oriented) factor analytic approach, the perfect congruent weights (PCW) method.* (Ten Berge, 1986), closely related to the multiple group method (Nunnally, 1978; Gorsuch, 1983), was employed to ,verify the 10 dimensions, postulated by Lipman et al. (1979), in the Dutch inpatient group. The hypothesized lo-dimension structure only differs from 9 postulated dimensions by the addition of a sleep difficulty subscale. Therefore, by cross-validating the 10 postulated dimensions the 9-dimension structure was evaluated as well. Hence, a binary 90 x 10 target matrix was defined by assigning all items (symptoms) theoretically comprising a particular dimension (e.g. Somatization) a (standard score) weight of 1 for that dimension and a 0 for all other 9 dimensions. The 4 items, not postulated to have univocal weights on any of the dimensions (poor appetite, overeating, thoughts of death or dying, feelings of guilt) were assigned a zero-weight for all 10 dimensions. Thus, apart from these 4 items, each row of the target matrix had only one non-zero entry; non-zero weights within a column referring to those items hypothetically composing that particular dimension. Allowing for the eventuality of a poor theoretical-empirical match, exploratory principal component analyses with varimax rotation of 3-9 factors were also performed. The discriminatory power of the SCL-90 was tested with t-test for independent samples, comparing (a) inpatient vs outpatient group and (b) the subgroup of inpatients diagnosed within the psychotic range (ICD 290-299) vs the subgroup of inpatients, diagnosed within the neurotic range (ICD 300-316). The magnitude of differences was expressed by the effect size index, proposed by Cohen (1977). The temporal stability (test-retest reliability) of SCL-90 scores was assessed after repeated administration of the scale in a group of 55 inpatients, completing the SCL-90 twice within a time-interval of 3-7 days. Temporal stability was established by Pearson correlations. *The PCW method does not examine whether factors are stable across samples, congruence is always achieved by using oblique rotation.

but how strong they are recoverable.

Perfect

Psychometric Table 1. Intercorrelations,

Somatization (Som) Obsessive-compulsive (Obs) Interpersonal sensitivity (Int) Depression (Dep) Anxiety (Anx) Anger-hostility (Ang) Phobic anxiety (Pho) Psychoticism (Psy) Paranoid ideation (Par) Sleep difficulty (Sk) Cronbach’s alpha Mean interitem correlation Global

Scale: Cronbach’s

Dimensional

evaluation

of the SCL-90

753

Cronbach’s alphas and mean interitem correlations on the XL-90

Sam

Obs

Int

1.00

0.49

0.42 0.61 1.00

I .oo

&P

Anx

0.58 0.74 0.63

0.74 0.58 0.54 0.69 1.oo

0.88 0.43

I .oo

subs&es

Ang

Pho

PSY

Par

Sk

0.44 0.44 0.51 0.51 0.41 1.00

0.51 0.30 0.36 0.33 0.63 0.20 1.00

0.49 0.66 0.68 0.71 0.63 0.51 0.36 1.00

0.36 0.53 0.76 0.62 0.44 0.58 0.14 0.71 1.00

0.37 0.44 0.31 0.43 0.33 0.22 0.12 0.38 0.31

0.76 0.35

0.89 0.54

0.76 0.24

0.80 0.40

0.79 0.56

I .oo 0.87 0.36

0.81 0.30

alpha = 0.97; mean interitem

0.84 0.37

0.89 0.39

correlation

= 0.24.

structure

Cross-validation of the 10 XL-90 subscales. The multiple-group factor structure contains the correlation of each item with all subscales as represented by the factors (Gorsuch, 1983). The structure matrix obtained from applying the hypothetical binary weights upon the present data was inspected for the correlation pattern of each item across the 10 hypothesized factors. Of course, each item was predicted to correlate highest with the factor to which it was assigned in advance, after correction for the part-whole effect (Gorsuch, 1983)*. As more stringent criteria (a) only items that correlated substantially (r > 0.40) with their ‘own’ factor were considered confirmatory of the hypothetical factorial composition, (b) provided that the next largest correlation with one of the other factors was substantially lower (a correlational difference of at least 0.20). Finally, as recommended by Nunnally (1978) interitem correlations ‘within’ factors were considered, as well as subscale intercorrelations. Out of 90 items 72 correlate highest with their ‘own’ factor. However, only 8 items fulfil both more stringent criteria. If criterion (b) is relaxed to a correlational difference of at least 0.10 the number of items fulfilling both criteria is increased to 35. Mean interitem correlations as well as subscale intercorrelations are shown in Table 1. Mean interitem correlations appear to be rather low (r < 0.40) for 6 of the 10 subscales. Of 45 subscale intercorrelations 5 are substantial (r 2 0.70), indicating that some XL-90 subscales (e.g. somatization and anxiety) cannot be distinguished very well from each other. All in all, poor empirical support is found for the theoretical composition of 10 (or 9) SCL-90 dimensions. To check whether more confirmatory results could be obtained for the outpatient control samples, the described procedure was repeated in this sample, applying the same criteria as formulated before. In the outpatient sample, 72 out of 90 items correlate highest with their own factor, of which 9 fulfil both formulated criteria. By defining criterion (b) less conservatively (to a correlational difference of 0.10) 29 items fulfil both criteria. Thus, the 10 (or 9) a priori SCL-90 dimensions are no more convincingly confirmed in the outpatient sample. Exploratory factor analyses. Since strong support for the original SCL-90 dimensions was not apparent, principal component analysis with 3-9 factors extracted and varimax rotation were performed additionally to explore a more limited (or differently composed) number of SCL-90 dimensions. All first 9 unrotated factors had eigenvalues exceeding 1. These factors together explain 50% of the variance. The first factor accounts for 26% of the variance, the next largest factor for 6.2%. Loading matrices of varimax rotated factor solutions were evaluated as follows: (a) only items having a highest factor loading of at least 0.40 on one of the factors were selected (comparable to Holcomb et al., 1983; Brophy et al., 1988; Cyr et al., 1988); (b) provided that the next largest loading on one of the other factors was at least 0.20 lower; and (c) at least 4 items within each factor should fulfil both criteria for that factor solution to be considered further. Only the 3- and 4-factor solutions met all 3 criteria. The 4-factor solution covering 50 items was chosen as the best *Item-total

correlations

are spuriously

raised

by the artifact

that the items are part of their ‘own’ factor.

754

A. HAFKENSCHEID

clinically interpretable, the 3-factor solution grouping somewhat heterogeneous items. The 4-factor solution accounts for 39.1% of the total variance. The item composition of the 4 remaining subscales is presented in Table 2. Internal consistency coefficients (Cronbach’s alpha) of the four subscales were high (as well as the global scale alpha), as Table 2 shows. For an indication of the homogeneity of the total scale (corrected) item-total correlations were inspected. All correlations were positive, with only 1 low correlation (r = 0.14) and 68 correlations equal to or exceeding 0.40. Intercorrelations of the 4 subscales (based on summing unweighted item scores) were small to moderate; the correlations between depression on the one hand and respectively hostile-suspiciousness and somatization on the other were the largest (r = 0.57 and 0.56, respectively). Further analyses were based on these 4 subscales. Test-retest

reliabilities

Temporal stability coefficients for the 4 subscales are also included in Table 2. With a somewhat lower coefficient found for the hostile-suspiciousness subscale, test-retest reliabilities turned out to be satisfactory for subscales as well as for the total scale. For all 4 subscales as well as the global scale significantly lower scores are observed on retest when pretest and retest means are compared (paired t; 2.37 < t < 4.15, CI= 0.10, df = 54, two-tailed). For the global scale this difference is of medium effect size (d = 0.56). Discriminatory

power

Preliminary analysis: demographic variables (sex and age) related to SCL-90 scores. Female patients appear to score higher on the phobic anxiety subscale (t = 3.69; c( = 0.01, df = 413, two-tailed), a difference of small effect size. Age and SCL-90 (sub)scale scores were found to be uncorrelated (- 0.03 < r < 0.07). Diagnosis. Patients diagnosed within the psychotic range (ICD 290-299) were predicted to report more distress on the hostile-suspiciousness subscale compared with neurotic patients (ICD 300-316). For the other subscales as well as for the total scale no predictions in this sense were made. Except for the phobic anxiety subscale-neurotics reporting more distress-(t = 5.29, medium effect size) neither predicted nor unpredicted differences occurred (t-test for independent samples, CI= 0.01, df = 410, two-tailed; hostile-suspiciousness: u = 0.01, one-tailed). Patient status. Inpatients were predicted to report more distress than outpatients on subscale as well as global scale level (a = 0.01, df = 704, one-tailed). All predictions are confirmed (2.71 < t < 6.21). However, only the differences for depression (d = 0.48) and global distress (d = 0.41) tend to a medium size effect. Comparison with ‘normal’ population. SCL-90 global scores of both inpatient and outpatient samples were classified according to (sex-specific) norms for healthy Ss as described by Arrindell and Ettema (1986). Table 2. Items comprising 4 SCL-90 subscales based on varimax rotated 4-factor solution 1. Depression: 3. unpleasant thoughts, 14. low in energy, 15. suicidal thoughts, 29. lonely, 30. blue, 31. worry, 32. no interest, 46. indecisiveness, 51. blank mind, 54. hopelessness, 55. poor concentration, 71. everything an effort, 79. worthlessness, 90. mind problems. (alpha = 0.91, rlI = 0.73). 2. Hostile-sospicioosness: 6. critical, 7. someone controls thoughts, 8. blaming others, 18. no trust, 21. shy with opposite sex, 24. temper outbursts, 35. people aware of thoughts, 36. no sympathy, 37. disliked, 43. watched, 61. uneasy when watched or talked about, 62. thoughts not your own, 68. ideas not shared, 74. frequent arguments, 76. no credit given, 81. tantrums or shouting, 83. taken advantage of, 84. bothering about sex, 85. should be punished. (alpha = 0.89, r12 = 0.68) 3. somatization: 1. headaches, 4. faintness, 12. chestpains, 27. low back pains, 39. heart pounding, 40. nausea, 42. muscle soreness, 44. trouble falling asleep, 49. hot-cold spells, 52. numbness, 58. heavy feeling, 66. restless sleep. (alpha = 0.85, r,2 = 0.78) 4. Phobic anxiety: 13. afraid of open spaces, 25. afraid to go out, 47. afraid to travel on buses, etc, 50. avoiding places or activities, 70. uneasy in crowds, 82. fearful in public. (alpha = 0.90, r,* = 0.91) item abbreviations

and item numbers; alpha: Cronbrach’ alpha, r,> = test-retest reliabilities (in parentheses). Global scale: r12= 0.79.

Psychometric evaluation of the SCL-90

755

In the inpatient sample 91.3% of the global SCL-90 scores are above the 80th percentile of the ‘healthy Ss from the general population’ distribution; 70.8% even score above the 95th percentile of this distribution. For the outpatient sample: 87.4% of the outpatients scored above the 80th percentile, 59.5% above the 95th percentile. The ratios of ‘false-negatives’ (i.e. inpatient scores below the 65th percentile of general population distribution) are low for both the inpatient (4.3%) and outpatient (6%) samples. Thus, although the clinical samples cannot be discriminated with the SCL-90 convincingly, both samples are clearly distinguishable from the general population with the scale. DISCUSSION

The 10 or 9 symptom dimensions supposed to underlie SCL-90 were not recovered in the present study, which subjected SCL-90 data of psychiatric inpatients to a confirmatory factor analysis. Similar to findings of previous (exploratory) factor analytic studies in psychiatric (outpatient) samples, a reduced multidimensionality was found. Thus, differential clinical interpretations of the 10 or 9 original SCL-90 symptom dimensions appear to be unwarranted, regardless of patient status (psychiatric inpatients or outpatients). Since SCL-90 was constructed primarily to tap a wide range of distinct symptom clusters, this conclusion is rather disappointing. Based on criteria of evaluating results of principal component analysis [e.g. the proportion of variance explained by the first unrotated factor; the number of items in (varimax) rotated factor structures, which have substantial loadings on more than one factor], most previous factor analytic studies on SCL-90 concluded that this self-report symptom inventory would best be considered a measure of global distress, i.e. a unidimensional measure. The dominance of a general distress component as represented by SCL-90 global scores is supported in the present study when the same criteria are applied. The general factor solutions found in this study and previous studies may be favoured by the heterogeneous nature of the inpatient and outpatient samples. However, most unselected clinical populations in which the SCL-90 is used for multidimensional assessment are heterogeneous with respect to symptoms and symptom combinations. Yet, interpretation of a limited number of SCL-90 dimensions turned out to be justified. Exploratory factor analyses yielded 4 homogeneous, clinically interpretable and moderately intercorrelated symptom dimensions in this study: depression, hostile-suspiciousness, somatization and phobic anxiety. These factors also emerged in previous research (although item content of these factors was somewhat variable across studies) and represent the most consistent dimensions in SCL-90. Interpretation of these dimensions might be useful to specify the relative contribution of each of these symptom clusters to the overall distress scores. SCL-90 ratings were temporally stable for at least 3 of 4 subscales and also for the global scale. Since test-retest reliabilities were high, the significantly lower scores found on retest are readily explained by statistical regression (i.e. the fact that extreme scores in a distribution will tend to move toward the mean as a function of repeated testing). Of course, ‘real’ change might be advanced as a rival explanation, the lower scores on retest (compared with pretest) reflecting actual (therapeutic) change rather than measurement error. However, for at least 2 reasons this ‘actual change construction’ is disputable. First, the short period of time (3-7 days) between the two administrations of the scale cover the whole reference period (‘the past 7 days including today’) of SCL-90 ratings. Second, the high stability coefficients denote that the relative position of SCL-90 is largely unchanged from pretest to retest. Thus, most patients show about the same degree of change in the days subsequent to the pretest, rather suggesting a social desirability phenomenon. When SCL-90 is used as a (pharmaco- or psycho) therapy outcome instrument, change scores should be corrected for this retest-effect. Without such a correction changes in SCL-90 scores might give too flattering a picture (Koeter, Ormel & Van den Brink, 1988). Only weak support was found for the discriminatory power of SCL-90, when different clinical groups (inpatients diagnosed within the psychotic range vs inpatients admitted with neurotic problems; psychiatric inpatients vs psychiatric outpatients) were compared. In contrast, SCL-90 discriminated quite well between both inpatient and outpatient samples on the one hand and healthy Ss from the general population on

A. HAFKENSCHEID

156

the other. Therefore, SCL-90 appears to be a promising screening instrument for the detection of potential psychiatric cases in the general population. Acknowledgemenfs-The following institutions contributed to the data collection of this study: psychiatric units of general hospitals Bleulandziekenhuis, Gouda (Mathieu Vosmeer); Nij Smellinghe, Drachten (John Weymar Schultz); Streekziekenhuis Waterland, Purmerend (Ruud Kol); observation units of psychiatric hospitals Jan Wier, Tilburg (Arjaan van Opstal, Coos Pels Rijcken); Licht en Kracht, Assen (Nice Knook); Sinai Centre, Amersfoort; the inpatient behaviour therapy unit of psychiatric hospital Joris Gasthuis, Delft (Maurits Kwee); the outpatient clinic of Licht en Kracht (Ton Marinkelle); the partial day hospital of Sinai Centre; the Jewish Ambulatory Mental Health Services, Amsterdam (Jitschak Elzas). The colleagues mentioned in parentheses made the data for this study available. The author is much indebted to Jan van der Ende for invaluable psychometric advice and for performing PCW calculations. Thanks are further due to Willem Arrindell and Paul Emmelkamp for their thoughtful comments. Completion of this paper was made possible by support of the Dutch Jewish Mental Health Services (Director: Johan Lansen, MD).

REFERENCES Arrindell, W. A. & Ettema, J. H. M. (1986). XL-90: handleiding bgeen mtdtidimensioneiepsychopathologie-indicator. Lisse: Swets & Zeitlinger. BolelouckC, Z. & Polach, J. (1989). Discriminant analysis of items in the SCL-90 questionnaire in borderline natients and controls-a methodological study. Activitas Nervosa Superior, 31, 1222124. . Brophv, Ch. J.. Norvell. N. K. & Kiluk. D. J. (1988). An examination of the factor structure and convergent and discriminant validity of the SCL-90R in’an outpatient clinic population. Journal of Personality Assessment, _5.?,?34-340. Clark, A. & Friedman, M. J. (1983). Factor structure and discriminant validity of the SCL-90 in a veteran psychiatric population. Journal of Personality Assessment, 47, 396404. Cohen, J. (1977). Statist&al power analysis for the behavioral sciences. New York: Academic Press. Cvr. J. J., Doxev. N. C. S. & Viana. C. M. (1988). Factorial comnosition of the SCL-90-R. Journal ofSocial Behavior and . Personality, 3, 245-252. Cyr, J. J., McKenna-Foley, J. M. & Peacock, E. (1985). Factor structure of the SCL-90-R: is there one? Journal of Personality Assessment, 49, 51 l-578. Derogatis, L. R. & Cleary, P. A. (1977a). Confirmation of the dimensional structure of the SCL-90: a study in construct validation. Journal of Clinical Psychology, 33, 981-989. Derogatis, L. R. & Cleary, P. A. (1977b). Factorial invariance across gender for the primary symptom dimensions of the SCL-90. British Journal of Social and Clinical Psychology, 16, 347-356. _ _ _ _ Derogatis. L. R., Lipman, R. S. SC Covi. L. (1973). SCL-90. an outpatient nsvchiatric ratine scale-oreliminarv _ renort. . I 1 Psychopharmacology Bulletin, 9, 13-28. ‘ Derogatis, L. R., Rickels, K. & Rock, A. F. (1976). The SCL-90 and the MMPI: a step in the validation of a new self-report scale. British Journal of Psychiatry, 128, 280-289. Dormaar, M., Dijkman, Ch. I. M. & De Vries, M. W. (1988). Psychotherapeutic outcome: measuring reliable clinical change. Psychotherapy and Psychosomatics, 50, 95-101. Frank, E., Carpenter, L. L. & Kuper, D. J. (1988). Sex differences in recurrent depression: are there any that are significant? American Journal of Psychiatry, 145, 4145. Frost, R. O., Sher, K. J. & Geen, T. H. (1986). Psychopathology and personality characteristics of non-clinical compulsive checkers. Behaviour Research and Therapy, 24, 133-143. Gorsuch, R. L. (1983). Facror analysis. Hillsdale, NJ: Lawrence Erlbaum. Hoffman, N. G. & Overall, P. B. (1978). Factor structure of the SCL-90 in a psychiatric population. Journal of Consulting and Clinical Psychology, 46, 1187-l 191. Holcomb, W. R., Adams, N. A. & Ponder, H. M. (1983). Factor structure of the Symptom checklist-90 with acute psychiatric inpatients. Journal of Consulting and Clinical Psychology, 51, 5355538. Koeter, M. W. J., Ormel, J. & Van den Brink, W. (1988). Totaalscore op de SCL-90 als maat voor de ernst van psychopathologic. Nederlands Tijdchrift voor de Psychologie en haar grensgebieden, 43, 38 I-391. Lipman, R. S., Covi, L. & Shapiro, A. K. (1979). The Hopkins symptom checklist: factors derived from the HSCL-90. Journal of Aflective Disorders, I, 9-24. Mazmanian, D., Mendonca, J. D., Holden, R. R. & Dufton, B. (1987). Psychopathology and response styles in the SCL-90 responses of acutely distressed persons. Journal of Psychopathology and Behavioral Assessment, 9, 135-148. Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. O’Donnel, W. E., Desoto, C. B. & Reynolds, D. McQ. (1984). A cognitive deficit subscale of the SCL-90-R. Journal of Clinical Psychology, 40, 241.-246. Perse, T. L., Greist, J. H., Jefferson, J. W., Rosenfeld, R. & Dar, R. (1987). Fluvoxamine treatment of obsessive-compulsive disorder. American Journal of Psychiatry, 144, 1543-1548. Rosenbaum, R. L., Horowitz, M. J. & Wilner, N. (1986). Clinician assessments of patient difficulty. Psychotherapy, 23, 417-425. Shutty, M. S., DeGood, D. E. & Schwartz, D. P. (1986). Psychological dimensions of distress in chronic pain patients: a factor analytic study of Symptom Checklist-90 responses. Journal of Consulting and Clinical Psychology, 54, 836842. Ten Berge, J. M. F. (1986). Rotation to perfect congruence and the cross-validation of component weights across populations. Multivariate Behavioral Research, 21, 4164. Turner, R. M. & McGovern, M. P. (1987). The impact of patient type upon the assessment of the degree of concordance among patients’, clinicians’, and independent evaluators’ ratings of patient level of functioning in the partial-hospital setting. International Journal of Partial Hospitalization, 4, 63-74. Turner, R. M., McGovern, M. & Sandrock, D. (1983). A multiple perspective analysis of schizophrenics’ symptoms and community functioning. American Journal of Community Psychology, 1 I, 593607. Wegner, J. T., Rabiner, Ch. & Kane, J. M. (1985). The cognitive deficit subscale of the SCL-90-R and prediction of outcome in first episode schizophrenics. Journal of Clinical Psychology, 41, 750-752. I