- Email: [email protected]
The control of social desirability in personality inventories: A study using the principal-factor deletion technique
JOURNAL
OF RESEARCH
19, 44-53 (1985)
IN PERSONALITY
The Control of Social Desirability in Personality Inventories: A Study Using the Principal-Factor Deletion Technique PETER BORKENAU University of Bielefeld AND MANFRED
AMELANG
University of Heidelberg The present study sought to investigate Paulhus’ (1981) principal-factor deletion technique (PFDT), a factor analytic method for controlling for socially desirable responding. Adult subjects (N = 330) responded to both forms of the Eysenck Personality Inventory and a 32-item lie scale. Peer ratings of extraversion and neuroticism served as criterion measures. The factor deleted by PFDT more closely reflected neuroticism than impression management. Factors resulting from PFDT were considerably less valid and less content saturated than those emerging from an ordinary factor solution. The implications of these results are discussed from the perspectives of predictive and construct validity. o 1985 Academic press. Inc.
A major difficulty in the development and use of personality scales is the operation of response sets. These include the tendency of subjects to respond in a manner that would be expected of well-adjusted people, thereby presenting themselves in a socially desirable light. To the extent that questionnaires reflect subjects’ intentions to present a socially desirable appearance, the proportion of variance associated with the intended content is lowered, leading to a reduction in the validity of the questionnaire. Therefore, numerous attempts have been made to control for social desirability (SD) in questionnaire and rating data. Recently, Paulhus (1981) has proposed a factor analytic approach for controlling for SD, The research reported in this paper was supported by a grant from Deutsche Forschungsgemeinschaft to Manfred Amelang (AZ.: Am 37/5). We thank Stephen G.’ West for his comments on an earlier version of this manuscript. Requests for reprints should be sent to Peter Borkenau, Abteilung Psychologie, Universitit Bielefeld, Postfach 8640, D4800 Bielefeld, West Germany. 44 0092-6566/85 $3.00 Copyright All rights
B 19S5 by Academic Press, Inc. of reproduction in any form reserved.
FACTOR
DELETION
45
TECHNIQUE
which has been termed the principal-factor deletion technique (PFDT). This method is based on the assumption that in an unrotated factor solution the loadings of the items on the first principal factor are highly correlated with their social desirability scale values (SDSV). Thus, the first principal factor may be assumed to assess socially desirable responding (Edwards & Diers, 1962). Paulhus (1981) suggested that this factor be deleted prior to rotation. Through this procedure, the influence of socially desirable responding should be substantially reduced. SOCIALLY
DESIRABLE
RESPONDING
AND LYING
In order to evaluate methods for controlling socially desirable responding (SDR), it is important to distinguish between (a) the tendency to actually behave in socially desirable ways, which should be mirrored in the item endorsements, and (b) the tendency to respond to test items without regard to actual behavior but only with regard to the social appearance which the person wishes to create (Edwards, 1953; Heilbrun, 1964; Paulhus, 1984; Wiggins, 1959, 1979). This distinction may be most clearly illustrated with regard to measures of emotional stability: Describing oneself as psychopathological necessarily implies reporting undesirable behaviors. Those items that best discriminate clinical from normal groups usually show extreme SDSV. Indeed, as Heilbrun (1964) noted “the dimensions of psychological health and social desirability are in large measure one and the same” (p. 385). From the perspective of predictive validity, the crux of the issue is to identify and eliminate only those components of SDR that result from a misrepresentation of the subject’s true personality. From the viewpoint of construct validity, it is debatable whether dimensions confounded with social desirability (Wiggins, 1979) or dimensions unconfounded with social desirability (Peabody, 1984) are preferable. If this distinction is applied to the PFDT, it becomes evident that the consequences of using PFDT depend on the meaning of the first principal axis: Is it primarily assessing individual differences in socially evaluated behaviors? Or, is it primarily assessing individual differences in impression management? Unfortunately, the usually high correlation of the items’ loadings on this factor with their SDSV does not contribute to the answer as this does not reveal why people differ in their endorsement of socially desirable items. How then can this problem be clarified? First, Crowne and Marlowe (1960) have suggested some guidelines for developing items to separate persons who misrepresent themselves on personality inventories from those who actually behave in a socially desirable way. The content of the lie items they called for should not have psychopathological implications. Accordingly, the correlation of a lie scale with diagnostic scales should be low. Furthermore, the scale should include items that describe (a) behaviors which are not only culturally sanctioned and approved but also improbable of occurrence
46
BORKENAU
AND
AMELANG
and (b) items that describe behaviors which are disapproved but very common within society. A social desirability scale constructed in accordance with these guidelines should be useful in identifying those persons who seek to present a positive impression of themselves on personality questionnaires. Some empirical evidence exists supporting the application of lie scales to increase the validity of scores on other questionnaires. For example, Amelang and Borkenau (1981) found that self-report measures of neuroticism and extraversion of subjects scoring high on a lie scale showed only low correlations with pertinent peer ratings, whereas these correlations were substantially higher for subjects who scored low on the lie scale. This effect was more pronounced with regard to neuroticism as compared to extraversion. Applying this reasoning to the principal factor deletion technique, subjects’ factor scores on the first principal axis should be highly correlated with their lie scores if the first principal factor is assessing impression management. Second, if the first principal factor accounts for variance that is mainly attributable to individual differences in the subjects’ tendency to misrepresent themselves on personality inventories, its deletion should not diminish the validity of the remaining factors. Moreover, their content saturation according to Jackson’s (1970) definition should be increased. If, however, the first principal axis reflects differences in the social desirability of the subjects’ actual behaviors, its elimination can be expected to decrease the validity and content saturation of the remaining factors. In the present study, the PFDT was investigated applying these criteria. METHOD Subjects. Subjects were 330 German adults (145 male, 185 female) from different occupational groups who were paid for their cooperation. Approximately 85% of the participants were between 18 and 30 years old. Procedrcre. The subjects were administered a German version of both Form A and Form B of the Eysenck Personality Inventory (EPI; Eysenck & Eysenck, 1964) and a 32-item lie scale originally developed by Ling (see Amelang & Bartussek, 1970) which had been constructed according to the principles stated by Crowne and Marlowe (1960).’ ’ The construction and validation of the Ling lie scale is detailed in Ling’s (1967) master’s thesis. Briefly, a pool of items was created using the following sources: (a) all items of a German translation of the Marlowe-Crowne Social Desirability scale; (b) the lie items of the German MMQ-Adaption (Eysenck, 1953); (c) all items from the MMPI lie scale which were not included in the MMQ; (d) two items from the German ENNR (Brengelmann & Brengelmann, 1960) not included in any of the above scales; and (e) new items formulated by Ling. All items were rated by independent judges together with filler items from other questionnaires with regard to the social desirability and relative frequency of the behaviors involved. A total of 55 items representing behavior either high in social desirability, but low in frequency or low in social desirability, but high in frequency were retained for further analyses. Items were selected that maximized item-total correlations for the lie scale and minimized correlations with neuroticism. The resulting 32-item scale had satisfactory psychometric quality.
FACTOR
DELETION
TECHNIQUE
47
In order to provide criteria for estimating the concurrent validity of the questionnaire scales, each subject was rated by three peers on 32 trait dimensions. These dimensions were constructed to reflect primarily a number of components of the realms of neuroticism and introversion-extraversion. Two examples are provided below: Anxious, insecure, worried Sociable, lively, unconstrained, gregarious
Anxiousness 7654321 Gregariousness 7654321
Nonanxious, self-confident, intrepid Unsociable, restrained, awkward
The SDSV of the items was assessed independently by a panel of psychologists using the procedure proposed by Edwards (19.53). All ratings were given on 7-point scales.
RESULTS For each of the 32 dimensions, ratings given by the three acquaintances were first summed for each subject in order to enhance the reliability of the scores (Horowitz, Inouye, & Siegelmann, 1979). The median reliability of these sum scores as estimated by an intraclass correlation coefficient (Form 1, 3 according to the taxonomy of intraclass correlations by Shrout & Fleiss (1979) was $J = S7. The peer ratings were factor analyzed using Hotelling’s principal factor solution. Five factors were retained and rotated according to the promax criterion (Hendrickson & White, 1964). One neuroticism (N) and one extraversion (E) factor were unambiguously identified (Amelang & Borkenau, 1982). For these two dimensions, factor scores were estimated and used as validity criteria for N and E in the course of the further analyses. These two factor scores were negatively correlated (r = - .44). Their respective validities relative to the EPI N and E scores amounted to r = .54 for both scales using Form A and to r = .46 and Y = .52 for N and E, respectively, using Form B. The reliability (Cronbach’s (x) of the Ling lie scale was 80, which was substantially higher than the internal consistencies of the EPI lie scales. These amounted to .45 for Form A and to .30 for Form B. Combining the Form A and B lie items into a single scale comprising 18 items resulted in an internal consistency of S9. Therefore, the Ling lie scale was used as a measure of SDR. Its correlation was r = .lO with the EPI Form A extraversion scale, r = - .32 with the neuroticism scale, and r = .46 with the EPJ lie scale. The respective correlations with the EPI Form B scales were r = .06, r = - .36, and r = S7, respectively. Thus, whereas the correlations with extraversion were negligible, those with neuroticism were substantial. The intercorrelation of the two neuroticism scales, however, amounted to r = .77 and was thereby far beyond their correlation with the lie scale. Thus, whereas neuroticism and SDR were negatively correlated to some degree, the SD scale undoubtedly measured something different from emotional stability. The
48
BORKENAU
AND
AMELANG
correlation between the Ling lie scale and the EPI lie scales nearly reached the possible maximum given the estimated reliability of each of the measures. The social desirability scale values given for each EPI item by 12 independent judges were averaged separately for each item. The reliability of the resulting mean ratings estimated by an intraclass correlation (Form 1, 12 according to Shrout & Fleiss, 1979). amounted to is, = .96.* An Ordinary
Factor Solution
In order to appraise the merits of the principal-factor deletion technique, it is useful to compare it to an ordinary factor solution. This was done separately for both Forms of the EPI so as to investigate the convergence of the results. The 57 E, N, and L items of each Form were factor analyzed using the method of principal-factor analysis with iterated communalities, provided in the Statistical Analyses System (SAS) package. Following the normal convention, the squared multiple correlations served as the initial communality estimates. It was decided a priori to retain four factors: One neuroticism factor and two factors of extraversion were expected to emerge. The fourth factor was retained in line with the suggestion of Paulhus (1981) that one additional factor be extracted prior to the application of PFDT. The four factors explained 21% of the total variance in Form A and 17%, in Form B. The principal factors were rotated to the Promax criterion using a power of k = 4 in creating the target matrix for the procrustean transformation (Hendrickson & White, 1964). The oblique Promax rotation was preferred to an orthogonal rotation because simple structure was more pronounced and the intercorrelations of questionnaire and correspondent peer rating factors were higher using the oblique method of rotation. Factor scores were estimated for each subject on the four rotated factors and were correlated with the N and E scores resulting from a factor analysis of the peer ratings as well as with the Ling lie scale as a measure of impression management. Table 1 presents the respective correlations. The differential validity indices given in the last two columns serve as a measure of content saturation adjusted for SDR. They were calculated using the formula DVI = (V* - SD*)‘“, where V is the validity coefficient given in either the first or second column and SD is the correlation with the lie scale given in the third column. Considering the results for Form A, Factor I can be identified as neuroticism. This interpretation is supported by the promax rotated pattern of factor loadings: Out of 23 items loading at least (Y = .30 on this factor (irrespective of sign), 21 belong to the neuroticism scale according to * As all variance between the judges is regarded as error in this coefficient, the estimate is a lower bound of the true reliability of the SDSV.
FACTOR
DELETION
TABLE RESULTS OF THE ORDINARY
EPI Factor Form A I II III IV Form B I II II IV
Correlation with N (peer ratings)
Correlation with E (peer ratings)
.54 .35 .26 .I5
-.17 - .56 .i5 - .03
.48 .29 .24 -.19
-.14 -.45 - .47 .32
49
TECHNIQUE
1 FACTOR SOLUTION
Correlation with Ling lie scale
Differential Validity Index for N
Differential Validity Index for E
- .32 -.I0 - .25 .50
.43 .34 .07 -
.55 -
- .33 -.24 .I2 .30
.35 .16 .21 -
.38 .45 .I1
the EPI scoring key (Eysenck & Eysenck, 1964). Note that for factor I, the factor scores are negatively correlated with SDR. The second factor should be interpreted as extraversion according to the correlation matrix of the factor scores. This interpretation is also supported by the matrix of factor loadings: Out of 13 items loading at least (Y = .30 on this factor, 12 are usually scored as extraversion items. The other two factors are difficult to identify from the matrix of factor loadings. As far as the intercorrelations of the factor scores are concerned, however, the last factor shows quite a high correlation with the lie scale. The results with regard to Form B of the EPI are quite similar although the factor structure seems to be somewhat less clear-cut, and two extraversion factors can be identified. Principal-Factor Deletion Once again the method of principal-factor analysis with iterated communalities was performed, so that the results of the tlrst step were identical with that of ordinary factor solution. This time, however, no rotation was performed. Instead, the loadings of the items on the four principal axes were correlated with the items’ mean SDSV, which were based on the ratings of the panel of psychologists. For Form A, the respective correlations were - .63, .03, .04, and .02; for Form B the corresponding correlations were - .64, .04, - .07, and .29. Thus, the SDSV were highly correlated with the loadings of the items on the first principal axis, whereas the correlations with the loadings on the other factors were minimal. Although the r 3 .70 criterion suggested by Paulhus (1981) was missed by a narrow margin, we assume our results to be by and large in agreement with his premises.
50
BORKENAU
AND
AMELANG
The high correlation of the items’ loadings on the first principal factor with their SDSV reveals that this factor is highly contaminated with SD. Therefore this factor may (a) measure variance in actual behaviors that are socially desirable, or (b) assess the tendency to misrepresent oneself in order to create an advantageous self-presentation. Evidence relevant to these two interpretations is provided by the factor scores which were estimated and correlated with the EPI extraversion and neuroticism scores as well as with the Ling lie scale. In order to perform two independent analyses, the first principal axis of the EPI Form A items was correlated with the Form A extraversion and neuroticism scores resulting from the application of the usual scoring key (Eysenck & Eysenck, 1964). Correspondingly, the first principal factor derived from the Form B items was correlated with the Form B extraversion and neuroticism scores. However, the Ling lie scale scores were used in both tests due to considerations of reliability. The resulting correlations are given in Table 2. For both forms, the first unrotated factor correlated almost perfectly with the neuroticism score, whereas the correlation with the measure of SDR was only modest. In order to continue the principal-factor deletion procedure this factor was removed from the further analyses and the communalities were adjusted using the Statistical Analyses System (SAS). The explained variance was diminished by half thereby, since the first principal factor had explained 52% of the common variance in Form A and 46% of the common variance in Form B. The remaining three factors were then rotated again using the Promax criterion. The loadings of the items on the three remaining rotated factors were correlated with their SDSV at r = .03, Y = - .05, and r = .Ol for Form A and r = .06, r = .09, and Y = .27 for Form B. Thus, the principal-factor deletion technique was successful in substantially reducing the correlations of the items’ factor loadings with their SDSV and in this sense the goal of controlling for SD was largely attained. In order to investigate the consequences for validity and content saturation, the factor scores for the three remaining rotated factors were estimated and subsequently correlated with the E and N scores based on the peer ratings, and the scores on the Ling lie scale. Table 3 presents the resulting correlation coefficients and the differential validity index.
CORRELATION
OF THE FIRST
PRINCIPAL
TABLE AXIS’
SCORESAND EPI form A B
2
FACTOR SCORE THE LING SCALE
WITH
THE
EPI E
AND
N
Correlation with E scale
Correlation with N scale
Correlation with Ling scale
-.43 - .61
.95 .88
- .33 - .28
FACTOR
DELETION
TABLE RESULTS OF THE PRINCIPAL
EPI factor Form A I II III Form B I II III
Correlation with N (peer ratings)
Correlation with E (peer ratings)
TECHNIQUE
51
3
FACTOR DELETION
Correlation with Ling lie scale
TECHNIQUE
Differential validity index for N
Differential validity index for E
-.02 .OS - .03
- A? .17 .41
.27 .28 - .22
-
.38 .35
.03 -.19 -.04
- .37 -.20 - .21
.34 -.I7 .12
.08 -
.I5 .II .17
In Form A, two extraversion factors but no neuroticism factor seem to emerge. The interpretation of the first factor as assessing introversionextraversion is supported by the Promax rotated pattern of factor loadings: Out of eight items loading at least (Y = .30 on this factor (irrespective of sign), six belong to the EPI extraversion scale. The interpretation of the third factor as assessing extraversion, however, is less clear: out of five salient items, only three are usually scored as E and two are scored as N items. If the principal-factor deletion technique is applied to Form B, there is also no neuroticism factor detectable, and the factor exhibiting the highest correlation with peer-rated extraversion is highly confounded with SDR as measured by the Ling lie scale. DISCUSSION The premise of the deletion technique that the first principal axis is usually highly contaminated with SD was by and large confirmed. More pronounced, however, were the correlations with the neuroticism scale of the respective EPI Forms. Thus, there were already hints at this stage of the analysis that the factor to be deleted might be neuroticsm. Exactly this outcome resulted from the application of PFDT: Whereas the ordinary factor solution revealed a N factor within the patterns of factor loadings and in the correlations of the factor scores with peer ratings, this factor could no longer be identified in either the patterns of factor loadings resulting from PFDT or in the correlations with peer ratings. In contrast, the correlation of the factor scores with the Ling lie scale was only marginally diminished by the removal of the first principal axis. The differential validity indices reveal that the factors emerging from PFDT are less content saturated compared to those resulting from an ordinary factor solution, with this pattern being more pronounced for neuroticism compared to extraversion. The diminished content variance for extraversion
52
BORKENAU
AND
AMELANG
may stem from the substantial correlations between the deleted factors and the EPI extraversion scales. These findings seem to originate from the meaning of the first principal axis. Our results suggest that it should be interpreted as neuroticism or psychopathology, traits that are heavily confounded with SD, but which nevertheless seem to mirror true interindividual differences in behavior. This is evidenced by the substantial validity coefficients of about r = SO between self- and peer-reported neuroticism in the present study. The first principal factor also assesses SDR or lying in the sense of impression management, but the obtained pattern of correlations indicates that this is not its principal meaning. Moreover, it is debatable whether the first principal factor measures “self-deception, where the respondent actually believes his or her positive self-reports” (Paulhus, 1984, p. 599). Our results do not support this view because of the substantial correlations between self-reports and peer reports regarding this factor. They are most compatible with a substantive point of view as proposed, for example, by Block (1965). What conclusions should be drawn for the application of PFDT? Because predictive losses instead of gains resulted from the deletion of the first principal axis, there are no empirical arguments for the control of this component of variance from a purely predictive viewpoint. From a construct point of view, however, there exist arguments for dimensions being both confounded (e.g., Wiggins, 1979) and unconfounded with SD (e.g., Peabody, 1984). Paulhus (1984) himself proposes not to use PFDT when the first principal factor mirrors individual differences in self-deception or in self-esteem. In contrast, according to Paulhus, the first factor should be deleted when it measures impression management. Thus, the problem remains how an impression management or lie factor might be identified in questionnaire responses. In those cases in which lie scale scores closely parallel the first factor obtained in the factor analysis, PFDT can be profitably employed to remove the influence of socially desirable responding. In many other cases in which lie scale scores deviate substantially from the first factor, PFDT would appear to be of little value. REFERENCES Amelang, M., & Bartussek, D. (1970). Untersuchung zur Validitaet einer neuen Luegenskala. Diagnostica, 16, 103-122. Amelang, M., & Borkenau, P. (1981). Untersuchungen zur Validitaet von Kontroll-Skalen fuer Soziale Erwuenschtheit und Akquieszenz. Diagnostica, 21, 295-319. Amelang, M., & Borkenau, P. (1982). Ueber die faktorielle Struktur und exteme Validitaet einiger Fragebogen-Skalen zur Erfassung von Dimensionen der Extraversion and emotionalen Labilitaet. Zeitschrift fuer Diffeerentielle und Diagnostische Psychologie, 3, 119-146. Anderson, N. H. (1968). Likableness ratings of 555 personality-trait words. Journal of Personality and Social Psychology, 9, 272-279.
FACTOR
DELETION
53
TECHNIQUE
Block, J. (1965). The challenge of response sets. New York: Appleton-Century-Crofts. Brengelmann, J. C., & Brengelmann, L. (1960). Deutsche Validierung von Fragebogen der Extraversion, neurotischen Tendenz und Rigiditaet. Zeitschrtft fuer Experimentelle und Angewandte Psychologie, 7, 291-331. Crowne, D. P., 18 Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. Edwards, A. L. (1953). The relationship between the judged desirability or a trait and the probability that the trait will be endorsed. Journal of Applied Psychology, 37, 90-93. Edwards, A. L., & Diers, C. J. (1962). Social desirability and the factorial interpretation of the MMPI. Educational and Psychological Measurement, 22, 501-509. Eysenck, H. J. (1953). Fragebogen als Messmittel der Persoenlichkeit: Eine experimentelle Untersuchung. Zeitschrtft fuer Experimentelle und Angewandte Psychologie, 1, 291335. Eysenck, H. J., & Eysenck, S. B. G. (1964). Manual of the Eysenck personality inventory. London: Univ. of London Press. Eysenck, H. J., & Eysenck, S. B. G. (1969). Personality structure and measurement. London: Routledge & Kegan Paul. Heilbrun, A. B. (1964). Social learning theory, social desirability, and the MMPI. Psychological Bulletin,
61, 371-387.
Hendrickson, A. E., & White, P. 0. (1964). Promax: A quick method for rotation to oblique simple structure. British Journal of Psychology, 17, 65-70. Horowitz, L. M., Inouye, D., & Siegelmann, E. Y. (1979). On averaging judges’ ratings to increase their correlation with an external criterion. Journal of Consulting and Clinical
Psychology,
47, 453-458.
Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (Vol. 2). New York: Academic Press. Ling, M. (1967). Die Konstruktion einer Luegenskala. Unpublished master’s thesis, University of Hamburg, Federal Republic of Germany. Paulhus, D. L. (1981). Control of social desirability in personality inventories: Principal factor deletion. Journal of Research in Personality, 15, 383-388. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality
and Social
Psychology,
46, 598-609.
Peabody, D. (1984). Personality dimensions through trait inferences. Journal and Social
Psychology,
of Personality
46, 384-403.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. Wiggins, J. S. (1959). Interrelationships among MMPl measures of dissimulation under standard and social desirability instructions. Journal of Consulting Psychology, 23, 419-427. Wiggins, J. S. (1979). A psychological taxonomy of trait-descriptive terms: The interpersonal domain. Journal of Personality and Social Psychology, 37, 395-412.
OF RESEARCH
19, 44-53 (1985)
IN PERSONALITY
The Control of Social Desirability in Personality Inventories: A Study Using the Principal-Factor Deletion Technique PETER BORKENAU University of Bielefeld AND MANFRED
AMELANG
University of Heidelberg The present study sought to investigate Paulhus’ (1981) principal-factor deletion technique (PFDT), a factor analytic method for controlling for socially desirable responding. Adult subjects (N = 330) responded to both forms of the Eysenck Personality Inventory and a 32-item lie scale. Peer ratings of extraversion and neuroticism served as criterion measures. The factor deleted by PFDT more closely reflected neuroticism than impression management. Factors resulting from PFDT were considerably less valid and less content saturated than those emerging from an ordinary factor solution. The implications of these results are discussed from the perspectives of predictive and construct validity. o 1985 Academic press. Inc.
A major difficulty in the development and use of personality scales is the operation of response sets. These include the tendency of subjects to respond in a manner that would be expected of well-adjusted people, thereby presenting themselves in a socially desirable light. To the extent that questionnaires reflect subjects’ intentions to present a socially desirable appearance, the proportion of variance associated with the intended content is lowered, leading to a reduction in the validity of the questionnaire. Therefore, numerous attempts have been made to control for social desirability (SD) in questionnaire and rating data. Recently, Paulhus (1981) has proposed a factor analytic approach for controlling for SD, The research reported in this paper was supported by a grant from Deutsche Forschungsgemeinschaft to Manfred Amelang (AZ.: Am 37/5). We thank Stephen G.’ West for his comments on an earlier version of this manuscript. Requests for reprints should be sent to Peter Borkenau, Abteilung Psychologie, Universitit Bielefeld, Postfach 8640, D4800 Bielefeld, West Germany. 44 0092-6566/85 $3.00 Copyright All rights
B 19S5 by Academic Press, Inc. of reproduction in any form reserved.
FACTOR
DELETION
45
TECHNIQUE
which has been termed the principal-factor deletion technique (PFDT). This method is based on the assumption that in an unrotated factor solution the loadings of the items on the first principal factor are highly correlated with their social desirability scale values (SDSV). Thus, the first principal factor may be assumed to assess socially desirable responding (Edwards & Diers, 1962). Paulhus (1981) suggested that this factor be deleted prior to rotation. Through this procedure, the influence of socially desirable responding should be substantially reduced. SOCIALLY
DESIRABLE
RESPONDING
AND LYING
In order to evaluate methods for controlling socially desirable responding (SDR), it is important to distinguish between (a) the tendency to actually behave in socially desirable ways, which should be mirrored in the item endorsements, and (b) the tendency to respond to test items without regard to actual behavior but only with regard to the social appearance which the person wishes to create (Edwards, 1953; Heilbrun, 1964; Paulhus, 1984; Wiggins, 1959, 1979). This distinction may be most clearly illustrated with regard to measures of emotional stability: Describing oneself as psychopathological necessarily implies reporting undesirable behaviors. Those items that best discriminate clinical from normal groups usually show extreme SDSV. Indeed, as Heilbrun (1964) noted “the dimensions of psychological health and social desirability are in large measure one and the same” (p. 385). From the perspective of predictive validity, the crux of the issue is to identify and eliminate only those components of SDR that result from a misrepresentation of the subject’s true personality. From the viewpoint of construct validity, it is debatable whether dimensions confounded with social desirability (Wiggins, 1979) or dimensions unconfounded with social desirability (Peabody, 1984) are preferable. If this distinction is applied to the PFDT, it becomes evident that the consequences of using PFDT depend on the meaning of the first principal axis: Is it primarily assessing individual differences in socially evaluated behaviors? Or, is it primarily assessing individual differences in impression management? Unfortunately, the usually high correlation of the items’ loadings on this factor with their SDSV does not contribute to the answer as this does not reveal why people differ in their endorsement of socially desirable items. How then can this problem be clarified? First, Crowne and Marlowe (1960) have suggested some guidelines for developing items to separate persons who misrepresent themselves on personality inventories from those who actually behave in a socially desirable way. The content of the lie items they called for should not have psychopathological implications. Accordingly, the correlation of a lie scale with diagnostic scales should be low. Furthermore, the scale should include items that describe (a) behaviors which are not only culturally sanctioned and approved but also improbable of occurrence
46
BORKENAU
AND
AMELANG
and (b) items that describe behaviors which are disapproved but very common within society. A social desirability scale constructed in accordance with these guidelines should be useful in identifying those persons who seek to present a positive impression of themselves on personality questionnaires. Some empirical evidence exists supporting the application of lie scales to increase the validity of scores on other questionnaires. For example, Amelang and Borkenau (1981) found that self-report measures of neuroticism and extraversion of subjects scoring high on a lie scale showed only low correlations with pertinent peer ratings, whereas these correlations were substantially higher for subjects who scored low on the lie scale. This effect was more pronounced with regard to neuroticism as compared to extraversion. Applying this reasoning to the principal factor deletion technique, subjects’ factor scores on the first principal axis should be highly correlated with their lie scores if the first principal factor is assessing impression management. Second, if the first principal factor accounts for variance that is mainly attributable to individual differences in the subjects’ tendency to misrepresent themselves on personality inventories, its deletion should not diminish the validity of the remaining factors. Moreover, their content saturation according to Jackson’s (1970) definition should be increased. If, however, the first principal axis reflects differences in the social desirability of the subjects’ actual behaviors, its elimination can be expected to decrease the validity and content saturation of the remaining factors. In the present study, the PFDT was investigated applying these criteria. METHOD Subjects. Subjects were 330 German adults (145 male, 185 female) from different occupational groups who were paid for their cooperation. Approximately 85% of the participants were between 18 and 30 years old. Procedrcre. The subjects were administered a German version of both Form A and Form B of the Eysenck Personality Inventory (EPI; Eysenck & Eysenck, 1964) and a 32-item lie scale originally developed by Ling (see Amelang & Bartussek, 1970) which had been constructed according to the principles stated by Crowne and Marlowe (1960).’ ’ The construction and validation of the Ling lie scale is detailed in Ling’s (1967) master’s thesis. Briefly, a pool of items was created using the following sources: (a) all items of a German translation of the Marlowe-Crowne Social Desirability scale; (b) the lie items of the German MMQ-Adaption (Eysenck, 1953); (c) all items from the MMPI lie scale which were not included in the MMQ; (d) two items from the German ENNR (Brengelmann & Brengelmann, 1960) not included in any of the above scales; and (e) new items formulated by Ling. All items were rated by independent judges together with filler items from other questionnaires with regard to the social desirability and relative frequency of the behaviors involved. A total of 55 items representing behavior either high in social desirability, but low in frequency or low in social desirability, but high in frequency were retained for further analyses. Items were selected that maximized item-total correlations for the lie scale and minimized correlations with neuroticism. The resulting 32-item scale had satisfactory psychometric quality.
FACTOR
DELETION
TECHNIQUE
47
In order to provide criteria for estimating the concurrent validity of the questionnaire scales, each subject was rated by three peers on 32 trait dimensions. These dimensions were constructed to reflect primarily a number of components of the realms of neuroticism and introversion-extraversion. Two examples are provided below: Anxious, insecure, worried Sociable, lively, unconstrained, gregarious
Anxiousness 7654321 Gregariousness 7654321
Nonanxious, self-confident, intrepid Unsociable, restrained, awkward
The SDSV of the items was assessed independently by a panel of psychologists using the procedure proposed by Edwards (19.53). All ratings were given on 7-point scales.
RESULTS For each of the 32 dimensions, ratings given by the three acquaintances were first summed for each subject in order to enhance the reliability of the scores (Horowitz, Inouye, & Siegelmann, 1979). The median reliability of these sum scores as estimated by an intraclass correlation coefficient (Form 1, 3 according to the taxonomy of intraclass correlations by Shrout & Fleiss (1979) was $J = S7. The peer ratings were factor analyzed using Hotelling’s principal factor solution. Five factors were retained and rotated according to the promax criterion (Hendrickson & White, 1964). One neuroticism (N) and one extraversion (E) factor were unambiguously identified (Amelang & Borkenau, 1982). For these two dimensions, factor scores were estimated and used as validity criteria for N and E in the course of the further analyses. These two factor scores were negatively correlated (r = - .44). Their respective validities relative to the EPI N and E scores amounted to r = .54 for both scales using Form A and to r = .46 and Y = .52 for N and E, respectively, using Form B. The reliability (Cronbach’s (x) of the Ling lie scale was 80, which was substantially higher than the internal consistencies of the EPI lie scales. These amounted to .45 for Form A and to .30 for Form B. Combining the Form A and B lie items into a single scale comprising 18 items resulted in an internal consistency of S9. Therefore, the Ling lie scale was used as a measure of SDR. Its correlation was r = .lO with the EPI Form A extraversion scale, r = - .32 with the neuroticism scale, and r = .46 with the EPJ lie scale. The respective correlations with the EPI Form B scales were r = .06, r = - .36, and r = S7, respectively. Thus, whereas the correlations with extraversion were negligible, those with neuroticism were substantial. The intercorrelation of the two neuroticism scales, however, amounted to r = .77 and was thereby far beyond their correlation with the lie scale. Thus, whereas neuroticism and SDR were negatively correlated to some degree, the SD scale undoubtedly measured something different from emotional stability. The
48
BORKENAU
AND
AMELANG
correlation between the Ling lie scale and the EPI lie scales nearly reached the possible maximum given the estimated reliability of each of the measures. The social desirability scale values given for each EPI item by 12 independent judges were averaged separately for each item. The reliability of the resulting mean ratings estimated by an intraclass correlation (Form 1, 12 according to Shrout & Fleiss, 1979). amounted to is, = .96.* An Ordinary
Factor Solution
In order to appraise the merits of the principal-factor deletion technique, it is useful to compare it to an ordinary factor solution. This was done separately for both Forms of the EPI so as to investigate the convergence of the results. The 57 E, N, and L items of each Form were factor analyzed using the method of principal-factor analysis with iterated communalities, provided in the Statistical Analyses System (SAS) package. Following the normal convention, the squared multiple correlations served as the initial communality estimates. It was decided a priori to retain four factors: One neuroticism factor and two factors of extraversion were expected to emerge. The fourth factor was retained in line with the suggestion of Paulhus (1981) that one additional factor be extracted prior to the application of PFDT. The four factors explained 21% of the total variance in Form A and 17%, in Form B. The principal factors were rotated to the Promax criterion using a power of k = 4 in creating the target matrix for the procrustean transformation (Hendrickson & White, 1964). The oblique Promax rotation was preferred to an orthogonal rotation because simple structure was more pronounced and the intercorrelations of questionnaire and correspondent peer rating factors were higher using the oblique method of rotation. Factor scores were estimated for each subject on the four rotated factors and were correlated with the N and E scores resulting from a factor analysis of the peer ratings as well as with the Ling lie scale as a measure of impression management. Table 1 presents the respective correlations. The differential validity indices given in the last two columns serve as a measure of content saturation adjusted for SDR. They were calculated using the formula DVI = (V* - SD*)‘“, where V is the validity coefficient given in either the first or second column and SD is the correlation with the lie scale given in the third column. Considering the results for Form A, Factor I can be identified as neuroticism. This interpretation is supported by the promax rotated pattern of factor loadings: Out of 23 items loading at least (Y = .30 on this factor (irrespective of sign), 21 belong to the neuroticism scale according to * As all variance between the judges is regarded as error in this coefficient, the estimate is a lower bound of the true reliability of the SDSV.
FACTOR
DELETION
TABLE RESULTS OF THE ORDINARY
EPI Factor Form A I II III IV Form B I II II IV
Correlation with N (peer ratings)
Correlation with E (peer ratings)
.54 .35 .26 .I5
-.17 - .56 .i5 - .03
.48 .29 .24 -.19
-.14 -.45 - .47 .32
49
TECHNIQUE
1 FACTOR SOLUTION
Correlation with Ling lie scale
Differential Validity Index for N
Differential Validity Index for E
- .32 -.I0 - .25 .50
.43 .34 .07 -
.55 -
- .33 -.24 .I2 .30
.35 .16 .21 -
.38 .45 .I1
the EPI scoring key (Eysenck & Eysenck, 1964). Note that for factor I, the factor scores are negatively correlated with SDR. The second factor should be interpreted as extraversion according to the correlation matrix of the factor scores. This interpretation is also supported by the matrix of factor loadings: Out of 13 items loading at least (Y = .30 on this factor, 12 are usually scored as extraversion items. The other two factors are difficult to identify from the matrix of factor loadings. As far as the intercorrelations of the factor scores are concerned, however, the last factor shows quite a high correlation with the lie scale. The results with regard to Form B of the EPI are quite similar although the factor structure seems to be somewhat less clear-cut, and two extraversion factors can be identified. Principal-Factor Deletion Once again the method of principal-factor analysis with iterated communalities was performed, so that the results of the tlrst step were identical with that of ordinary factor solution. This time, however, no rotation was performed. Instead, the loadings of the items on the four principal axes were correlated with the items’ mean SDSV, which were based on the ratings of the panel of psychologists. For Form A, the respective correlations were - .63, .03, .04, and .02; for Form B the corresponding correlations were - .64, .04, - .07, and .29. Thus, the SDSV were highly correlated with the loadings of the items on the first principal axis, whereas the correlations with the loadings on the other factors were minimal. Although the r 3 .70 criterion suggested by Paulhus (1981) was missed by a narrow margin, we assume our results to be by and large in agreement with his premises.
50
BORKENAU
AND
AMELANG
The high correlation of the items’ loadings on the first principal factor with their SDSV reveals that this factor is highly contaminated with SD. Therefore this factor may (a) measure variance in actual behaviors that are socially desirable, or (b) assess the tendency to misrepresent oneself in order to create an advantageous self-presentation. Evidence relevant to these two interpretations is provided by the factor scores which were estimated and correlated with the EPI extraversion and neuroticism scores as well as with the Ling lie scale. In order to perform two independent analyses, the first principal axis of the EPI Form A items was correlated with the Form A extraversion and neuroticism scores resulting from the application of the usual scoring key (Eysenck & Eysenck, 1964). Correspondingly, the first principal factor derived from the Form B items was correlated with the Form B extraversion and neuroticism scores. However, the Ling lie scale scores were used in both tests due to considerations of reliability. The resulting correlations are given in Table 2. For both forms, the first unrotated factor correlated almost perfectly with the neuroticism score, whereas the correlation with the measure of SDR was only modest. In order to continue the principal-factor deletion procedure this factor was removed from the further analyses and the communalities were adjusted using the Statistical Analyses System (SAS). The explained variance was diminished by half thereby, since the first principal factor had explained 52% of the common variance in Form A and 46% of the common variance in Form B. The remaining three factors were then rotated again using the Promax criterion. The loadings of the items on the three remaining rotated factors were correlated with their SDSV at r = .03, Y = - .05, and r = .Ol for Form A and r = .06, r = .09, and Y = .27 for Form B. Thus, the principal-factor deletion technique was successful in substantially reducing the correlations of the items’ factor loadings with their SDSV and in this sense the goal of controlling for SD was largely attained. In order to investigate the consequences for validity and content saturation, the factor scores for the three remaining rotated factors were estimated and subsequently correlated with the E and N scores based on the peer ratings, and the scores on the Ling lie scale. Table 3 presents the resulting correlation coefficients and the differential validity index.
CORRELATION
OF THE FIRST
PRINCIPAL
TABLE AXIS’
SCORESAND EPI form A B
2
FACTOR SCORE THE LING SCALE
WITH
THE
EPI E
AND
N
Correlation with E scale
Correlation with N scale
Correlation with Ling scale
-.43 - .61
.95 .88
- .33 - .28
FACTOR
DELETION
TABLE RESULTS OF THE PRINCIPAL
EPI factor Form A I II III Form B I II III
Correlation with N (peer ratings)
Correlation with E (peer ratings)
TECHNIQUE
51
3
FACTOR DELETION
Correlation with Ling lie scale
TECHNIQUE
Differential validity index for N
Differential validity index for E
-.02 .OS - .03
- A? .17 .41
.27 .28 - .22
-
.38 .35
.03 -.19 -.04
- .37 -.20 - .21
.34 -.I7 .12
.08 -
.I5 .II .17
In Form A, two extraversion factors but no neuroticism factor seem to emerge. The interpretation of the first factor as assessing introversionextraversion is supported by the Promax rotated pattern of factor loadings: Out of eight items loading at least (Y = .30 on this factor (irrespective of sign), six belong to the EPI extraversion scale. The interpretation of the third factor as assessing extraversion, however, is less clear: out of five salient items, only three are usually scored as E and two are scored as N items. If the principal-factor deletion technique is applied to Form B, there is also no neuroticism factor detectable, and the factor exhibiting the highest correlation with peer-rated extraversion is highly confounded with SDR as measured by the Ling lie scale. DISCUSSION The premise of the deletion technique that the first principal axis is usually highly contaminated with SD was by and large confirmed. More pronounced, however, were the correlations with the neuroticism scale of the respective EPI Forms. Thus, there were already hints at this stage of the analysis that the factor to be deleted might be neuroticsm. Exactly this outcome resulted from the application of PFDT: Whereas the ordinary factor solution revealed a N factor within the patterns of factor loadings and in the correlations of the factor scores with peer ratings, this factor could no longer be identified in either the patterns of factor loadings resulting from PFDT or in the correlations with peer ratings. In contrast, the correlation of the factor scores with the Ling lie scale was only marginally diminished by the removal of the first principal axis. The differential validity indices reveal that the factors emerging from PFDT are less content saturated compared to those resulting from an ordinary factor solution, with this pattern being more pronounced for neuroticism compared to extraversion. The diminished content variance for extraversion
52
BORKENAU
AND
AMELANG
may stem from the substantial correlations between the deleted factors and the EPI extraversion scales. These findings seem to originate from the meaning of the first principal axis. Our results suggest that it should be interpreted as neuroticism or psychopathology, traits that are heavily confounded with SD, but which nevertheless seem to mirror true interindividual differences in behavior. This is evidenced by the substantial validity coefficients of about r = SO between self- and peer-reported neuroticism in the present study. The first principal factor also assesses SDR or lying in the sense of impression management, but the obtained pattern of correlations indicates that this is not its principal meaning. Moreover, it is debatable whether the first principal factor measures “self-deception, where the respondent actually believes his or her positive self-reports” (Paulhus, 1984, p. 599). Our results do not support this view because of the substantial correlations between self-reports and peer reports regarding this factor. They are most compatible with a substantive point of view as proposed, for example, by Block (1965). What conclusions should be drawn for the application of PFDT? Because predictive losses instead of gains resulted from the deletion of the first principal axis, there are no empirical arguments for the control of this component of variance from a purely predictive viewpoint. From a construct point of view, however, there exist arguments for dimensions being both confounded (e.g., Wiggins, 1979) and unconfounded with SD (e.g., Peabody, 1984). Paulhus (1984) himself proposes not to use PFDT when the first principal factor mirrors individual differences in self-deception or in self-esteem. In contrast, according to Paulhus, the first factor should be deleted when it measures impression management. Thus, the problem remains how an impression management or lie factor might be identified in questionnaire responses. In those cases in which lie scale scores closely parallel the first factor obtained in the factor analysis, PFDT can be profitably employed to remove the influence of socially desirable responding. In many other cases in which lie scale scores deviate substantially from the first factor, PFDT would appear to be of little value. REFERENCES Amelang, M., & Bartussek, D. (1970). Untersuchung zur Validitaet einer neuen Luegenskala. Diagnostica, 16, 103-122. Amelang, M., & Borkenau, P. (1981). Untersuchungen zur Validitaet von Kontroll-Skalen fuer Soziale Erwuenschtheit und Akquieszenz. Diagnostica, 21, 295-319. Amelang, M., & Borkenau, P. (1982). Ueber die faktorielle Struktur und exteme Validitaet einiger Fragebogen-Skalen zur Erfassung von Dimensionen der Extraversion and emotionalen Labilitaet. Zeitschrift fuer Diffeerentielle und Diagnostische Psychologie, 3, 119-146. Anderson, N. H. (1968). Likableness ratings of 555 personality-trait words. Journal of Personality and Social Psychology, 9, 272-279.
FACTOR
DELETION
53
TECHNIQUE
Block, J. (1965). The challenge of response sets. New York: Appleton-Century-Crofts. Brengelmann, J. C., & Brengelmann, L. (1960). Deutsche Validierung von Fragebogen der Extraversion, neurotischen Tendenz und Rigiditaet. Zeitschrtft fuer Experimentelle und Angewandte Psychologie, 7, 291-331. Crowne, D. P., 18 Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. Edwards, A. L. (1953). The relationship between the judged desirability or a trait and the probability that the trait will be endorsed. Journal of Applied Psychology, 37, 90-93. Edwards, A. L., & Diers, C. J. (1962). Social desirability and the factorial interpretation of the MMPI. Educational and Psychological Measurement, 22, 501-509. Eysenck, H. J. (1953). Fragebogen als Messmittel der Persoenlichkeit: Eine experimentelle Untersuchung. Zeitschrtft fuer Experimentelle und Angewandte Psychologie, 1, 291335. Eysenck, H. J., & Eysenck, S. B. G. (1964). Manual of the Eysenck personality inventory. London: Univ. of London Press. Eysenck, H. J., & Eysenck, S. B. G. (1969). Personality structure and measurement. London: Routledge & Kegan Paul. Heilbrun, A. B. (1964). Social learning theory, social desirability, and the MMPI. Psychological Bulletin,
61, 371-387.
Hendrickson, A. E., & White, P. 0. (1964). Promax: A quick method for rotation to oblique simple structure. British Journal of Psychology, 17, 65-70. Horowitz, L. M., Inouye, D., & Siegelmann, E. Y. (1979). On averaging judges’ ratings to increase their correlation with an external criterion. Journal of Consulting and Clinical
Psychology,
47, 453-458.
Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (Vol. 2). New York: Academic Press. Ling, M. (1967). Die Konstruktion einer Luegenskala. Unpublished master’s thesis, University of Hamburg, Federal Republic of Germany. Paulhus, D. L. (1981). Control of social desirability in personality inventories: Principal factor deletion. Journal of Research in Personality, 15, 383-388. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality
and Social
Psychology,
46, 598-609.
Peabody, D. (1984). Personality dimensions through trait inferences. Journal and Social
Psychology,
of Personality
46, 384-403.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. Wiggins, J. S. (1959). Interrelationships among MMPl measures of dissimulation under standard and social desirability instructions. Journal of Consulting Psychology, 23, 419-427. Wiggins, J. S. (1979). A psychological taxonomy of trait-descriptive terms: The interpersonal domain. Journal of Personality and Social Psychology, 37, 395-412.