Measuring rationality independent of social desirability

Prrao~. IIldll.id. D~fl. Vol. 3. pp. 303 10 309. 1987 Printed in Grear Britain. All rights reserved

MEASURING

RATIONALITY INDEPENDENT SOCIAL DESIRABILITY*

GORDON Departments

of ‘Psychiatry

0191-X869/82:030303-07 803.00 0 Copyright 0 1982 Pergamon Press Ltd

E.

BARNEs’t and

and ‘Psychology. Manitoba, (Receiced

BRENT

A.

VULCANO’

University of Manitoba. Canada R3E 0W3 19 October

OF

770 Bannatyne

Ave. Winnipeg,

1981)

Summary-The development of a test to measure Ellis’ concept of rationality is described. In the first study discussed. a 58-item test is developed to measure rationality. and the reliability and convergent validity of the test is described. In a second study, the discriminant validity of the test is examined. An attempt is also made to reduce social-desirability content in the test by eliminating items most highly correlated with a Social Desirability Scale. The final 44-item test is found to be high in both reliability and validity. The factor structure of the test is also examined.

INTRODUCTION In recent years numerous scales have been developed which attempt to measure the extent to which respondents hold irrational beliefs. As of 1976 Murphy and Ellis (1976) were able to find over 15 scales based on Ellis’ view of irrational beliefs. The fact that none of these scales seems to have established priority as the measure of ‘rationalityirrationality’ suggests that some problems are attached to each of these measures. The five of these tests which were chosen to be in more advanced stages of development by Martin et al. (1977) were: (1) Irrational Beliefs Test (Jones, 1969) (2) Fifteen Questions for Rating Reason (Argabrite and Nidorf, 1968) (3) The Ellis Scale (MacDonald and Games, 1972) (4) The Adult Irrational Ideas Inventory (Fox and Davies, 1971) (5) Test of Irrational Ideas (Laughridge, 1971).

In addition to the five tests mentioned above there have been three tests featured in recent issues of The Journal of Rational Living including: (1) Bard (1973) A Self Rating Scale for Rationality (2) Plutchik (1976) The Self-inventory (3) Higginbotham (1976) Irrationality Scale. And one test presented in the Journal of Educational and Psychological Measurement (Shorkey and Whiteman, 1977; Whiteman and Shorkey, 1978). In general, these tests have proven to be fairly reliable and some evidence for the convergent validity of the scales has usually been provided. Most studies however, have failed to investigate another important type of validity described by Campbell (Campbell, 1960; Campbell and Fiske, 1959), that being discriminant validity. In particular, it would seem important to show that a test measuring rationality or general adjustment is not contaminated by a heavy social-desirability component. Unfortunately the one test which has been shown to possess discriminate validity (i.e. low correlation with Crowne-Marlowe Social Desirability Scale), the Ellis scale (MacDonald and Games, 1972), has been shown to have rather poor reliability (Crabtree and Ward, 1975) and convergent validity (Martin et al., 1977; Crabtree and Ward, 1975) possibly because of the small number (9) of items in the test. * Basedon a paper presented at 25th Anniversary Conference (3rd National Conference) on Rational Emotive Therapy, New York, 5-8 June 1980. t TO whom requests for reprints and copies of the BVRT with scoring instructions should be submitted. 303

GORDON E. BARNES and BRENT A. VULCANO

304

The purpose of the present paper will be to describe preliminary attempts at developing a test of rationality that would be reliable and demonstrate both convergent and discriminant validity. STUDY

No. 1

Method Procedure

The initial stage of this project involved constructing a measure of rationality based on the Ellis and Harper (1961) Ten Irrational Ideas. These 10 ideas are listed below: (1) Idea that a person must receive love or approval from all the people they consider significant. (2) Idea that a person must be thoroughly competent, adequate and achieving. (3) Idea that when people act obnoxiously, and unfairly, they should be blamed and damned and viewed as bad, wicked or rotten individuals. (4) The idea that a person must view things as awful, terrible, horrible, and catastrophic when they get seriously frustrated, treated unfairly, or rejected. (5) The idea that emotional misery comes from external pressures and that a person has little ability to control or change their feelings. (6) The idea that if something seems dangerous or fearsome, a person must be preoccupied with it and make oneself anxious about it. (7) The idea that you can more easily avoid facing many life difficulties and selfresponsibilities than undertake more rewarding forms of self-discipline. (8) The idea that your past remains all important and that because something once strongly influenced your life, it has to keep determining your feelings and behaviour today. (9) The idea that people and things should turn out better than they do and that you must view it as awful and horrible if you do not find good solutions to life’s grim realities. (10) The idea that you can achieve maximum human happiness by inertia and inaction or by passively and uncommittedly enjoying oneself. Items were written or borrowed from existing tests to measure each of the Ten Irrational Ideas.* An initial item pool of 103 items was then administered to a sample of adults along with additional questionnaires administered for purposes of determining convergent validity. Item analyses were then performed on the original item pool and 58 items with item whole correlations greater than 0.20 were selected. Following the selection of the 58-item test correlations were computed with the scales included for purposes of determining convergent validity. Predictions were that a high score on the rationality test would be associated with lower scores on neuroticism, depression, external locus of control and fear and higher scores on ego-strength. Correlations with the Eysenck Extraversion and Lie scales were not expected to reach significance. Subjects

The Ss in this study included 172 adults with the mean age in the sample being 22. Materials

Tests that were administered included :

for determining

the convergent

validity of the scale

(1) the Eysenck Personality Inventory (Eysenck and Eysenck, 1968); * Items which were borrowed from existing scales included the 8 items from the self-control

subscale of the Reid and Ware (1974) I-E measure (used as items measuring irrational idea No. 5). and 10 items from the Pearson (1970) Desire for Novelty Scale (used to measure irrational idea No. 10). These items were adapted to the 5-point format (agree strongly to disagree strongly) used for all of the other items in the test.

Measuring

rationality

independent

of social desirability

305

Table 1. Correlat;ons between Rationality Scale scores and validity criteria before and after elimination of high social desirability items

Variable Extraversion Neuroticism Lie Scale Ego-strength Depression Internal-external Fear

control

%-Item test (N = 172)

44-Item test (N = 177)

0.092 -0.:33* 0.028 0.479* -0.552* - 0.256’ - 0.309*.

0.110 -0.501* -0.003 0.475* -0.551% - 0.247* - 0.309*

* P < 0.001. (2) (3) (4) (5)

the the the the

Barron (1953) Ego-strength Scale; Zung (1965) Self Rating Depression Inventory; Reid and Ware (1974) Internal External Locus of Control Wolpe and Lang (1964) Fear Survey Schedule.

Scale;

Results The Hoyt estimate of reliability for the 58-item test was 0.91. Correlations between the 5%item Rationality Test and the scales listed above are provided in Table 1. Results showed, as predicted, that Ss who scored higher on rationality also scored lower on neuroticism, depression, external locus of control, and fear and scored higher on egostrength. As expected the Rationality Test did not correlate significantly with either the Eysenck Lie or Extraversion scales. STUDY

No.

2

Method Procedure Study No. 2 was designed to determine the discriminant validity of the Rationality Test. Specifically, we were concerned with the social-desirability component in the test and the best ways for minimizing this component. Two strategies were employed for determining the social-desirability content in the Rationality Test. The first strategy used was to examine the correlation between the rationality test and the Marlowe-Crowne Social Desirability Scale (Crowne and Marlowe, 1960). In order to accomplish this objective Ss were administered a battery of tests including the Social Desirability Scale, the Eysenck and Eysenck (1968) EPI scale and the Rationality Test. An additional strategy employed was to examine the effects of a ‘fake good’ instructional set on Ss’ responses to the test. Presumably a test with high social-desirability content would be easily faked with Ss shifting their responses in the more desirable direction under the instructions to fake good adjustment. Subjects in our second study were asked to complete the questionnaires under normal and faking-adjustment instructional sets. Instructions in the fake adjustment condition were as follows: “I would like you to imagine that your scores are being used to decide whether or not you should be admitted to a mental hospital. You should answer the questions so that you will appear as normal and well adjusted as possible.” In order to try and determine how the effects of a social-desirability response set might be minimized, two courses of action were employed. First a Lie Scale (Eysenck and Eysenck, 1968) was employed to determine what percentage of subjects would be detected as ‘liars’ under the ‘fake good’ instructional set. Next, correlations between the Marlowe-Crowne Scale and individual items on the Rationality Test were computed. Items with high correlations with the Marlowe-Crowne Scale (>0.20) were than deleted from the test. The new 44-item reduced scale was then tested for reliability in both

306

GORDON E. BARNES and BRENT A. VULCANO Table 2. BVRT items and factor

loadings Varimax

Item No. 1. 2. 3. 4. 5. 6. 7. 8. 9. IO.

11. 12. 13.

14. 15. 16. 17. 18.

19.

20. 21. 22

23. 24. 25. 26.

27. 28. 29. 30. 31.

rotated

factor

loadings

Item I do not need to feel that everyone I meet likes me. I frequently worry about things over which I have no control. I find it easy to overcome my irrational fears. 1 can usually shut off thoughts that are causing me to feel anxious. Life is a ceaseless battle against irrational worries. I frequently worry about death. Crowds make me nervous. I frequently worry about the state of my health. I tend to worry about things before they actually occur. If I were told that someone had a criminal record I would not hire him or her to work for me. When I make a mistake I feel worthless and inadequate. When someone is wrong I sure let them know it. When 1 am frustrated the first thing I do is ask myself whether there is anything I can do to change it-now! Whenever something goes wrong I ask myself “Why did this have to happen to me?” Whenever things go wrong I say to myself, “I don’t like this, I can’t stand it.” I can usually find a cure for my own unhappiness when it occurs. Once I am depressed it takes me a long while to recover. I feel that when I become depressed or unhappy it is caused by other people or the events that happen. People have little or no ability to control their sorrows or rid themselves of their negative feelings. When I become angry I can usually control my anger. I can usually control my appetites for food or alcohol. The value of a human being is directly proportionate to his accomplishments, if he is not thoroughly competent and adequate in achieving he might as well curl up and die. The important part of playing a game is that you succeed. I feel badly when my achievement level is lower than others. I feel that I must succeed at everything I undertake. When I feel doubts about potential success I avoid participating and risking the chance of failure. When I set out to accomplish a task I stick with it to the end. If I find difficulties in life I discipline myself to face them. If 1 try to do something and encounter problems I give up easily. I find it difficult to work at tasks that have a long range payoff. I usually like to face my problems head on.

1

0.07

0.24

-0.11

-0.21

0.28

6 6

0.10 0.06

0.24 0.32

0.48 0.23

-0.08 -0.26

0.20 0.08

6

0.00

0.32

0.24

-0.17

0.10

6 6 6

0.12 0.27 0.18

0.05 0.15 0.24

0.49 0.39 0.31

0.17 0.16 0.05

0.09 - 0.08 0.02

6

0.17

0.01

0.49

0.15

0.00

6

0.12

0.24

0.63

0.11

0.06

0.28

0.24

3

-0.01

-0.12

0.11

3

0.13

0.17

0.26

0.17

0.23

3

0.03

0.04

0.12

0.23

0.29

0.52

0.M)

0.10

0.06

4

-0.13

4

0.16

0.03

0.37

0.00

0.11

4

0.25

0.24

0.26

0.08

0.3 1

5

0.07

0.48

0.12

0.14

0.06

5

0.12

0.27

0.31

0.25

0.14

5

0.07

0.00

0.30

0.03

0.33

5

0.17

0.24

0.20

0.46

0.09

5

0.07

0.25

0.00

0.07

5

0.00

0.27

0.12

0.02

0.02

2

- 0.04

0.11

0.09

0.61

0.19

2

-0.02

0.04

0.45

0.22

0.06

0.07

0.58

0.06

0.12

0.62

-0.02 0.02

-0.01

2

0.05

2

0.06

2

0.26

0.39

7

0.11

0.50

7

0.16

0.58

0.06

0.10

0.05

7

0.11

0.50

0.10

0.22

0.04

I 7

0.12 0.18

0.38 0.56

0.13 -0.07

0.07 -0.01

0.14 0.04

-0.10

0.11 -0.04

-0.02 - 0.04

0.34 -0.16

Measuring

rationality

independent Table2.

of social desirability

307

contd Varimax

Item No. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44.

Item

of the test and scoring

instructions

factor

b t; E

b z G

z 5 i;

8 e 6

8 8

0.24 0.22

0.15 0.25

- 0.06 -0.17

0.56 0.31

- 0.03 - 0.06

8 8

0.18 0.08

-0.04 0.25

0.28 0.14

0.23 0. I7

0.23 0.15

9

0.14

-0.08

0.38

9 IO

0.06 0.61

0.10 0.23

0.07 0.19

IO

0.50

0.13

0.29

-0.08

0.13

IO IO

0.03 0.76

0.07 0.05

0.13 0.16

0.23 -0.02

0.08 0.13

IO

0.66

0.07

0.17

0.09

0.06

IO IO

0.54 0.57

0.17 0.18

0.19 0.29

0.04 - 0.02

0.13 0.08

are available

2 ;j G

loadings

Irrational idea No.

A person never learns from his/her mistakes. Life is what you make it. Unhappy childhoods inevitably lead to problems in adult life. I try not to brood over past mistakes. People who are selfish make me mad because they really should not be that way. If 1 had to nag someone to get what I wanted 1 would not think it was worth the trouble. I frequently find that life is boring. 1 often wish that something new and exciting would happen. I experience life as just the same old thing from day-to-day. I often wish life were more stimulating. 1 often feel that everything is tiresome and dull. I wish I could change places with someone who lives an exciting life. I often wish life were diflerent than it is.

Copies

rotated

-0.01 0.06 0.15

0.04 0.22 0.03

from the authors.

samples. To determine whether there had been any shrinkage in the validity of the 44-item test, Study No. 1 data was utilized and correlations computed between 44-item scale scores and validity criteria. Subjects The Ss for the second study were 177 adults (78 male, 97 female). The mean age of this sample was 27 years of age. Results

Results showed that there was a moderately high correlation (r = 0.34, P < 0.01) between the Marlowe-Crowne Scale and the Rationality Test. A Sex x Trials (normal vs ‘fake adjusted’ instructional set) analysis of variance, with Rationality Scale scores as a dependent variable, revealed significant trials (F = 51.45, df = 1,175; P -c 0.001) and Sex x Trials interaction effects (F = 6.93, df = 1,175; P < 0.01). Under the instructions to ‘fake adjusted’ Ss gave more rational responses with females changing in the direction of more rational responses to a greater extent than males. Using a cut-off point of four or more on the Eysenck Lie Scale, 12 Ss (6.8%) were identified as liars under the normal instructional set while 68 Ss (38%) were identified as liars under the ‘fake adjusted’ instructional set. In an attempt to reduce the social-desirability content of the Rationality Test correlations were computed between Rationality Scale items and the Marlowe-Crowne Scale. Fourteen items having a correlation of over 0.20 with the Marlowe-Crowne Scale were then eliminated from the test. The shorter version of the test had Hoyt estimates of reliability of 0.87 and 0.86 in the two samples with the mean scores being 151.4 and 146.7. The correlation between the final 44item test and social desirability was 0.24. To determine whether the reduced scale had suffered any drop in convergent validity correlations were computed again, using the Study No. 1 sample, between the 44-item scale and the validity criteria. No appreciable drop in the magnitude of these correlations occurred (see Table 1). I’.*.,.,,

3 3-F

308

GORDON E. BARNESand BRENT A. VLJLCANO

STUDY

No. 3

Method Procedure

In order to determine the factor structure of the final 44item test the two samples from Study No. 1 and Study No. 2 were combined providing 349 cases. Scores on the 44-item test in this total sample were then subjected to factor analysis using the principal axes technique. Results

The principal axes factor analysis produced five factors with eigenvalues 2 1 (6.7, 2.0, 1.8, 1.5, 1.0) with the first factor accounting for 51.9% of the common variance. The first five factors were rotated using the Varimax criterion of simple structure. The factor loadings of the 44 Rationality Scale items on the five rotated factors are provided in Table 2. The first factor to emerge in the rotated-factor solution appeared to be measuring irrational idea No. 10 (i.e. the idea that you can achieve maximum human happiness by inertia and inaction or by passively and uncommittedly enjoying oneself). All seven of the items measuring irrational idea No. 10 loaded appreciably (i.e. loadings greater than 0.30) on this factor while none of the other scale items loaded on this factor. A person who scored high on this factor would not be bored or dissatisfied with their life. On the second factor in the rotated solution all of the items measuring irrational idea No. 7 (i.e. the idea that you can more easily avoid facing many life difficulties and self-responsibilities than undertake more rewarding forms of self-discipline) loaded appreciably with two other items (Nos 13 and 16) which also seem to measure selfdiscipline, weighting highly on this factor. On the third factor in the rotated solution most of the items written to measure irrational idea No. 6 (i.e. idea that if something seems dangerous or fearsome, a person must be preoccupied with it and make oneself anxious about it) loaded appreciably. This factor seems to measure a tendency to think negatively. Items 14 and 36, which had fairly high loadings on this factor, also seem to be measuring this tendency. Factors four and five were less clearly defined with fewer items loading appreciably on these factors. No interpretation of these factors will be attempted at this time. GENERAL

DISCUSSION

Results of the original scale construction study (Study No. 1) showed that a highly reliable 5%item test had been constructed. Considerable evidence was also found for the convergent validity of the test. The high negative correlations found between rationality and neuroticism and depression were in agreement with rational emotive theory and the results of prior research (e.g. MacDonald and Games, 1972) showing irrational beliefs to be associated with poorer adjustment and Eysenck’s Neuroticism in particular. The finding that endorsing irrational beliefs was associated with having an external locus of control has also been reported in prior research (MacDonald and Games, 1972). Other findings, that holding irrational beliefs was associated with low ego-strength and having more fears, were predicted from, and are consistent with rational emotive theory. Study No. 2 was primarily concerned with determining and improving the discriminant validity of the Rationality Test. Eliminating items that were highly correlated with the Marlowe-Crowne Scale from the Rationality Test reduced the correlations of the overall scale with the Marlowe-Crowne without reducing the convergent validity of the test. The remaining social-desirability content in the scale probably does not constitute a serious problem for users of the BVRT. In situations where it is absolutely essential to eliminate all hints of social desirability, in the test the use of a Lie Scale would prove somewhat helpful. Where possible, another strategy for eliminating social desirability would be to include a measure of social desirability ‘in research and remove socialdesirability content by using analysis of covariance.

Measuring

Factor principal

analysis of the 44item component accounting

rationality

independent

of social desirability

309

Rationality Test revealed that there was a fairly large for over 50% of the variance in the test. The Varimax

rotated factor solutions showed that there were three factors that could be interpreted clearly with two factors having a less well-defined factor structure. Future developmental work is required on the Rationality Test. Research needs to be conducted to show that the scale can discriminate clinical from abnormal populations. The test should also show improvement (increased rationality) after treatment particularly if the therapy is rational emotive therapy or some similar type of cognitive therapy. More research also seems required in order to determine the stability of the factor structure and whether or not constructing subscales within the Rationality Test would be warranted. A~~noMlrtlyemmrs~During No. 6607-I 155-48.

this project

G.E.B.

was supported

by National

Health

Research

Scholar

Award

REFERENCES ARCABRITE A. H. and NIDORF L. J. (1968) Fifteen questions for rating reason. Rufional Liciny 3(l). 9-l 1. BARD J. A. (1973) A self rating scale for rationality. Rurioncll Lirimg 8( 1). 19. BARRON F. (1953) An ego-strength scale which predicts response to psychotherapy. J. consulr. fsy~hol. 17(5), 227-233. CAMPBELL D. T. (1960) Recommendations for APA test standards regarding construct, trait and discriminant validity. Am. Psycho/. 15, 546553. CAMPBELL D. T. and FISKE D. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psycho/. Bull. 56, 81-105. CRABTREE R. G. and WARD G. (1975) A note on three RET-related scales. Paper presented at the West Virginia Academy of Sciences, April 1975. CROWNF. D. P. and MARLOWE D. (1960) A new scale of social desirability independent of psychopathology. J. ccmulr. Ps@ml. 24(4), 349-354. ELLIS A. and HARPER R. A. (1961) A Guide to Rational Lioing. Wilshire Book Co., Hollywood, California. EYSENCK H. J. and EYSENCK S. B. G. (1968) Manual ,fbr tile Eyso& Persmtality ~r~renrory. Educational’ g, Industrial Testing Service, San Diego. Fox E. E. and DAVIES R. L. (1971) Test your rationality. Rational Lacing 5(Z). 23-25. HIGGINBOTHAM T. E. (1976) Irrationality m college students. Rarionul Liring II(l). 34. JONES R. G. (1969) A factored measure of Ellis’ irrational belief system. with personality and maladjustment correlates (Doctoral dissertation. Texas Technological College). University Microfilms. No. 69-6443. Ann Arbor, Michigan. LAUGHRIDGE S. T. (1971) A test of irrational thinking as it relates to psychological maladjustment. Unpublished masters thesis. University of Oregon. MACDOXALD A. P. JR and GAMES R. G. (1972) Ellis’ irrational values: a validation study. Roriorurl Lirimq 7(2). 25 -28.

MARTIN L. M., DOLLIUER R. H. and IRVIN J. A. (1977) A construct validity study of five measures of irrational beliefs. Rational Living 12(l). MURPHY R. and ELLIS A. (1976) Rationulit) Scalrs: A Bibliography. Institute for Ratlonal Living. New York. PEARSON P. H. (1970) Relationships between global and specified measures of novelty seeking. J. comult. cliu. Psycho/. 34, 199-204. PLUTCHIK R. (1976) The self-inventory: a measure of irrational attitudes and behaviour. Rutiorml Liriuy ll( I). 31-33. REID D. W. and WARE E. E. (1974) Multidimensionality of internal vs. external locus of control: addition of a third dimension and nondistinction of self versus others. Cm. J. hehor. Sci. 6(2). 131-142. SHORKEY C. T. and WHITEMAN V. L. (1977) Development of the rational behaviour inventory: initial validity and reliability. Educ. psycho/. Mrasur. 37, 527-533. WHITEMAN V. L. and SHORKEY C. T. (1978) Validation testing of the rational behaviour inventory. Educ. psychol. Measur. 38. 1143-l 149. WOLPE J. and LANG P. J. (1964) A fear survey schedule for use in behaviour therapy. B&c. Rrs. T/W. 2. 27-30. ZUNG W. W. K. (1965) A self-rating depression scale. Archs gen. Psyclliut. 12, 63-70.