Validity of self-ratings of abilities and competencies

Journal of Vocational

Behavior 31, l-13 (1987)

Validity of Self-Ratings of Abilities and Competencies RODNEY

L. LOWMAN

Divisions of Medical Psychology and Occupational Duke University Medical Center

Medicine,

AND RICHARD Colorado

E. WILLIAMS State University

This study examined the validity of self-estimates of ability on a vocational interest measure (the Self-Directed Search). Self-ratings of 149 female undergraduates were correlated with well-validated measures of abilities corresponding to each of Holland’s six interest domains. Employing a multitrait, multimethod methodology, the study found a pattern of, at best, moderate correlations between self-ratings of ability and the objective ratings. Higher correlations were found between self-ratings of abilities and objective measures than for self-ratings of competencies on the Self-Directed Search. However, for both areas there were findings contrary to predictions. Implications for the use of self-ratings in the measurement of vocational interests and related constructs are discussed. o 1%~ Academic

Press, Inc.

Despite what might be imagined as obvious biases toward inaccuracy and misrepresentation of self-ratings of abilities and other personal characteristics, they are commonly used in many self-report inventories, selection devices, and research instruments (Williams, 1985). While some individual studies (e.g., Levine, Flory, & Ash, 1977) have reported strong associations between self-assessments and objective abiity The research reported in this paper was supported in part by Faculty Research Grant 34404, North Texas State University, where both authors were affiliated at the time the data reported in this article was collected. The following assisted with data collection efforts: Gordon Leeman, Don Olive, Jackie Cruickshank, and Donna Vanek. The authors express thanks to John Holland and to anonymous reviewers for helpful comments on earlier drafts of this paper. Correspondence and requests for reprints should be- sent to Rodney L. Lowman, Ph.D., Division of Occupational Medicine, Duke University Medical Center, Box 2914, Durham, NC 27710. 1 OOOl-8791/87 $3.00 Copyright Q PI87 by Academic Press, Inc. Au rights of reproduction in any form reserved.

2

LOWMAN

AND

WILLIAMS

measures, other investigators have strongly challenged the appropriateness of self-ratings of work performance, finding such ratings to be more lenient, to have more bias, and less discriminant validity than supervisory ratings (e.g., Thornton, 1978). Mabe and West’s (1982) review of 55 studies found a mean correlation of from .29 to .31 (depending on the meta-analytic methodology employed) between self-ratings and objective ratings of ability. They further reported the accuracy of self-ratings of ability, the focus of the present article, to vary with a number of factors, including intelligence, locus of control, type of ability rated, and the context in which self-evaluation occurs. Primoff (1979) also noted the importance of the ratee defining the dimensions on which self-rating is to occur in the same manner as that intended by the examiner. Lunneborg (1982) found high school students to over- or underrate themselves on ability depending on interest and occupational sex stereotypes, among other variables. Measures of vocational interest, especially the widely used Self-Directed Search, incorporate self-ratings of abilities as an important part of their measurement of vocational preferences. Although vocational interest instruments such as the SDS are seldom used as exclusive appraisal measures in personnel selection contexts, they generally constitute an important component of career counseling assessment. The accuracy of the selfreporting on such measures is therefore an important factor in determining their validity. Holland’s Self-Directed Search (SDS) (1985) is, by any standard, one of the most widely used measures of vocational interests. It relies on self-reports of abilities in two of its subcomponents: Self-Estimates and Competencies. Although the number of items varies among the four subcomponents of the SDS, the scoring system of assigning relative weight to each area is such that the four areas are approximately equally weighted in determining the summary scores on the Self-Directed Search. The validity with which self-ratings of Self-Estimates and Competencies are completed is therefore not a trivial matter in determining the effectiveness of the SDS in properly classifying respondents. Studies examining the validity of self-ratings on the SDS have focused on correlations with achievement test results (Baird, 1%9), independent self-ratings either of perceived abilities or of actual accomplishments (Holland, 1963, 1964, 1968), scholastic aptitude test scores (Holland, 1963; Holland & Nichols, 1964), and measures of overall intellectual ability (Schaefer, 1976). Perhaps the most extensive examination of the validity of self-ratings on the SDS to date was that of Kelso, Holland, and Gottfredson (1977), who examined the association between SDS selfratings and scores on the Armed Services Vocational Aptitude Battery (ASVAB). While finding a pattern of low to moderate overlap between objective measures and self-ratings, this study is limited by not measuring

VALIDITY

OF SELF-RATINGS

3

abilities across the full range of Holland “types” (i.e., no measures of musical, artistic, social, or managerial abilities, among others). The purpose of the present study was to examine the validity of selfratings of abilities in a sample of college women when abilities relevant to each of the six Holland personality types were assessed independently using well-validated measures of relevant abilities. This study thus provides the first comprehensive analysis of the relationship between self-ratings of abilities and competencies and objective measures of similar constructs. METHOD Subjects Female college students (n = 149), aged 18-25, were subjects for this study. All participants were undergraduates at a large, state-supported university located near a large metropolitan area in the southwest United States. Participants were recruited by newspaper announcements, notices posted on campus, and appeals made to classes. In order to ensure a heterogeneous sample of interests and abilties, students from all areas of campus were recruited to take part in the investigation. In return for their participation in the project, detailed feedback of individual results on the various tests was provided. A few subjects also obtained extra credit points for participation. Instruments Abilities measured in this study were selected on the basis of their correspondence to each of the six Holland occupational types. Because limited literature exists empirically establishing the relationship of specific abilities to Holland personality types, expert judgment was used to select abilities theoretically expected to correspond to each of the six Holland types. Table 1 summarizes the abilities measured in the study, the Holland self-rating scale to which the abilities were judged to correspond, and the specific measure of ability employed in this study. When available, instruments with demonstrated reliability and validity and long research histories were chosen. Briefly, the measures employed were as follows: Realistic. The Bennet Mechanical Comprehension Test (BMCT) requires respondents to identify pictorially a correct response from among several possibilities to a question or issue concerning mechanical and physical principles (Bennett, 1%9, 1980). The Revised Minnesota Paper Form Board Test (Form AA) (MPFB) is a classic measure of spatial abilities which are required in many aspects of mechanical intelligence (Likert & Quasha, 1970). Investigative. The Raven Standard Progressive Matrices (RSPM) (Raven, Court, & Raven 1977) is a nonverbal measure of general intelligence, emphasizing fluid intellectual ability. It requires verbal and (especially)

4

LOWMAN

Operationalized Holland type and SDS self-ratings” Realistic Mechanical ability Manual skills Investigative Scientific ability Math ability Artistic Musical ability

Artistic ability

AND WILLIAMS

TABLE 1 Ability Measures for the Six Holland Areas

Relevant ability Mechanical comprehension Spatial ability

General intelligence Verbal reasoning

Musical ability

Esthetic judgment

Social Teaching ability Friendliness

Interpersonal

Enterprising Sales ability Managerial skills

Group leader skills

Conventional Clerical ability Offlce skills

Clerical ability Computational ability

skills

Operational measure(s) Bennett Mechanical Comprehension Test Minnesota Paper Form Board Raven’s Standard Progressive Matrices Critical Thinking Appraisal Seashore Measures of Musical Talents Pitch Rhythm Tonal Memory Meier Art Judgment Test Interpersonal Social Abilities Test Leaderless Group Discussion Minnesota Clerical Test Numbers Names Wide Range Achievement Test (Arithmetic subtest)

Mean

SD

41.75

6.05

43.47

9.09

50.82

4.98

56.11

8.86

38.96 27.59 25.48

7.95 2.20 4.63

93.95

10.45

12.97

3.02

1.72

1.12

123.16 125.15

22.08 25.86

27.13

4.86

a Item stems from Self-Directed Search self-estimates. See Holland (1985).

nonverbal reasoning, and is self-administered. No reading skills are required. The Watson-Glaser Critical Thinking Appraisal (CTA) measures verbal reasoning abilities, inference making, ability to abstract and make verbal generalizations (Watson & Glaser, 1980). Artistic. The Meier Art Judgment Test (MAJT), another classic measure, was designed to measure esthetic judgment. This aptitude is thought to be associated with artistic accomplishment in many different areas of artistic expression (Meier, 1942). The Seashore Measures of Musical

VALIDITY

OF

SELF-RATINGS

5

Talent (Pitch, Rhythm, and Tonal Memory subtests) (SMMT-P, SMMTR, SMMT-TM) has been used for many years as a measure of various aspects of musical talent (Seashore, Lewis, & Saetveit, 1960). The three subtests were selected for this study because they have been found to be most reliable and theoretically sound among the various Seashore measures. Social. The Interpersonal Problem Solving Assessment (College Form) (IPSAT) is a paper-and-pencil measure of social abilities (Getter & Nowinski, 1981). Respondents are presented with a series of social situations to which they must indicate their most likely way of handling the situation. Enterprising. A Leaderless Group Discussion (LGD) was employed as a measure of leadership ability. The LGD is one of the most widely used measures in the assessment of leadership talent and requires observation of actual behavior in an unstructured group which must reach consensus on an assigned discussion topic. The LGD had demonstrated excellent validity and reliability in a variety of studies (Anastasi, 1982). Conventional. The Minnesota Clerical Test (Name Comparison and Number Comparison Subtests) (Andrew, Paterson, & Longstaff, 1979) measure two aspects of perceptual speed and accuracy both of which are commonly demanded in conventional-type employment. The Wide Range Achievement Test (Arithmetic subtest) (WRAT-A) measures basic competency in performing arithmetic processes, which are presumed by Holland (1985) to be associated with the Conventional type. Further information about the instruments’ reliabilities and validities is provided in an earlier, related article (Lowman, Williams, & Leeman, 1985). Procedures

The tests were administered in a group setting in two administrations. The following order of test administrations was used for all participants: Self-Directed Search (SDS), Wide Range Achievement Test-Arithmetic (WRAT-A), Minnesota Paper Form Board (MPFB), Critical Thinking Appraisal (CTA), Seashore Measures of Musical Talent (SMMT), and Interpersonal Ability Test (IPSAT) in the first of two testing sessions, and the Minnesota Clerical Test (MCT), Raven’s Standard Progressive Matrices (RSPM), Bennett Mechanical Comprehension Test (BMCT), Leaderless Group Discussion (LGD), and the Meier Art Judgment Test (MAJT) in the second session. The elapsed time between testing sessions averaged 2 weeks. No more than 10 participants were tested at a time in the second testing session to allow accurate assessment of each subject’s behavior during the LGD. Each test was administered using standardized instructions from the test manuals. The LGD was introduced using instructions developed specifically for this study (qv, Lowman et al., 1985).

6

LOWMAN

AND WILLIAMS

RESULTS Table 2 displays multitrait, multimethod (MTMM) (Campbell & Fiske, 1959) correlations for the self-ratings on the SDS compared to the objective ratings on the various measures of ability. MTMM matrices are useful in evaluating the validity of a particular construct (in this case, of selfratings of ability), because they address issues both of convergent and discriminant validity and of reliability. MTMM matrices can be evaluated by the extent to which they meet the following criteria: (a) the reliability coefficients (monotrait, monomethod) should be higher than other correlations in the matrix; (b) the validity coefficients (monotrait, heteromethod) should be sufficiently high to encourage further research and exceed their adjacent row and column correlations (heterotrait-heteromethod triangles); (c) the same pattern of correlations should be found in all heterotrait triangles, both for the monomethod and heteromethod approaches; and (d) higher correlations would be expected between the same traits measured by different methods than for different traits measured by the same methods (Campbell & Fiske, 1959). In examining the correlation matrices for the self-ratings displayed in Table 2, the reliabilities for the various measures are generally acceptably high and exceed the other correlation coefficients. (Exceptions to this generalization are the internal consistency estimates for Artistic and Social areas on the SDS. Though higher than the correlations in their respective rows and columns, these reliabilities are below desired levels. It should also be noted, however, that the SDS self-rating reliabilities are based on only two items.) The validity diagonals (monotrait, heteromethod correlations), although in all but two cases statistically significant (mean validity coefficient = + .23), generally but not always exceed the heterotrait, heteromethod correlations in their respective rows and columns. The desired pattern is not supported for the Minnesota Paper Form Board (Realistic), MCTNames, or WRAT-A (Conventional). Exceptions to a generally supportive pattern were found for the Watson-Glaser (Investigative), and the Meier Art Judgment Test (MAJT). The Social and Enterprising criterion measures met the desired pattern very well except for being more highly correlated with each other than Holland’s theory would predict. Most consistent with the ideal validity pattern were the correlations for the Seashore musical talents measures. The patterns of correlation in the heterotrait and heteromethod triangles generally follow the same pattern for all the triangles, for the most part mirroring the Holland interest pattern in which adjacent types are correlated more highly than nonadjacent ones as in Holland’s (1985) hexagonal model. However, there are exceptions to this pattern, especially for some of the ability intercorrelations (see also Lowman et al., 1985).

+I

II Is* 04

SMMT-TM IPSAT LGD

MCT-NU MCT-NA

14* 17* 45**

26** -11 16;

03 19* 10

17* 26:’ 1n* 35**

04 12

-05

18**

03 16:

23** 46** -07 13*

26** 16* 45**

22** 21** 17*

(45) 18* 18* -15*

A

BMCT WC

RAVEN

MAJT

07

01 06

11 01 19** 18*

-01 -07 -09

-02 07 13

00

-05 -01

-07 -14* 26’* 21**

-17* -04 -14*

-07 -03 00

m**

16* 05

-04 -05 -05 -04

03 -11 -02

- 14* -09 03

28** -07 06

20** II 08 22’S 37**

19s 35** 01

39** 50** 30”’ 44s”

16: 36** 47** 32** II 36** IO

(57)

49**

(85)

38**

04 27**

32** 21** 22** 03 32**

(73) 45** 12

-07 07

18* 42**

14* -03

-04

(77) Ia* 05 Is*

23** OS

26** 35** -OS

(93) 27** 27**

-11 23** -01 04 26’*

59**

(83) 27**

Ratings

Matrix

- 14* 06 00 07 17:

48**

(6-O

SMMT-R

Multimethod

SMMT-P

MPFB

2

B. Objective (94)

C

TABLE

Abilities: Multitrait,

42;’ 06

(62) 19*

E

of

(36)

S

of abilities

Self-Estimates

-08 09 03 18* 26**

(83)

SMMT-TM

of Ability

(95) 17* 01 05 00

IPSAT

(92) -02 02 13

IDD

(80) 6l** 17*

MCT-NU

(85) 32**

MCT-NA

WI

WRAT-R

** P 5 .Ol.

Now. n = 149; Boldface correlations are the coefficients in the validity diagonal. Reliabilities are in parentheses. Abbreviations: R-Realistic; lInvestigative; A-Artistic; !&Social; E-Enterprising; C-Conventional; MPFB-Minnesota Paper Form Board; BMCT-Bennett Mechanical Comprehension Test: WG-Watson Glaser Critical Thinking Appraisal; RAVEN-Raven’s Standard Progressive Matrices: MAJT-Meier Art Judgment Test; SMMT-P, -R, -TM-Seashore Measures of Musical Talent: Pitch, Rhythm, Tonal Memory; lPSAT-Interpersonal Problem Solving Assessment Technique; LGDLeaderless Group Discussion; MCT-NU, NA-Minnesota Clerical Test, Numbers, Names; WRAT-A-Wide Range Achievement Test-Arithmetic. * P I .os.

WRAT-A

03 02 06

MAJT SMMT-P SMMT-R

06 Zfi** 07

07 01

00

(61) 21**

16* 12 14* -05

BMCT WG RAVEN

MPFB

Investigative Artistic Social Enterprising

I

R

(73) SO** 37**

A. Self-ratings

SDS

8

LOWMAN

AND WILLIAMS

Competencies. The MTMM matrices for competencies are presented in Table 3. There is less support for the convergent and discriminant validity of the competencies than for the self-estimates ratings when using objective measures of ability as the criterion. Here, the validity coefficients in the diagonal (monotrait, heteromethod method) averaged only + .14, with only 4 of the 13 coefficients reaching statistical significance. Specifically, it is only the SMMT subtests and the LGD which meet the desired pattern. In addition, the IPSAT and the MCT-NA generally are consistent with the validity requirements except for the lack of differentiation between the Social and Enterprising measures and the fact that the validity coefficient for the MCT-NU was not statistically significant. The patterns of correlations within the various triangles are generally consistent except as previously noted for the ability correlation matrix. Finally, the correlations of each of the four subcomponents of the SDS with the respective objective measures of ability were tabulated to provide an overview of the pattern of relationships. Table 4 summarizes the correlations found between each of the four subcomponents of the SDS and the corresponding objective ability measure across the six Holland areas. For example, the first entry in Table 4 (r = .lO) is the average of the correlation between the Realistic activities score and respondents’ scores on the Minnesota Paper Form Board and the correlation between the Realistic activities score and the Bennett Mechanical Comprehension Test. Although this method of analysis has limitations (e.g., if an inappropriate measure was selected as a criterion, this problem is magnified in the summary matrix), it does provide a global overview of a large number of individual correlations. Table 4 demonstrates that self-estimates had the strongest pattern of correlations with objective measures compared to activities, competencies and occupations. In no case, however, did the average correlation exceed the teens. Moreover, there was a differential pattern of association across the six types (column averages in Table 4), suggesting that individuals may have varying abilities to self-assess accurately, depending on the nature of what is being self-evaluated. The “best” results were obtained for Artistic and Enterprising abilities; the “worst” for Realistic, Social, and Conventional. DISCUSSION Of the four major subparts of the SDS, two (abilities and competencies) rely on self-ratings of ability. There was a pattern of mixed support for the validity of each of these when well-validated measures of ability in each area were employed. Particularly weak in its support was the validity of the self-ratings of competencies. The least ambiguous pattern of association occurred in the case of artistic measures (Seashore scales and the MAJT). Perhaps it is easier

Q

(S)

(El (C)

R

02 II -01 06 16* 06 - IO 04 02 06 01 10 -08

(77) 35** 41** - 10 2v* 26**

01 04 08 10 03 00 -02 06 -03 07 06 06 16;

175) 26** 04 17* 20**

I

(60) 42'* 16*

s

26** -01 03 -07

01 05 -01 12

23** 08 -06 -03

2s** 22** 30** 04 21**

-08 OS 03

-05 -01 -03 15”

- IS’ -02

12 13

01 - 13* -05

-04 -02

(79) 21'

E

Competencies

16' -05 IO -04 OS . -02

33** 34** 20**

(68)

A

of

3

05 11

02 04 04 03 I1

03 - I2 -01

-07 00

- 16*

(77)

C

20** II 08 22**

36** IO

36** 47** 32** II

(85) 49** 16*

MPFB

06 37**

35** 01 28’* -07

50** 30** 44** 19*

(87) 39**

BMCT

27** 38**

21** 22** 03 32*” 04

45** 12 32**

(73)

wo

IS* 42**

35** -05 23** 05

27** 26**

(93) 27**

RAVEN

-03 -07 07

05 15* -04 14’

(77) IF

MAJT

04 26**

23** -01

-II

59**

(83) 27**

SMMT-P

B. Objective

07 17:

06 00

- 14*

(67) 48**

SMMTF

Ratings

IS* 26**

09 03

(83) -08

SMMT-TM

of Ability

Multimethod Matrix

05 Mt

17* 01

(95)

WSAT

02 13

-02

6Q** 17*

(80)

MCT-NV

(92)

LOO

(85) 32**

MCT-NA

(94

WRAT-R

* P I .os. ** P I .Ol.

Note. n = 149; Boldface correlations are the coefficients in the validity diagonal. Reliabilities are in parentheses. Abbreviations: R-Realistic; IInvestigative; A-Artistic; S-Sock& &Enterprising; C-Conventional; MPFB-Minnesota Paper Form Board; BMCT-Bennett Mechanical Comprehension Test; WG-Watson Glaser Critical Thinking Appraisal; RAVEN-Raven’s Standard Progressive Matrices; MAJT-Meier Art Judgment Test; SMMT-P, -R, -TM-Seashore Measures of Musical Talent: Pitch, Rhythm, Tonal Memory; IPSAT-Interpersonal Problem Solving Assessment Technique; LGDLeaderless Group Discussion; MCT-NU, NA-Minnesota Clerical Test, Numbers, Names; WRAT-A-Wide Range Achievement Test-Arithmetic.

MCT-N U MCT-NA WRAT-A

SMMT-TM IPSAT LGD

SMMT-R

RAVEN MAJT SMMT-P

BMCT WG

MPFB

Enterprising Conventional

Social

Artistic (A)

Realistic (R) Investigative (I)

A. Self-Ratings

TABLE

SDS Self-Estimates of Competencies: Multitrait,

10

LOWMAN

AND WILLIAMS

TABLE 4 Summary of the Average Correlations Between Objective Ability Measures and Corresponding Self-Directed Search Subcomponents R Activities Competencies Occupations Self-estimates Column average

10 06 02 16 8.5

I

A

01 05 10 26

29 22 14 33

12

22

Note. n = 149. Decimal points omitted. R-Realistic; Social; E-Enterprising; C-Conventional.

E

C

Row average

10 12 00 19

25 26 0.5 20

08 10 10 16

14.8 13.5 06.8 21.1

10.3

19

11

S

I-Investigative;

A-Artistic;

S-

to self-rate oneself accurately in an area perceived to be narrowly distributed within the population and in which not having the ability is not intricately tied to self-esteem (e.g., “musical ability” versus “friendliness” or academic intelligence). It should also be noted that the correlations between self-estimates and abilities may be underestimated by the particular manner in which they are measured on the SDS. For each of the six Holland areas, the respondent is self-rated on only two abilities (e.g., mechanical ability and manual skills for the Realistic area). In contrast, occupations include 14 items for each area and competencies and activities, 11. It is certainly possible that the six interest areas may be associated with a much broader span of abilities than the two self-rated abilities for each Holland area on the SDS. In the Investigative area, for example, a future social scientist might not rate highly on “math ability,” or even “scientific ability,” yet still be Investigative. Similarly, “sales ability” and “managerial skills,” the two self-estimates for the Enterprising domain, appear to be rating two distinct and hardly comprehensive aspects of managerial talent (see, for example, Bray, Campbell, & Grant, 1974). It must also be noted that in the case of some of the primary abilities employed in this study, a clear relationship between the primary ability measured and the Holland type has not been established. For example, the MPFB was thought to measure an ability called for in the implementation of Realistic interests; in fact, it appears that these abilities may be more closely associated, at least for women, with Artistic and to a lesser extent Investigative abilities (Lowman et al., 1985). Thus, the absence of a clear relationship between the self-rated ability and the objective measure may in some cases be less because of instrumentation problems than because of the absence of a demonstrated link between the ability measured in this study and the self-rating on the SDS. Contrasting with these factors potentially attenuating the relationship

VALIDITY

OF SELF-RATINGS

11

of self-ratings and actual abilities is the finding that persons of higher intelligence tend to be more accurate in self-ratings (Mabe & West, 1982). In that this sample consisted exclusively of college women (while the SDS is directed to the general population), accuracy of self-estimates, if anything, should have been overestimated when compared to the general public. Further, the context for this testing was a research study in which there was no formal stake (other than feedback) in the outcome; there was far less to be gained by self-misrepresentation in this study than had the SDS been employed in a selection or career counseling context. This study has demonstrated that, as measured using the Self-Directed Search, the validity of self-ratings of primary abilities as part of a vocational interest exploration measure has not clearly been established. At best, moderate relationships were found between the self-rating and the actual ability, and oftentimes the pattern of association was contrary to expections. This implies that the inclusion of self-ratings on measures of vocational interests may be a questionable practice. This is not to imply, of course, that the Self-Directed Search or other measures which use self-ratings as a part of their instrumentation are invalid or inappropriate for use for their intended purpose. To the extent the Self-Directed Search is a measure of vocational interests, its validity must be judged by its overall effectiveness in describing and classifying individuals. There is evidence (e.g., Lowman et al., 1985) that vocational interests and abilities are separate domains. To the extent this finding is generalizable, separate instrumentation for measuring the two domains may be required. Further studies are needed to understand better the differences and the overlap between abilities and interests. A more complex question concerns the validity of the SDS summary scores, given the decidedly mixed pattern of validity support for the selfrated subscores. In fairness to the SDS, it must be noted that the appropriate criteria for validating an instrument of this nature are the summary scores, not individual sections of the instrument. The validity of the SDS (and therefore its usefulness in clinical practice) must be judged by the extent to which the summary scores have been demonstrated to be associated wth relevant criteria. In general, the validity findings for the SDS summary scores have been quite supportive of Holland’s (1985) model. However, it remains open to debate what precisely the SDS is measuring. To the extent that Holland’s SDS is a measure of vocational interests, there are both conceptual and empirical objections to the inclusion of self-estimates of ability. To the extent it is a “self-exploration device,” there is less basis for theoretical objection to including self-estimates of abilities, though the limited pattern of validity support for the self-rated abilities remains problematic. In general, it apppears that instruments like the SDS include selfestimates of abilities and competencies at some risk to validity. Test

12

LOWMAN

AND WILLIAMS

developers contemplating the inclusion of such ratings are probably best advised that, if they are to be included at all, self-estimates of abilities should be included on a highly selective basis but also with a sufficiently large number of items such that threats to validity are minimized. REFERENCES Anastasi, A. (1982). Essentials of psychological testing (5th ed.). New York: Macmillan CO. Andrew, D. M., Paterson, D. G., t Longstaff, H. P. (1979). Minnesota Clerical Test (manual). New York: Psychological Corporation. Baird, R. L. (1969). Testing Holland’s theory. Measurement and Evaluation in Guidance, 4, 107-l 14. Bennett, G. K. (1%9). Bennett Mechanical Comprehension Test (test manual). New York: Psychological Corporation. Bennett, G. K. (1980). Bennett Mechanical Comprehension Test (manual supplement). New York: Psychological Corporation. Bray, D. W., Campbell, R. J., & Grant, D. L. (1974). Formative years in business. A long-term AT&T study of managerial lives. Malabar, FL: Krieger. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. DeNisi, A. !I., & Shaw, J. B. (1977). Investigation of the uses of self-reports of abilities. Journal of Applied Psychology, 6X4), 641-644. Getter, H., & Nowinski, J. K. (1981). A free response test of interpersonal effectiveness. Journal of Personality Assessment, 45, 301-307. Gottfredson, G. D., & Holland, J. L. (1975). Some normative self-report data on activities, competencies, occupational preferences, and ability ratings for high school and college students and employed men and women. JSAS Catalog of Selected Documents in Psychology, 5, 192(Ms. No. 859). Hansford, B. C., & Hattie, J. A. (1982). The relationship between self and acheivemerit/performance measures. Review of Educational Research, 52(l), 123-142. Holland, J. L. (1%3). Explorations of a theory of vocational choice and achievement: II. A four-year prediction study. Psychological Reports, 12, 545-594. Holland, J. L. (1964). Explorations of a theory of vocational choice: V. A one year prediction study. Moravia, NY: Chronical Guidance Professional Service. Holland, J. L. (1968). Explorations of a theory of vocational choice: IV. A longitudinal study using a sample of typical college students. Journal of Applied Psychology, 52, l-37. Holland, J. L. (1985). Making vocational choices. A theory of vocational personalities and work ertvironments (2d ed.). Englewood Cliffs, NJ: Prentice-Hall. Holland, J. L., & Nichols, R. C. (1964). Explorations of a theory of vocational choice: III. A longitudinal study of change in major field of study. Personnel and Guidance Journal, 43, 235-242. Kelso, B. I., Holland, J. L., & Gottfredson, G. D. (1977). The relation of self-reported competencies to aptitude test scores. Journal of Vocational Behavior, 10, W-103. Levine, E. L., Flory, A. III, & Ash, R. A. (1977). Self-assessment in personnel selection. Journal of Applied Psychology, 6%4), 428-435. Likert, R., & Quasha, W. H. (1970). Revised Minnesota Paper Form Board Test (manual). New York: Psychological Corporation. Lowman, R. L., Williams, R. E., & Leeman, G. E. (1985). The structure and relationship of college women’s primary abilities and vocational interests. Journal of Vocational Behavior. 27, 298-315.

VALIDITY

OF SELF-RATINGS

13

Lunneborg, C. E. (1982). Systematic biases in brief self-ratings of vocational qualifications. Journal

of Vocational

Behavior,

20, 255-275.

Mabe, P. A. III, & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67(3), 280-2%. Meier, N. C. (1942). Art in human affairs. New York: McGraw-Hill. Primoff, E. S. (1979). The use of self-assessments in examining. Professional Series 79-l. Washington, DC: Personnel Research and Development Center, Office of Personnel Management. Raven, J. C., Court, J. H., & Raven, J. (1977). Manualfor Raven’s Progressive Matrices and Vocabulary Scales. Section 3. Standard Progressive Matrices. London: Lewis & Co., Ltd. Schaefer, B. E. (1976). Holland’s SDS: Is its effectiveness contingent upon selected variables? Journal of Vocational Behavior, 8, 113-123. Seashore, C. E., Lewis, D., & Saetveit, J. G. (1960). Seashore Measures of Musical Talents. New York: Psychological Corporation. Thornton, G. C., III, (1978, August). Psychometric properties of seIf-appraisals of job performance. Paper presented at the meeting of the American Psychological Association, Toronto. Watson, G., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. New York: Psychological Corporation. Williams, R. E. (1985). Holland’s Self-Directed Search: A measure of interests or abilities? Unpublished Master’s thesis, North Texas State University. Received: October 29, 1986.