- Email: [email protected]
The effects of computer-mediated vocational guidance procedures on the appropriateness of vocational preference
Journal of Vocational
Behavior,
6, 61-72 (1975)
The Effects of Computer-Mediated Vocational Guidance Procedures on the Appropriateness of Vocational Preference’ I* GUY T. PILATO The Pennsylvania State Universi@ and ROGER A. MYERS Teachers College, Columbia Universiv
To assess the effectiveness of vocational guidance procedures proposed for inclusion in a computer-based vocational exploration system, 139 eleventh-grade male subjects were randomly assigned to three experimental groups and a control group. One group was given a computergenerated accuracy of self-knowledge feedback, another group was taught an occupational classification scheme, a third group experienced both. Utilizing a method of explicit comparison between students’ measured characteristics and the measured characteristics of occupational norm groups, the results showed students in the groups receiving feedback increased in the appropriateness of the occupational level of their fast occupational choice. In addition, students in the combined treatment group increased in the appropriateness of the level of their total occupational preferences. A delayed posttest indicated that these increases persisted.
Assistance in educational and vocational decision making should lead to the client’s selection of better goals for his vocational future. This assertion is so basic to vocational guidance activity it has persisted despite several thoughtful warnings about the methodological difficulties encountered when one attempts to test it (Gonyea, 1963; Hewer, 1966; Super, 1961; Super, Kowalski, & Gotkin, 1967). The concept of goodness of goals has not been an easy one to translate into satisfactory criteria, but researchers have continued to work at the problem for more than three decades. Their efforts to index *This study is part of the fust author’s unpublished doctoral dissertation at Columbia University, for which the second author was chairman. The authors are indebted to Donald E. Super, Jean Pierre Jordaan. and E. Behin Williams, committee members. The contributions of Frank J. Minor is also acknowledged gratefully. *Request for reprints should be sent to Guy T. Pilato, The Mental Health Center, Ritenour Health Center, The Pennsylvania State University, University Park, Pennsylvania 16802.
61 Copyright @ 1975 by Academic Press, Inc. All rights of reproduction in any form reserved.
62
PILATO
AND MYERS
approprirrteness(or realism, or wisdom) of vocational preference (or choice) have led to a solution that is at least partially acceptable. The early attempts (Hutson & Webster, 1943; Nick, 1942) to operationalize appropriateness of vocational preference consisted of comparing a distribution of the preferences of a group of subjects to a distribution of actual employment opportunities. Movement from dissimilarity to similarity between the two types of distributions was presumedto represent increasein appropriateness. Because this procedure involved some tenuous assumptions and because it provided no measure of individual change, its use has been abandoned. A second method of measuring appropriateness has involved judges’ ratings of the congruence of clients’ characteristics with the presumed requirements of their preferred occupations. A number of investigators have employed this procedure in assessingthe outcomes of vocational guidance treatments (Biersdorf, 1968; Bilovsky, McMasters, Shorr & Singer, 1953; Conyea, 1962; Hewer, 1959; Hoyt, 1955; Speer& Jasker, 1949; Stone, 1948). Its main weaknesseshave seemedto be its reliance on the expertness of the judges about the goodnessof fit, and, as Hewer (1966) made clear, in the reliability of the ratings. A third attack on the problem obviates the assumption that an expert can decide about goodnessof fit and provides a completely reliable measure of appropriateness.It consists of an explicit comparison between a client’s measured characteristics and the measuredcharacteristics of the norm group of the occupation to which he aspires.Though the limits of existing normative data make this procedure less than totally satisfactory, there are bases for assumingthat correspondencebetween intelligence and occupational level and between interests and field of work can be meaningfully scored (Jordaan & Myers, 1970; Super & Crites, 1962; Tyler, 1964). A number of investigators (Bidwell, 1969; Margolis, 1967; Milliken, 1962; Super, 1961) have used this procedure with apparent success. The present study was designedto evaluate certain vocational guidance procedures prior to inclusion in a computer-based vocational exploration system. The central question was whether subjects exposed to these procedures improved in appropriatenessof vocational preference. METHOD Subjects. Subjects of this study were 139 eleventh-grademales from a high school in a suburban community near New York City. The community is primarily middle to upper-middle class,with a wide range of income. Subjects rangedin age from 15 to 18; the median agewas 16.
COMPUTER-MEDIATED GUIDANCE
63
Instruments. In addition to two questionnaires designed to elicit selfestimates of learning ability and vocational interests (see Pilato & Myers, 1973), each subject was administered an intelligence test, an interest inventory, and a questionnaire about his occupational preferences. 1. Intelligence was measured by the Vochdu~ Test-GT (1960) a 20-word multiple-choice, wide range vocabulary test (see Pilato & Myers, 1973). 2. Interests were measured by the Kuder General Interest Survey, Form E (Kuder, 1964). 3. Occupational preferences were elicited with an Occupational PreferenceQuestionnaire which asked the following two questions: 1. When you think about how you could earn your living when you leave school or college, what occupations are in your lit of possibilities? (Take into account your abilities, preferences, and probable opportunities.) Write them down here. 2. Of the possibilities which you have listed, which one do you think is most likely to be your future occupation?
Procedure. Data were collected at three times. The pretest included the Kuder E, the Vocabulary Test, and Occupational Preference Questionnaire, and the two selfestimate questionnaires. Immediately following the treatments, i.e., 13 days after the pretest, an immediate posttest consisting of a readministration of the Occupational Preference Questionnaire took place. Forty-two days after the immediate posttest, the delayed posttest was conducted using the sameinstrument. Appropriate Verification scores on the Kuder E were obtained for 128 of the 139 subjects. These 128 were randomly divided into three treatment groups and one control group of 32 subjects each. Minor adjustments in the randomly assignedgroups were made in order to equate the means of the groups on the Vocabulary Test. These adjustments required the moving of three subjects. Experimental group 1 (El) was given accuracy of self-knowledge feedback comprised of a subject’s selfestimate of his learning ability compared with his actual intelligence quartile score as measuredby the Vocabulary Test, and of his estimates of the strength of his vocational interests contrasted with his scores on the Kuder. Feedback was given in the form of a computer generatedprint-out containing each subject’s selfestimates, his test results, and a statement of the degreeand direction of the differences (seePilato & Myers, 1973). An experimenter explained the meaning of the print-out, instructed the subjects to study it for 10 min, collected the forms, then administered the Occupational Preference Questionnaire for the second time (i.e., immediate posttest). Experimental group 2 (E2) was not given the accuracy of self-knowledge
64
PILATO
AND MYERS
feedback. Instead, the subjects were introduced to the structure of the world of work in the form of a system called “The Modified Miner-Roe Occupational Classification” (Pilate, 1969, pp. 109-112). The subjects were given a guided reading experience of materials which described the classification constructed by listing occupations within eight field categories in the same manner as outlined by Roe (1956). Occupations were listed within four levels, however, on the basis of Miner’s (1957) direct measurements of intelligence levels among occupational groups. At the end of the session the immediate posttest data was collected. Experimental group 3 (E3) received a combined treatment consisting of both accuracy of self-knowledge feedback and the teaching of the occupational classification plus advice as to how the two could be related. Afterwards the immediate posttest data were collected. The Control group (C) undertook a “fdler exercise” which was designed to have occupational relevance without effecting changes in the dependent variables. The exercise consisted of working through a programmed booklet about the career paths of surveyors, followed by the collection of immediate posttest data. The delayed posttest data were collected from all subjects simultaneously 42 days after the immediate posttests. Scoring Appropriateness of Vocational Preference Appropriateness of vocational preference was measured in four ways. The scoring method for each is described below. 1. Appropriateness of vocational preferences-level. The use of the plural “preferences” in subsequent discussions and in the tables refers to all of the choices which a student listed under the first question of the Occupational Preference Questionnaire. This usage will distinguish such discussion from that dealing with the first choice occupation elicited in the second question. As a point of information, the average number of preferences per student in answer to the first question was 2.64, with a range from six choices to one. There was high consistency in the number and range of preferences across groups and across testing times. Appropriateness of vocational preferences by level was measured by categorizing the student’s preferences according to level. For each preference the student’s raw score on the Vocabulary Test-CT was converted to a standard score based on the corresponding occupational level norm group reported in Miner’s (1957) study. Finally, the average of all the individual appropriateness scores was computed. The conversion into standard scores was guided by the assumption that a person with a high intelligence is acting inappropriately by preferring a low
COMPUTER-MEDIATED GUIDANCE
65
level occupation. The scoring procedure needed, therefore., to avoid automatically assigninghigher scoresto students with higher intelligence. Instead, appropriatenesswas computed in terms of congruence with the mean of the appropriate occupational level norm group. The mean (standard score= 100, SD = 10) was considered the highest score. Thus, standard scoresabove the mean decreased, rather than increased, in value; what would normally be computed as a standard score of 110 was instead regardedas a score of 90. Logic imposed two exceptions to the above rule. When a student whose intelligence exceeded the Level I norm’ group mean indicated a preferencefor a Level I occupation, it could not be said that his score tended in the direction of inappropriateness, since there are no higher occupational levels to which he could aspire. For the sake of comparability among levels, all such scores were given the high score of 100. Conversely in the case of Level IV preferences,it could not be maintained that scores below the Level IV norm group mean tended in the direction of inappropriateness,becausethere are no lower levels toward which to aim. Thus for Level IV preferences,scoresbelow the mean were also kept at 100. 2. Appropriateness of vocational preferences-field. Appropriateness of vocational preferencesby field was measuredby first categorizing a student’s preferencesaccording to the occupational field. The score for appropriateness was then computed by taking the average of his scores on those Kuder interests which, in terms of Roe’s (1956) scheme, were related to the preferred occupational fields. Only standard scoreswere used; they were based on the 9-12th grade Kuder norm group. The mean of the scoreswas set, as in the caseof preferencesby level, at 100 and the standard deviation at 10. The procedure differed from that employed for determining appropriateness of vocational preferences by level in that scores above the mean were scored positively. That is, the higher the scoreson the relevant interest scales, the higher the appropriatenessscore. 3. Appropriateness of first choice preference-level. The procedure for scoring appropriatenessof first choice by level was the same as that used for all preferences,except that the operation was confined to the one preference. When no first preference was noted, the computation was based on the occupational level most often mentioned among the total preferences. 4. Appropriateness of fzrst choice preference-field. Again the scoring procedure was the sameas in the caseof the total number of preferences,but basedon one preference only. Reliabilities
The reliabilities of the main variables were computed from the scoresof the control group over the three testing periods. They are presented in Table 1.
66
PILATO AND MYERS TABLE 1 Coefficients of Stability of the Dependent Variables for the Control Group Only (N = 32) After 13 days
After 55 days
.99
.91
2. Appropriateness of Vocational Preferences-Field
.93
.91
3. Appropriateness of First Choice-Level
.91
.97
4. Appropriateness of First Choice-Field
.94
.91
Variable 1. Appropriateness of Vocational Preferences-Level
TABLE 2 (Xi-Square Analysis: Frequency of Changes in Appropriateness of Vocational Preferences-Level (df= 3) Test 1 to Test 2 Increase Group
Yes
Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 3 Increase
No
Test 2 to Test 3 Increase
Yes
No
Yes
No
2 I 6 11
30 25 25 20
1 4 6 4
31 28 25 21
4 28 7 25 8 23 15 16 x2 = 11.06**
x” = 8.28*
x’ = 4.03
*p < .05. **p < .02.
TABLE 3 (X-Square Analyses: Comparisons Between Groups on Changes in Appropriateness of Vocational Preferences-Level (df= 1) Group comparisons CvsE, C vs E2 C vs E3 E, vs E3 Ez vs E3 E, vsEz *p < .Ol.
Test 1 to Test 2
Test 1 to Test 3
X1
X2
0.99 0.78 9.62* 0.78 3.39 0.13
3.23 2.44 8.22* I .48 2.03 0.06
67
COMPUTER-MEDIATED GUIDANCE TABLE 4 Means and Standard Deviations of Changes in Appropriateness of Vocational Preferences-Field
Group Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 2
Test 1 to Test 3
Test 2 to Test 3
Mean
SD
Mean
SD
Mean
SD
1.13 -0.16 0.35 0.14
3.02 4.01 4.16 4.07
1.65 0.06 -0.16 1.29
3.36 5.00 4.03 4.26
0.52 0.22 -0.52 0.54
2.24 5.63 3.32 3.35
RESULTS The four categories of appropriateness of vocational preference are treated separately in what follows. For all categories,however, the expectations were the same. That is, it was expected that each of the treatment groups would show greater increasesin appropriatenessthan would the control group. Furthermore, it was expected that subjects in the combined treatment group (E3) would exceed the changes in either of the other two treatment groups, accuracy of self-knowledge feedback (El) and teaching of occupational classification (E2). Al predicted differences were expected in time periods from test 1 to test 2 and from test 1 to test 3. Appropriateness of Vocational heferences-Level
Since the distribution of change scores was heavily skewed in the direction of no change, the nonparametric chi-square technique was used to evalua#ethe treatment effects on appropriatenessof vocational preferencesby level. Table 2 presents the results of the overali analysis of differences. It indicates that there were significant differences among the four groups in the period rom test 1 to test 2 and from test 1 to test 3. Table 3 reports the &i-square analyses of the differences between individual groups for the two time periods in which overall significance was detected. The only significant differences observed were between the combined treatment group (E3) and the control group for both time periods. Appropriateness of Vocational Preferences-Field
Table 4 presents descriptive data concerning changesin the appropriateness of vocational preferences by field. Table 5 reports the results of the analysis of variance for each of the three time periods. It is evident that no significant changesresulted from any of the treatments.
68
PILATO AND MYERS TABLE 5 One-Way Analyses of Variance: Changes in Appropriateness Of Vocational References-Field Source of variation
Period
df
MS
F
Test 1 to Test 2
Treatments Error
3 120
9.42 15.41
0.61
Test 1 to Test 3
Treatments Error
3 120
24.61 18.26
1.35
Test 2 to Test 3
Treatments Error
3 120
1.59 15.24
.50
TABLE 6 Chi-Square Analysis: Number of Students Who Changed in Appropriateness of First Choice Preference-Level (df = 3) Test 1 to Test 2 Increase Group Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 3 Increase
Yes
No
Yes
0 4 1 6
32 28 30 25
Test 2 to Test 3 Increase
No
Yes
No
0
0
4 1 I
3 0 3
32 29 31 28
32 28 30 24 X * = 11.26**
x2 = 9.20*
x2 = 6.30
*p < .05. **p < .02.
TABLE 7 Chi-Square Analyses: Comparisons Between Groups on Changes in Appropriateness of First Choice Preference-Level (df = 1) Group comparisons CvsE, cvs El-2 CvsEs El vs Es E2 vs Es E, vs Es *p < .05.
***p < .Ol.
Test 1 to Test 2 X2
4.27* 1.00 6.85*** 0.55 4.02 1.85
Test 1 to Test 3 X2
4.21* 1.05 8.13*** 1.85 5.17* 1.85
69
COMPUTER-MEDIATED GUIDANCE TABLE 8 Means and Standard Deviations of Changes in Appropriateness of First Choice Preference-Field Test 1 to Test 3
Test 1 to Test 2 Group Control Experimental 1 Experimental 2 Experimental 3
Test 2 to Test 3
Mean
SD
Mean
SD
Mean
SD
0.26 0.45 0.42 0.26
3.47 6.83 6.64 3.51
1.00 -2.61 -0.19 1.25
4.23 8.51 5.75 5.19
0.74 -3.07 -0.61 0.96
4.23 8.18 3.23 4.57
TABLE 9 One-Way Analyses of Variance: Changes in Appropriateness of First Choice Preference-Field
Period
Source of variation
df
MS
F
Test 1 to Test 2
Treatments Error
3 120
0.33 29.74
0.01
Test 1 to Test 3
Treatments Error
3 120
95.92 38.83
2.47
Test 2 to Test 3
Treatments Error
8 120
106.29 29.97
3.55*
*p < .05. Appropriateness of First Choice Reference-Level For appropriateness of first choice occupation by level, the differences were tested by chi-square technique. Table 6 shows that there was a statistically significant difference among the four groups. Table 7 presents a further breakdown. As in the case of appropriateness of total preferences by level, the results show that the subjects in the combined treatment showed significantly more change than the controls. In this instance, however, the results reveal that subjects in the accuracy of self-knowledge feedback group also showed more changes than the controls. Furthermore, the combined treatment was significantly more effective than the teaching of occupational classification treatment. Appropriateness of First Choice Preference-Field Table 8 presents descriptive data on change scores for appropriateness of first choice preference by field. Table 9 reports the results of the analysis of variance.
70
PILATO AND MYERS
The results of the analysis of variance show no significant differences among the groups for the time periods for which the hypotheseswere made. Significant difference did appear, however, for the period from the immediate posttest to the delayed posttest. Post hoc contrasts computed by the Scheffe Method (1959) showed one significant difference at the .05 level, between El and E3 (.08 < 0 < 7.98). This difference appearsto be the function of a large decrease in appropriateness within the group receiving accuracy of selfknowledge feedback as compared to a slight gain among the group receiving the combined treatment. To summarizethe findings reported above: 1. Subjects receiving the accuracy of self-knowledgefeedback (El) did not increase in appropriatenessof all preferencestaken together, but did increasemore than controls on appropriatenessof first choice by level. The changespersisted through the delayed posttest. 2. Subjects who were taught the occupational classification alone (E2) did not increasemore than controls on any categoriesof appropriateness. 3. Subjects receiving the combined treatment (E3) showed more increasesthan the controls in appropriatenessby level, for both total preferencesand first choice. The changespersisted. 4. Subjects in the combined treatment group did not exceed those in the other two treatment groups in increasing appropriatenessexcept in one of four comparisons (E3 > E2 for appropriateness of first choice by level). 5. None of the treatment conditions were related to differential changes in appropriatenessby field. DISCUSSION The central question of this study was whether the vocational guidance procedures employed could be shown to have sufficient efficacy to warrant their inclusion in a computer-based vocational exploration system. The findings, and those previously reported (Pilato & Myers, 1973), do argue for the efficacy of the combined treatment, which can be regarded as a direct translation of the time-honored prescription of Parsons(1909), more recently endorsedby Calia (1966). In combining self study and occupational study, the intent is to establish a reciprocal relationship in which one serves as an impetus for the learning of the other. Used in isolation, the accuracy of self-knowledgefeedbackcondition has some value. It has been shown to influence both accuracy of self-knowledge about intelligence (Pilato & Myers, 1973) and appropriatenessof first preference by level. Its failure to promote changesin subjects’ knowledge about
COMPUTER-MEDIATED GUIDANCE
71
their interests and in their appropriatenesswhen considered by field, suggests that modifications are clearly needed. When taken by itself, the teaching of an occupational classification schemehad no observable effects on either self-knowledgeor appropriateness. No one has reason to be surprised by this. Brayfield (1951) argued long ago that mere dissemination of occupational information should be discardedand that the use of such information should be a highly personalized, individualized matter. Furthermore, the use of occupational titles without descriptions may have weakened the treatment, as Osipow’s (1962) research suggested.Subjectsmay have been prevented from indicating more appropriate preferences by sheer lack of knowledge about the occupations. Nevertheless, when it is combined with self-knowledge feedback, the teaching about occupations does seem to add something valuable to the power of the procedure. REFERENCES Bidwell, G. P. Ego strength, self-knowledge, and vocational planning of schizophrenics. Journal of Counseling Psychology, 1969, 16,4549. Biersdorf, K. R. The effectiveness of two group vocational guidance treatments. Unpublished doctoral dissertation, University of Maryland, 1958. Bilovsky, D., McMasters, W., Shorr, J. E., & Singer, S. L. Individual and group counseling. Personnel and Guidance Journal, 1953, 31, 363-365. Brayfield, A. H. Dissemination of occupational information. Occupations, 1951, 29, 411413. Calia, V. F. Vocational guidance: after the fall. Personnel and Guidance Journal, 1966, 45, 320-321. Gonyea, G. C. Appropriatenessof-vocational-choice as a criterion of counseling outcome. Journal of Counseling Psychology, 1962,9, 213-219. Gonyea, G. C. Appropriatenessof-vocationalchoice of counseled and uncounseled college students. Journal of Counseling Psychology, 1963, 10, 269-275. Hewer, V. H. Group counseling, individual counseling, and a college class in vocations. Personnel and Guidance Journal, 1959, 37, 660-665. Hewer, V. H. Evaluation of a criterion: realism of vocational choice. Journal of Counseling Psychology, 1956, 13, 289-294. Hoyt, D. P. An evaluation of group and individual programs in vocational guidance. Journal of Applied Psychology, 1955, 39, 26-30. Hutson, P. V. & Webster, A. D. An experiment in the education and vocational guidance of tenth-grade pupils. Educational and Psychological Measurement, 1943, 3, 3-21. Jordaan, J. P. & Myers, R. A. Individual differences in the world of work. In J. R. Davitz & S. Ball (Eds.), Psychology of the educational process. New York: McGraw-Hill, 1970. Pp. 391-427. Kuder, G. F. Kuder E General Interest Survey Manual. Chicago: Science Research Associates, 1964. Margolis, V. H. Kuder-Strong discrepancy in relation to conflict and congruence of vocational preference. Unpublished doctoral dissertation, Columbia University, 1967.
72
PILATO AND MYERS
Millikin, R. L. Realistic occupational appraisal by high school seniors. Personnel and Guidance Journal, 1962,40,541-544. Miner, J. B. IntelZigence in the United S?ates. New York: Springer, 1957. Nick, E. W. High school boys choose vocations. Occupations, 1942,20, 264-269. Osipow, S. H. Perceptions of occupations as a function of titles and descriptions. Journal of Counseling Psychology, 1962, 9, 106-109. Parsons, F. Choosing a vocation. Boston: Houghton Mifflin, 1909. guidance treatments on some aspects of Pilato, G. T. The effects of three voc~tiod vocational preference and self-knowledge. (Doctoral dissertation, Columbia University) Ann Arbor, Michigan: University Microfdms, 1969, No. 69-3088. Pilato, G. T. & Myers, R. A. Effects ‘of computer-mediated vocational guidance procedures: Accuracy of self-knowledge. Journal of Vocational Behavior, 1973, 3, 167-174. Roe, A. The psychology of occupations. New York: Wiley, 1956. Speer, G. S. & Jasker, L. The influence of occupational information on occupational goals. Occupations, 1949,28, 15-17. Stone, H. C. Are vocational orientation courses worth their salt? Educational and Psychological Measurement, 1948, 8, 161-18 1. Super, D. E. Consistency and wisdom of vocational preferences as indices of vocational maturity in the ninth grade. Journal of Educational Psychology, 1961, 52, 3543. Super, D. E. & Crites, J. 0. Appraising vocational jJtness. (Rev. Ed.) New York: Harper & Row, 1962. Super, D. E., Kowalski, R. S., & Gotkin, E. H. Floundering and trial after high school. Cooperative Research Project, 1967, No. 1393. Tyler, L. E. Work and individual differences. In Henry Borow (Ed.), Man in Q world at work. Boston: Houghton Mifflin, 1964. Pp. 174-195. Vocabulary Test-CT, directions norms. New York: Institute of Psychological Research, 1960. Received: January 7, 1974.
Behavior,
6, 61-72 (1975)
The Effects of Computer-Mediated Vocational Guidance Procedures on the Appropriateness of Vocational Preference’ I* GUY T. PILATO The Pennsylvania State Universi@ and ROGER A. MYERS Teachers College, Columbia Universiv
To assess the effectiveness of vocational guidance procedures proposed for inclusion in a computer-based vocational exploration system, 139 eleventh-grade male subjects were randomly assigned to three experimental groups and a control group. One group was given a computergenerated accuracy of self-knowledge feedback, another group was taught an occupational classification scheme, a third group experienced both. Utilizing a method of explicit comparison between students’ measured characteristics and the measured characteristics of occupational norm groups, the results showed students in the groups receiving feedback increased in the appropriateness of the occupational level of their fast occupational choice. In addition, students in the combined treatment group increased in the appropriateness of the level of their total occupational preferences. A delayed posttest indicated that these increases persisted.
Assistance in educational and vocational decision making should lead to the client’s selection of better goals for his vocational future. This assertion is so basic to vocational guidance activity it has persisted despite several thoughtful warnings about the methodological difficulties encountered when one attempts to test it (Gonyea, 1963; Hewer, 1966; Super, 1961; Super, Kowalski, & Gotkin, 1967). The concept of goodness of goals has not been an easy one to translate into satisfactory criteria, but researchers have continued to work at the problem for more than three decades. Their efforts to index *This study is part of the fust author’s unpublished doctoral dissertation at Columbia University, for which the second author was chairman. The authors are indebted to Donald E. Super, Jean Pierre Jordaan. and E. Behin Williams, committee members. The contributions of Frank J. Minor is also acknowledged gratefully. *Request for reprints should be sent to Guy T. Pilato, The Mental Health Center, Ritenour Health Center, The Pennsylvania State University, University Park, Pennsylvania 16802.
61 Copyright @ 1975 by Academic Press, Inc. All rights of reproduction in any form reserved.
62
PILATO
AND MYERS
approprirrteness(or realism, or wisdom) of vocational preference (or choice) have led to a solution that is at least partially acceptable. The early attempts (Hutson & Webster, 1943; Nick, 1942) to operationalize appropriateness of vocational preference consisted of comparing a distribution of the preferences of a group of subjects to a distribution of actual employment opportunities. Movement from dissimilarity to similarity between the two types of distributions was presumedto represent increasein appropriateness. Because this procedure involved some tenuous assumptions and because it provided no measure of individual change, its use has been abandoned. A second method of measuring appropriateness has involved judges’ ratings of the congruence of clients’ characteristics with the presumed requirements of their preferred occupations. A number of investigators have employed this procedure in assessingthe outcomes of vocational guidance treatments (Biersdorf, 1968; Bilovsky, McMasters, Shorr & Singer, 1953; Conyea, 1962; Hewer, 1959; Hoyt, 1955; Speer& Jasker, 1949; Stone, 1948). Its main weaknesseshave seemedto be its reliance on the expertness of the judges about the goodnessof fit, and, as Hewer (1966) made clear, in the reliability of the ratings. A third attack on the problem obviates the assumption that an expert can decide about goodnessof fit and provides a completely reliable measure of appropriateness.It consists of an explicit comparison between a client’s measured characteristics and the measuredcharacteristics of the norm group of the occupation to which he aspires.Though the limits of existing normative data make this procedure less than totally satisfactory, there are bases for assumingthat correspondencebetween intelligence and occupational level and between interests and field of work can be meaningfully scored (Jordaan & Myers, 1970; Super & Crites, 1962; Tyler, 1964). A number of investigators (Bidwell, 1969; Margolis, 1967; Milliken, 1962; Super, 1961) have used this procedure with apparent success. The present study was designedto evaluate certain vocational guidance procedures prior to inclusion in a computer-based vocational exploration system. The central question was whether subjects exposed to these procedures improved in appropriatenessof vocational preference. METHOD Subjects. Subjects of this study were 139 eleventh-grademales from a high school in a suburban community near New York City. The community is primarily middle to upper-middle class,with a wide range of income. Subjects rangedin age from 15 to 18; the median agewas 16.
COMPUTER-MEDIATED GUIDANCE
63
Instruments. In addition to two questionnaires designed to elicit selfestimates of learning ability and vocational interests (see Pilato & Myers, 1973), each subject was administered an intelligence test, an interest inventory, and a questionnaire about his occupational preferences. 1. Intelligence was measured by the Vochdu~ Test-GT (1960) a 20-word multiple-choice, wide range vocabulary test (see Pilato & Myers, 1973). 2. Interests were measured by the Kuder General Interest Survey, Form E (Kuder, 1964). 3. Occupational preferences were elicited with an Occupational PreferenceQuestionnaire which asked the following two questions: 1. When you think about how you could earn your living when you leave school or college, what occupations are in your lit of possibilities? (Take into account your abilities, preferences, and probable opportunities.) Write them down here. 2. Of the possibilities which you have listed, which one do you think is most likely to be your future occupation?
Procedure. Data were collected at three times. The pretest included the Kuder E, the Vocabulary Test, and Occupational Preference Questionnaire, and the two selfestimate questionnaires. Immediately following the treatments, i.e., 13 days after the pretest, an immediate posttest consisting of a readministration of the Occupational Preference Questionnaire took place. Forty-two days after the immediate posttest, the delayed posttest was conducted using the sameinstrument. Appropriate Verification scores on the Kuder E were obtained for 128 of the 139 subjects. These 128 were randomly divided into three treatment groups and one control group of 32 subjects each. Minor adjustments in the randomly assignedgroups were made in order to equate the means of the groups on the Vocabulary Test. These adjustments required the moving of three subjects. Experimental group 1 (El) was given accuracy of self-knowledge feedback comprised of a subject’s selfestimate of his learning ability compared with his actual intelligence quartile score as measuredby the Vocabulary Test, and of his estimates of the strength of his vocational interests contrasted with his scores on the Kuder. Feedback was given in the form of a computer generatedprint-out containing each subject’s selfestimates, his test results, and a statement of the degreeand direction of the differences (seePilato & Myers, 1973). An experimenter explained the meaning of the print-out, instructed the subjects to study it for 10 min, collected the forms, then administered the Occupational Preference Questionnaire for the second time (i.e., immediate posttest). Experimental group 2 (E2) was not given the accuracy of self-knowledge
64
PILATO
AND MYERS
feedback. Instead, the subjects were introduced to the structure of the world of work in the form of a system called “The Modified Miner-Roe Occupational Classification” (Pilate, 1969, pp. 109-112). The subjects were given a guided reading experience of materials which described the classification constructed by listing occupations within eight field categories in the same manner as outlined by Roe (1956). Occupations were listed within four levels, however, on the basis of Miner’s (1957) direct measurements of intelligence levels among occupational groups. At the end of the session the immediate posttest data was collected. Experimental group 3 (E3) received a combined treatment consisting of both accuracy of self-knowledge feedback and the teaching of the occupational classification plus advice as to how the two could be related. Afterwards the immediate posttest data were collected. The Control group (C) undertook a “fdler exercise” which was designed to have occupational relevance without effecting changes in the dependent variables. The exercise consisted of working through a programmed booklet about the career paths of surveyors, followed by the collection of immediate posttest data. The delayed posttest data were collected from all subjects simultaneously 42 days after the immediate posttests. Scoring Appropriateness of Vocational Preference Appropriateness of vocational preference was measured in four ways. The scoring method for each is described below. 1. Appropriateness of vocational preferences-level. The use of the plural “preferences” in subsequent discussions and in the tables refers to all of the choices which a student listed under the first question of the Occupational Preference Questionnaire. This usage will distinguish such discussion from that dealing with the first choice occupation elicited in the second question. As a point of information, the average number of preferences per student in answer to the first question was 2.64, with a range from six choices to one. There was high consistency in the number and range of preferences across groups and across testing times. Appropriateness of vocational preferences by level was measured by categorizing the student’s preferences according to level. For each preference the student’s raw score on the Vocabulary Test-CT was converted to a standard score based on the corresponding occupational level norm group reported in Miner’s (1957) study. Finally, the average of all the individual appropriateness scores was computed. The conversion into standard scores was guided by the assumption that a person with a high intelligence is acting inappropriately by preferring a low
COMPUTER-MEDIATED GUIDANCE
65
level occupation. The scoring procedure needed, therefore., to avoid automatically assigninghigher scoresto students with higher intelligence. Instead, appropriatenesswas computed in terms of congruence with the mean of the appropriate occupational level norm group. The mean (standard score= 100, SD = 10) was considered the highest score. Thus, standard scoresabove the mean decreased, rather than increased, in value; what would normally be computed as a standard score of 110 was instead regardedas a score of 90. Logic imposed two exceptions to the above rule. When a student whose intelligence exceeded the Level I norm’ group mean indicated a preferencefor a Level I occupation, it could not be said that his score tended in the direction of inappropriateness, since there are no higher occupational levels to which he could aspire. For the sake of comparability among levels, all such scores were given the high score of 100. Conversely in the case of Level IV preferences,it could not be maintained that scores below the Level IV norm group mean tended in the direction of inappropriateness,becausethere are no lower levels toward which to aim. Thus for Level IV preferences,scoresbelow the mean were also kept at 100. 2. Appropriateness of vocational preferences-field. Appropriateness of vocational preferencesby field was measuredby first categorizing a student’s preferencesaccording to the occupational field. The score for appropriateness was then computed by taking the average of his scores on those Kuder interests which, in terms of Roe’s (1956) scheme, were related to the preferred occupational fields. Only standard scoreswere used; they were based on the 9-12th grade Kuder norm group. The mean of the scoreswas set, as in the caseof preferencesby level, at 100 and the standard deviation at 10. The procedure differed from that employed for determining appropriateness of vocational preferences by level in that scores above the mean were scored positively. That is, the higher the scoreson the relevant interest scales, the higher the appropriatenessscore. 3. Appropriateness of first choice preference-level. The procedure for scoring appropriatenessof first choice by level was the same as that used for all preferences,except that the operation was confined to the one preference. When no first preference was noted, the computation was based on the occupational level most often mentioned among the total preferences. 4. Appropriateness of fzrst choice preference-field. Again the scoring procedure was the sameas in the caseof the total number of preferences,but basedon one preference only. Reliabilities
The reliabilities of the main variables were computed from the scoresof the control group over the three testing periods. They are presented in Table 1.
66
PILATO AND MYERS TABLE 1 Coefficients of Stability of the Dependent Variables for the Control Group Only (N = 32) After 13 days
After 55 days
.99
.91
2. Appropriateness of Vocational Preferences-Field
.93
.91
3. Appropriateness of First Choice-Level
.91
.97
4. Appropriateness of First Choice-Field
.94
.91
Variable 1. Appropriateness of Vocational Preferences-Level
TABLE 2 (Xi-Square Analysis: Frequency of Changes in Appropriateness of Vocational Preferences-Level (df= 3) Test 1 to Test 2 Increase Group
Yes
Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 3 Increase
No
Test 2 to Test 3 Increase
Yes
No
Yes
No
2 I 6 11
30 25 25 20
1 4 6 4
31 28 25 21
4 28 7 25 8 23 15 16 x2 = 11.06**
x” = 8.28*
x’ = 4.03
*p < .05. **p < .02.
TABLE 3 (X-Square Analyses: Comparisons Between Groups on Changes in Appropriateness of Vocational Preferences-Level (df= 1) Group comparisons CvsE, C vs E2 C vs E3 E, vs E3 Ez vs E3 E, vsEz *p < .Ol.
Test 1 to Test 2
Test 1 to Test 3
X1
X2
0.99 0.78 9.62* 0.78 3.39 0.13
3.23 2.44 8.22* I .48 2.03 0.06
67
COMPUTER-MEDIATED GUIDANCE TABLE 4 Means and Standard Deviations of Changes in Appropriateness of Vocational Preferences-Field
Group Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 2
Test 1 to Test 3
Test 2 to Test 3
Mean
SD
Mean
SD
Mean
SD
1.13 -0.16 0.35 0.14
3.02 4.01 4.16 4.07
1.65 0.06 -0.16 1.29
3.36 5.00 4.03 4.26
0.52 0.22 -0.52 0.54
2.24 5.63 3.32 3.35
RESULTS The four categories of appropriateness of vocational preference are treated separately in what follows. For all categories,however, the expectations were the same. That is, it was expected that each of the treatment groups would show greater increasesin appropriatenessthan would the control group. Furthermore, it was expected that subjects in the combined treatment group (E3) would exceed the changes in either of the other two treatment groups, accuracy of self-knowledge feedback (El) and teaching of occupational classification (E2). Al predicted differences were expected in time periods from test 1 to test 2 and from test 1 to test 3. Appropriateness of Vocational heferences-Level
Since the distribution of change scores was heavily skewed in the direction of no change, the nonparametric chi-square technique was used to evalua#ethe treatment effects on appropriatenessof vocational preferencesby level. Table 2 presents the results of the overali analysis of differences. It indicates that there were significant differences among the four groups in the period rom test 1 to test 2 and from test 1 to test 3. Table 3 reports the &i-square analyses of the differences between individual groups for the two time periods in which overall significance was detected. The only significant differences observed were between the combined treatment group (E3) and the control group for both time periods. Appropriateness of Vocational Preferences-Field
Table 4 presents descriptive data concerning changesin the appropriateness of vocational preferences by field. Table 5 reports the results of the analysis of variance for each of the three time periods. It is evident that no significant changesresulted from any of the treatments.
68
PILATO AND MYERS TABLE 5 One-Way Analyses of Variance: Changes in Appropriateness Of Vocational References-Field Source of variation
Period
df
MS
F
Test 1 to Test 2
Treatments Error
3 120
9.42 15.41
0.61
Test 1 to Test 3
Treatments Error
3 120
24.61 18.26
1.35
Test 2 to Test 3
Treatments Error
3 120
1.59 15.24
.50
TABLE 6 Chi-Square Analysis: Number of Students Who Changed in Appropriateness of First Choice Preference-Level (df = 3) Test 1 to Test 2 Increase Group Control Experimental 1 Experimental 2 Experimental 3
Test 1 to Test 3 Increase
Yes
No
Yes
0 4 1 6
32 28 30 25
Test 2 to Test 3 Increase
No
Yes
No
0
0
4 1 I
3 0 3
32 29 31 28
32 28 30 24 X * = 11.26**
x2 = 9.20*
x2 = 6.30
*p < .05. **p < .02.
TABLE 7 Chi-Square Analyses: Comparisons Between Groups on Changes in Appropriateness of First Choice Preference-Level (df = 1) Group comparisons CvsE, cvs El-2 CvsEs El vs Es E2 vs Es E, vs Es *p < .05.
***p < .Ol.
Test 1 to Test 2 X2
4.27* 1.00 6.85*** 0.55 4.02 1.85
Test 1 to Test 3 X2
4.21* 1.05 8.13*** 1.85 5.17* 1.85
69
COMPUTER-MEDIATED GUIDANCE TABLE 8 Means and Standard Deviations of Changes in Appropriateness of First Choice Preference-Field Test 1 to Test 3
Test 1 to Test 2 Group Control Experimental 1 Experimental 2 Experimental 3
Test 2 to Test 3
Mean
SD
Mean
SD
Mean
SD
0.26 0.45 0.42 0.26
3.47 6.83 6.64 3.51
1.00 -2.61 -0.19 1.25
4.23 8.51 5.75 5.19
0.74 -3.07 -0.61 0.96
4.23 8.18 3.23 4.57
TABLE 9 One-Way Analyses of Variance: Changes in Appropriateness of First Choice Preference-Field
Period
Source of variation
df
MS
F
Test 1 to Test 2
Treatments Error
3 120
0.33 29.74
0.01
Test 1 to Test 3
Treatments Error
3 120
95.92 38.83
2.47
Test 2 to Test 3
Treatments Error
8 120
106.29 29.97
3.55*
*p < .05. Appropriateness of First Choice Reference-Level For appropriateness of first choice occupation by level, the differences were tested by chi-square technique. Table 6 shows that there was a statistically significant difference among the four groups. Table 7 presents a further breakdown. As in the case of appropriateness of total preferences by level, the results show that the subjects in the combined treatment showed significantly more change than the controls. In this instance, however, the results reveal that subjects in the accuracy of self-knowledge feedback group also showed more changes than the controls. Furthermore, the combined treatment was significantly more effective than the teaching of occupational classification treatment. Appropriateness of First Choice Preference-Field Table 8 presents descriptive data on change scores for appropriateness of first choice preference by field. Table 9 reports the results of the analysis of variance.
70
PILATO AND MYERS
The results of the analysis of variance show no significant differences among the groups for the time periods for which the hypotheseswere made. Significant difference did appear, however, for the period from the immediate posttest to the delayed posttest. Post hoc contrasts computed by the Scheffe Method (1959) showed one significant difference at the .05 level, between El and E3 (.08 < 0 < 7.98). This difference appearsto be the function of a large decrease in appropriateness within the group receiving accuracy of selfknowledge feedback as compared to a slight gain among the group receiving the combined treatment. To summarizethe findings reported above: 1. Subjects receiving the accuracy of self-knowledgefeedback (El) did not increase in appropriatenessof all preferencestaken together, but did increasemore than controls on appropriatenessof first choice by level. The changespersisted through the delayed posttest. 2. Subjects who were taught the occupational classification alone (E2) did not increasemore than controls on any categoriesof appropriateness. 3. Subjects receiving the combined treatment (E3) showed more increasesthan the controls in appropriatenessby level, for both total preferencesand first choice. The changespersisted. 4. Subjects in the combined treatment group did not exceed those in the other two treatment groups in increasing appropriatenessexcept in one of four comparisons (E3 > E2 for appropriateness of first choice by level). 5. None of the treatment conditions were related to differential changes in appropriatenessby field. DISCUSSION The central question of this study was whether the vocational guidance procedures employed could be shown to have sufficient efficacy to warrant their inclusion in a computer-based vocational exploration system. The findings, and those previously reported (Pilato & Myers, 1973), do argue for the efficacy of the combined treatment, which can be regarded as a direct translation of the time-honored prescription of Parsons(1909), more recently endorsedby Calia (1966). In combining self study and occupational study, the intent is to establish a reciprocal relationship in which one serves as an impetus for the learning of the other. Used in isolation, the accuracy of self-knowledgefeedbackcondition has some value. It has been shown to influence both accuracy of self-knowledge about intelligence (Pilato & Myers, 1973) and appropriatenessof first preference by level. Its failure to promote changesin subjects’ knowledge about
COMPUTER-MEDIATED GUIDANCE
71
their interests and in their appropriatenesswhen considered by field, suggests that modifications are clearly needed. When taken by itself, the teaching of an occupational classification schemehad no observable effects on either self-knowledgeor appropriateness. No one has reason to be surprised by this. Brayfield (1951) argued long ago that mere dissemination of occupational information should be discardedand that the use of such information should be a highly personalized, individualized matter. Furthermore, the use of occupational titles without descriptions may have weakened the treatment, as Osipow’s (1962) research suggested.Subjectsmay have been prevented from indicating more appropriate preferences by sheer lack of knowledge about the occupations. Nevertheless, when it is combined with self-knowledge feedback, the teaching about occupations does seem to add something valuable to the power of the procedure. REFERENCES Bidwell, G. P. Ego strength, self-knowledge, and vocational planning of schizophrenics. Journal of Counseling Psychology, 1969, 16,4549. Biersdorf, K. R. The effectiveness of two group vocational guidance treatments. Unpublished doctoral dissertation, University of Maryland, 1958. Bilovsky, D., McMasters, W., Shorr, J. E., & Singer, S. L. Individual and group counseling. Personnel and Guidance Journal, 1953, 31, 363-365. Brayfield, A. H. Dissemination of occupational information. Occupations, 1951, 29, 411413. Calia, V. F. Vocational guidance: after the fall. Personnel and Guidance Journal, 1966, 45, 320-321. Gonyea, G. C. Appropriatenessof-vocational-choice as a criterion of counseling outcome. Journal of Counseling Psychology, 1962,9, 213-219. Gonyea, G. C. Appropriatenessof-vocationalchoice of counseled and uncounseled college students. Journal of Counseling Psychology, 1963, 10, 269-275. Hewer, V. H. Group counseling, individual counseling, and a college class in vocations. Personnel and Guidance Journal, 1959, 37, 660-665. Hewer, V. H. Evaluation of a criterion: realism of vocational choice. Journal of Counseling Psychology, 1956, 13, 289-294. Hoyt, D. P. An evaluation of group and individual programs in vocational guidance. Journal of Applied Psychology, 1955, 39, 26-30. Hutson, P. V. & Webster, A. D. An experiment in the education and vocational guidance of tenth-grade pupils. Educational and Psychological Measurement, 1943, 3, 3-21. Jordaan, J. P. & Myers, R. A. Individual differences in the world of work. In J. R. Davitz & S. Ball (Eds.), Psychology of the educational process. New York: McGraw-Hill, 1970. Pp. 391-427. Kuder, G. F. Kuder E General Interest Survey Manual. Chicago: Science Research Associates, 1964. Margolis, V. H. Kuder-Strong discrepancy in relation to conflict and congruence of vocational preference. Unpublished doctoral dissertation, Columbia University, 1967.
72
PILATO AND MYERS
Millikin, R. L. Realistic occupational appraisal by high school seniors. Personnel and Guidance Journal, 1962,40,541-544. Miner, J. B. IntelZigence in the United S?ates. New York: Springer, 1957. Nick, E. W. High school boys choose vocations. Occupations, 1942,20, 264-269. Osipow, S. H. Perceptions of occupations as a function of titles and descriptions. Journal of Counseling Psychology, 1962, 9, 106-109. Parsons, F. Choosing a vocation. Boston: Houghton Mifflin, 1909. guidance treatments on some aspects of Pilato, G. T. The effects of three voc~tiod vocational preference and self-knowledge. (Doctoral dissertation, Columbia University) Ann Arbor, Michigan: University Microfdms, 1969, No. 69-3088. Pilato, G. T. & Myers, R. A. Effects ‘of computer-mediated vocational guidance procedures: Accuracy of self-knowledge. Journal of Vocational Behavior, 1973, 3, 167-174. Roe, A. The psychology of occupations. New York: Wiley, 1956. Speer, G. S. & Jasker, L. The influence of occupational information on occupational goals. Occupations, 1949,28, 15-17. Stone, H. C. Are vocational orientation courses worth their salt? Educational and Psychological Measurement, 1948, 8, 161-18 1. Super, D. E. Consistency and wisdom of vocational preferences as indices of vocational maturity in the ninth grade. Journal of Educational Psychology, 1961, 52, 3543. Super, D. E. & Crites, J. 0. Appraising vocational jJtness. (Rev. Ed.) New York: Harper & Row, 1962. Super, D. E., Kowalski, R. S., & Gotkin, E. H. Floundering and trial after high school. Cooperative Research Project, 1967, No. 1393. Tyler, L. E. Work and individual differences. In Henry Borow (Ed.), Man in Q world at work. Boston: Houghton Mifflin, 1964. Pp. 174-195. Vocabulary Test-CT, directions norms. New York: Institute of Psychological Research, 1960. Received: January 7, 1974.