Validity of college admission indices for Jews and Arabs in Israel

Person.

indiuid. DI@

Printed in Great

Vol.

Britain.

8, No. 4, pp. 587-589, All rights reserved

1987 Copyright

0

0191-8869/87 $3.00 +O.OO 1987 Pergamon Journals Ltd

Validity of college admission indices for Jews and Arabs in Israel MOSHE ZEIDNER School of Education, University of Haifa, Mt Carmel 31999, Israel (Received II August 1986) Summary-This study examines Israeli cultural group differences in scholastic aptitude and achievement indices as well as the differential construct and predictive validity of precollege admission indices for Jewish and Arab students in Israel. The sample was composed of 1778 Jewish and 1017 Arab student candidates applying for admission to a major university. The admission indices include five scholastic aptitude subtests appearing on both the Hebrew and Arabic versions of the college entrance test battery and matriculation certificate grades in three required subjects. The predictor battery shows comparable factor structure across cultural groups. The correlations between aptitude and achievement predictors and first-year college G.P.A. were homogeneous across the groups. A slight degree of intercept bias was evidenced with the predictive indices tending to overpredict the scholastic achievement of Arab students and underpredict that of their Jewish counterparts. The results are consistent with previous research in America showing negligible differences in the construct and predictive validity of college admission tests for different cultural and racial groups.

INTRODUCTION In the ongoing crusade for equality of educational and social opportunity for cultural minority groups in Israel, representatives of the Arab student population, which is seriously under-represented in the Israeli academic system (Goldman, 1980), have recently initiated a massive and vehement anti-test campaign. College admission criteria, particularly aptitude tests, are accused of being unfavorably biased against Arab candidates and therefore, it is contended, should be banned until more culturally fair measures are available (Nevo, 1985). Cultural bias in testing has recently become one of the most burning issues engaging the concern of Israeli psychologists. If the cultural bias contention is indeed tenable, one could hardly expect a better context for it to be evidenced than in the test scores of Jews vs Arabs in Israel. As is well agreed, Arab and Jewish subgroups differ on a wide array of sociocultural parameters, such as mother tongue, religion, customs and mores, norms, and social status. In fact, Arabs are commonly referred to as ‘minority group members’ by the media, and are generally deemed to be socially and educationally disadvantaged relative to their Jewish counterparts (Kleinberger, 1969). Thus, a test of the bias contention with respect to the two cultural groups under consideration provides a critical test of the cross-cultural validity of the bias hypothesis. Unfortunately, very little research has been devoted to systematically testing the cultural bias claim as pertains to the two most divergent and contrasted cultural groups in Israel. At present, few experts would feel confident in generalizing to other cultural contexts the results of the numerous studies of racial group differences (cf. Cleary, 1968, Jensen, 1980; 1984; Linn, 1973; Stanley and Porter, 1967), carried out in the American culture. Furthermore, very little is known about cultural group differences in college admission scholastic aptitude test performance in Israel. METHOD Subjects The original sample, deemed by university testing authorities to be representative of the student population tested during that year, comprised 1778 Jewish (39% males) and 1017 Arab (59% males) who applied for admission to a major Israeli university. The candidates ranged in age from 16 to 68 yr, with a mean age of 22.6 yr. Tests and procedures The college admission indices consist of both scholastic aptitude test scores as well as matriculation certificate grades. Scholastic aptitude tests were given as part of the precollege entrance requirements. The aptitude test measures are scores on the five standardized scholastic aptitude subscales, common to the Hebrew and Arabic versions of the college admission test battery, which were (a) general information, (b) figural reasoning, (c) mathematical reasoning, (d) analytical thinking, and (e) English language aptitude. In addition, the predictor set includes matriculation certificate grades in three subjects, required of both Jewish and Arab student candidates: Mathematics, English Proficiency, and Hebrew Language. Also, the first-year cumulative G.P.A.s were obtained for all members of the examinee pool who were registered as full-time university students within a period of 2 yr following test administration (about 43% of the original applicant pool). It was not feasible to match cultural groups with respect to majors or courses of study, particularly in view of the differential courses of study of Jewish and Arab groups, with a relatively higher proportion of Jewish students concentrated in the more rigorous quantitative courses of study (statistics, mathematics, computer sciences, economics, etc.).

Cultural group differences in scholastic aptitude and achievement Table 1 presents the means and SDS for scholastic aptitude and achievement of cultural effect size (sigma differences and point-biserial correlations). 587

predictors,

by culture,

along with estimates

588

NOTES

Table

AND

SHORTER

COMMUNICATIONS

I. A comparison of the scholastic aptitude test performance and matriculation for Jewish (N = 1778) vs Arab IN = 10171 student candidates Jewish

Aptitude .x&s Information Figures Mathematics Analytical English Composite Matriculation grades English Maths Hebrew Average

test scores

Arab

M

SD

M

SD

r;b

&t

52.22 52.64 51.25 52.59 52.94 53 20

10.09 9.05 10.48 9.61 IO.15 9.69

46. I3 45.3 I 47.79 45.39 44.86 44.41

8.57 9.05 8.68 8.97 7.30 7.x4

0.30 0.35 0.17 0.35 0.39 0.42

0.65 0.81 0.35 0.73 0.88 I .03

52.21 51.29 52. I3 50.68

9.37 9.09 8.39 9.85

46.14 47.75 46.27 48.20

9.90 I I .07 10.72 IO.10

0.29 0.17 0.28 0.10

0.63 0.35 0.60 0.25

*All point-biserial correlations are significant at P -C 0.01. td designates the mean differences (JewssArabs) divided by the average

wthm

group

SD.

Culture has a highly significant effect [1(2793) = 24.69, P c O.OOl] on scaled composite test scores, accounting for about 18% of the test score variance. Accordingly, Jewish student candidates outperform their Arab counterparts by a margin of about 1 SD (53.20 > 44.41, d = 1.03). Jewish candidates outscore their Arab counterparts by a margin of about a fifth of a SD on average matriculation certificate grade scores, with statistically reliable (P < 0.01) cultural group differences observed on each of the three matriculation subjects used as predictors. Interestingly, the greatest cultural group differences, on both aptitude and achievement measures, are associated with language; the smallest group differences are in mathematics. The construct validity of college admission indices, by culture To determine the dimensions assessed by the predictive indices for Jewish and Arab subgroups, the intercorrelations among the predictors were submitted to principal factor analysis followed by Varimax rotation, separately by culture. As shown in Table 2, two nontrivial factors, labelled Achievement and Aptitude, emerge in both Jewish and Arab subgroups. The Achievement factor. marked mainly by matriculation certificate grades, accounts for about 24% and 29% of the total variance within Jewish and Arab subgroups, respectively. The Aptitude factor, marked mainly by the five scholastic aptitude scales, accounts for 24% and 22% of the variance in the Jewish and Arab subgroups, respectively. The two factors together explain about 50% of the total variance within each group. The coefficients of congruence (Cattell, 1978. pp. 251-255) between Arab and Jewish groups are 0.98 for the Achievement factor and 0.97 for the Aptitude factor, indicating equivalence in both factors across cultural subgroups. * The predictive validity of college admission indices for Jewish and Arab student groups Table 3 presents the summary statistics for the predictor variables along with first-year college G.P.A., by culture. A comparison of the aptitude test scores for the original examinee pool (see Table 1) and the restricted student pool (see Table 3) shows, with few exceptions, highly similar cultural group difference profiles. Pearsonian correlations between first-year G.P.A. and composite test scores were uniformly significant, but low, within both Jewish (r = 0.16, P -C 0.001) and Arab (r = 0.15, P < 0.001) subgroups; the correlation is 0.29 in the combined groups. Furthermore, G.P.A. correlates slightly higher with average matriculation grades than with composite test scores; this holds true for the combined groups (r = 0.26, P < 0.001) as well as for the Jewish (r = 0.28, P < 0.01) and the Arab (r = 0.21. P < 0.001) groups. Neither the aptitude nor achievement predictive validity coefficients were found to be statistically heterogeneous by cultural group. Among the various predictors, English language aptitude shows the highest correlation with G.P.A. in combined groups (r = 0.32) and in the Jewish (r = 0.23) and Arab (r = 0.17) subgroups. A test for cultural group differences in the regression of G.P.A. on composite aptitude test scores, given the common regression line, was conducted via analysis of covariance procedure (cf. Freund and Littell, 1981). The analysis of covariance reveals that culture does not significantly (F < 1) interact with aptitude test scores in predicting G.P.A. Thus, there are no significant differences in the regression relationship, by culture. However. the Jewish and Arab groups differ significantly

Table 2. Intercorrelations

among pre-college admiwon

indices and loadings on Varimax Arab (N = 1017) subgroups

rotated principal

factors for Jewish (N = 1778) and

Varimax factors Correlation I

Information Figures Maths Analytic English English* Hebrew* Maths’

20 28 I7 27 I6 I8 I3

F

M

A

E

20

32

50 44 29 I9 I6 22

49 40 29 16 I7 22

24 48 43

51 31 34 43

36 19 I5 I7

Jewish group

matrix

29 26 25

E* I6 24 22 30 37 82 71

Arab group

H’

M*

Fl

F2

h2

FI

F2

h’

04 16 I4 23 I6 73

06 27 29 25 I7 58 70

01 I8 I5 21 14 76 86 74

54 58 61 60 65 26 06 17

29 37 40 40 44 64 75 58

I2 08 08 08 22 84 96 80

34 68 68 61 47 18 I3 20

I3 47 47 38 27 73 94 69

80

Note: The Jewish group intercorrelations appear above the diagonal; the Arab group intercorrelations appear below the diagonal. Decimals were omitted for correlations, all of which are significant at P < 0.05. Matriculation grades are designated by asterisks. The Achievement and Aptitude factors, respectively, account for 24% of the total variance each in the Jewish group and 29 and 22% of the total variance in the Arab group.

NOTES

AND

SHORTER

589

COMMUNICATIONS

Table 3. A comparison of the scholastic aptitude test performance and matriculation for Jewish (N = 824) vs Arab (IV = 364) students Jewish Scale

grade scores

Arab

M

SD

M

SD

r%

dt

53.35 53.07 51.70 53.14 53.99 54.19

10.15 9.24 10.39 9.20 9.94 9.22

47.65 47.17 48.98 47.54 47.38 46.89

9.30 a.99 a.52 a.73 7.35 7.64

0.26 0.28 0.14 0.28 0.31 0.36

0.59 0.65 0.29 0.62 0.76 0.87

52.90 52.03 53.04 51.20 73.24

9.02 8.68 8.18 9.92 II.51

50.04 52.29 51.56 49.57 60.15

8.99 9.52 8.14 10.35 13.33

0.14 -0.01 0.10 0.07 0.43

0.21 0.01 0.18 0.16 I .05

Aptifude scales

Information Figures Mathematics Analytical English Composite

grades English Maths Hebrew Average First-year G.P.A. Matriculation

Notes: Scores on all measures, except first-year G.P.A., were scaled to a mean of 50 and SD of 10 for the entire sample, collapsed across cultural groups (N = 2795); l-tests for cultural group mean differences on variables were all significant at the P < 0.01 level. lr designates the point-biserial correlation between culture (I = Jewish, 0 = Arab) and the measure under consideration, td is the ratio of the mean dilTerences (Jews-Arabs) divided by the average within group SD. on G.P.A. [F(l, 1184) = 205.40, P < O.OOl], indicating a slight degree of intercept bias. A test for cultural group heterogeneity in the regression of G.P.A. against average matriculation certificate grades proves to be nonsignificant. Moderate (multiple correlation) validity coefficients, employing individual subtests as predictors of C.P.A., are found for both the Jewish (R = 0.32, P < 0.001) and the Arab (R = 0.20, P < 0.08) groups. Furthermore, differences were calculated between observed and predicted mean G.P.A.s (observed-predicted), based on the multiple regression equation for the combined groups, and expressed in SD units. For Jewish candidates, the observed scores were about +0.23 SDS above what would be predicted from the common regression line; for Arab candidates they were about -0.45 SD units below what would be predicted on the basis of the common regression line. Furthermore, the use of the optimal weighted linear combination of the five aptitude tests as predictors, exclusive of matriculation grades, would result in the underprediction of Jewish student scholastic attainment by about a quarter of an SD (d = +0.23), and the overprediction of Arab student attainment by close to half a standard deviation (d = -0.47). Furthermore, a test for cultural group differences in the significance of mean residual scores shows significant effects [F(l, 1185) = 133.47, P < O.OOl], with predictive indices significantly overpredicting the test scores of Arab candidates and underpredicting those of Jewish candidates.

DISCUSSION The present investigation is generally consistent with findings reported in America, demonstrating remarkable invariance in the factor structure of college admission predictors across greatly differing ethnic and sociocultural groups. Furthermore, the data support the commonly held notion that scholastic aptitude indices are factorially differentiable from measures of previous scholastic achievement. In addition, this study provides little evidence for differential predictive validity of college admission indices as a function of cultural group membership. The combined aptitude and achievement predictors actually tend to discriminate far less among cultural groups relative to their more marked differentiation on the criterion measure. Accordingly, the Arab group intercept was observed to be slightly below that of the Jewish group intercept, which curiously would result in the overprediction of the former group’s criterion performance when the common or Jewish group regression line is employed. The finding of essentially equal regression lines across groups that differ on both SES and culture means that neither SES nor culture is relevant to the relation between the predictors and scholastic performance. That is, neither SES nor culture affects criterion performance except through its impact on aptitude and prior achievement. In sum, the cultural bias hypothesis, contending that college entrance predictive indices are biased against minority groups, is once again not confirmed. This finding in the Israeli cultural context generalizes the prevailing cgnclusions regarding the cultural bias hypothesis beyond the American scene. The present study increases confidence in the appropriateness of the cognitive and achievement indices under consideration for student selection among candidates of both Jewish and Arab background in Israel. REFERENCES Cattell R. B. (1978) The Scientific Use of Factor Analysis in Behavioral and Ltfe Sciences. Plenum Press, New York. Cleary T. A. (1968) Test bias: prediction of grades of negro and white students in integrated colleges. J. educ. Meas. 5, 115-124. Freund R. J. and Littell R. C. (1981) SAS for Linear Models: a Guide to the ANOVA and GLM Procedures. SAS Institute, Cary. Goldman M. (Ed.) (1980) Sociefy in Israel: Statistical Highlights, Vol. 11, Central Bureau of Statistics, Jerusalem. Jensen A. R. (1980) Bias in Mental Testing. Free Press, New York. Jensen A. (1984) Test bias: concepts and criticisms. In Perspectives on Bias in Mental Testing (Edited by Reynolds C. R. and Brown R. T.), pp. 507-586. Plenum Press, New York. Kleinberger A. F. (1969) Society, Schools and Progress in Israel. Pergamon Press, Oxford. Linn R. L. (1973) Fair test use in selection. Rev. educ. Res. 43, 139-161. Nevo B. (1985 February). Obtaining feedback from examinees. In Psychometric Testing: the Exuminee Perspecriue (Chairman M. Zeidner), Symposium conducted at the annual meeting of the Israeli Psychological Association. Ramat Gan, Israel. Stanley J. C. and Porter A. C. (1967) Correlation of scholastic aptitude test scores with college grades for negroes versus whites. J. educ. Meas. 4. 199-218.