- Email: [email protected]
The impossibility of IQ
Economics Letters 2 (1979) 95-97 0 North-Holland Publishing Company
95
THE IMF’OSSIBILITY OF IQ Peter J.W.N. BIRD Universityof Stirling, Stirling, FK?J 4LA, Scotland Received
January
1979
Arrow’s Impossibility Theorem is applied to the problem of measuring IQ. It is necessary to combine specific ability orderings into a general intelligence ranking, but this is impossible without infringing one desirable condition. IQ scores have no place in economics.
Only rarely do the uncertainties of one discipline survive transplantation into another. Few cases illustrate this better to our profession than that of the Intelligence Quotient (IQ). Psychologists divide over whether there exists such a thing as general intelligence, and over how much it can be measured by existing IQ tests. Yet economists happily incorporate the results of such tests into their econometric models - the seminal work of Becker (1964) on Human Capital theory is an apt example. The purpose of this paper is to argue against the use of IQ test scores in economic investigation. Arrow’s Impossibility Theorem is used to criticise the concept of measuring general intelligence; it follows that IQ test scores have as much, which means as little, scientific status as representations of a Social Welfare function. We proceed by assuming that there exist n individuals, x, y, z, .. . . to be assessed by a psychologist who considers m specific abilities, i = 1,2,3, . . .. m, to be components of general intelligence. The psychologist is assumed capable, for each ability i, of ordering the individuals in some way, the order relation corresponding to the notion ‘is at least as intelligent as’. An IQ function is then defined as a functional relation that for any set of orderings of the individuals by the various qualities determines a final general intelligence, or IQ, ordering. Formally, let X be the set of individuals. The ordering by the psychologist of the individuals according to the ith ability is Ri, and let there be m such abilities. Let Q denote the desired IQ ordering. It is assumed that for each i there is an ordering Ri that is reflexive, transitive, and complete. Then an ZQ function is a functional relationship f such that for any set of m orderings of the individuals R 1, .... R, (one for each quality), one and only one IQ ordering, Q, is determined. The four Arrow conditions can now be imposed upon the IQ function. Thus:
P.J. W.N. Bird f The impossibility of IQ
96
Condition U (Unrestricted Domain). The domain of the function f must include all logically possible combinations of individual ability orderings. Condition P
(Pareto).
For any pair (x, y) in X, x Riy for all i implies x Q y.
Condition I (Independence of Irrelevant Individuals). Let Q and Q’ be two IQ. orderings determined by the IQ function f corresponding respectively to two sets of orderings (RI, .... R,) and (R; , .... Rk). If for all pairs of individuals (x, y) in a subset S of X, x Ri y impliesx R:y for all i, then C(S, R) and C(S, R’) are the same, where C(S, R) is the choice set corresponding to the relation R. There is no ability i such that for every Condition D (No Dominant Ability). element in the domain off, for all (x, y) in X, x Ri y implies that x Qy. Such a formulation parallels exactly Sen’s (1970) statement problem. It thus follows directly that:
of the social choice
There is no IQ function that can satisfy simultaneously conditions U, P, I and D. The relevance of this IQ Impossibility Theorem depends upon two conditions. The first, and the crucial, condition is that in any assessment of ability, the tester can apply to subjects only an ordinal and not a cardinal measure. The second is that any meaningful ordinal measure of general intelligence must necessarily involve, explicitly or implicitly, constructing a general ordering from a number of particular orderings. Both these propositions are valid. To illustrate the first condition, consider three ways in which an apparently cardinal measure of a particular ability can be constructed. The first is that used by psychologists in reporting test scores. The rankings of a standardization sample are mapped directly to a normal distribution, with mean of 100 and standard deviation of 15. The numbers so obtained are far from objective measures of intelligence; rather they are statistical artefacts. A similar procedure is implicit in the grading in universities of students’ examination scripts. A second road to cardinality is provided by the multiple-item test. The cardinal score is the aggregate of correct answers. But cardinalization is obtained at the expense of Condition U: the domain of the IQ function is restricted to that small subset of abilities by which individuals can be assessed on a possess/do not possess basis. A third procedure is to cardinalise with some mechanical proxy, such as the time taken to solve a problem. i Again, the illusion of cardinality is false: the set of such times can be no more than a monotonic transformation of the ranking of individuals according to the ability. 1 I am indebted for this suggestion to an anonymous reviewer of an earlier draft.
P.J. W.N. Bird / The impossibility of IQ
97
We have considered here just three examples, but the point concerning the first condition is a general one. For any particular ability, no more than an ordinal ranking of individuals can be achieved. The second condition for the theorem’s applicability is that any final IQ ordering must represent a distillation of individual ability orderings. This is less contentious; it seems open to dispute only if some concept of general intelligence can be defined that is observable and testable independently of abilities in specific areas. An advantage of applying Arrow’s Theorem to IQ measurement is that it provides a framework for the common criticisms of IQ testing. Consider the frequent argumenf that IQ tests discriminate unfairly against those from minority cultures. The argument arises from the use of cardinalization to combine test orderings in different abilities. Each test has to be standardized. The resultant overall ranking of a group being tested thus depends on the performance in the test of the standardization sample - who generally are members from the majority culture. Condition I is infringed: the members of the standardization sample constitute ‘Irrelevant Individuals’ . ‘No Dominant Ability’ is the other condition often broken in practice. Selectivity in education is an example. Frequently, children are chosen on the basis of a single dominant component, verbal ability. To argue that a selective educational system cannot do justice to children’s varied abilities is no more than to emphasise the importance of the Condition D. Our arguments have implications both for psychology and for economics. For the psychologist, we have contributed a formal framework in which intelligence testing can be discussed. The criticisms are not new; the problems of culture-bias, of dependence on a standardization sample, and of the narrow nature of educational selectivity are freely admitted both by advocates and by opponents of testing. But, more generally, a critique has been produced which shows how inevitable it is that any testing process will be unsatisfactory in at least one respect. For the economist, the conclusion is stark. IQ is an unsatisfactory variable: it has no role to play in serious economic analysis.
References Becker, Gary, Sen, Amartya,
1964, Human capital (National Bureau of Economic Research, New York). 1970, Collective choice and social welfare (Oliver and Boyd, London).
95
THE IMF’OSSIBILITY OF IQ Peter J.W.N. BIRD Universityof Stirling, Stirling, FK?J 4LA, Scotland Received
January
1979
Arrow’s Impossibility Theorem is applied to the problem of measuring IQ. It is necessary to combine specific ability orderings into a general intelligence ranking, but this is impossible without infringing one desirable condition. IQ scores have no place in economics.
Only rarely do the uncertainties of one discipline survive transplantation into another. Few cases illustrate this better to our profession than that of the Intelligence Quotient (IQ). Psychologists divide over whether there exists such a thing as general intelligence, and over how much it can be measured by existing IQ tests. Yet economists happily incorporate the results of such tests into their econometric models - the seminal work of Becker (1964) on Human Capital theory is an apt example. The purpose of this paper is to argue against the use of IQ test scores in economic investigation. Arrow’s Impossibility Theorem is used to criticise the concept of measuring general intelligence; it follows that IQ test scores have as much, which means as little, scientific status as representations of a Social Welfare function. We proceed by assuming that there exist n individuals, x, y, z, .. . . to be assessed by a psychologist who considers m specific abilities, i = 1,2,3, . . .. m, to be components of general intelligence. The psychologist is assumed capable, for each ability i, of ordering the individuals in some way, the order relation corresponding to the notion ‘is at least as intelligent as’. An IQ function is then defined as a functional relation that for any set of orderings of the individuals by the various qualities determines a final general intelligence, or IQ, ordering. Formally, let X be the set of individuals. The ordering by the psychologist of the individuals according to the ith ability is Ri, and let there be m such abilities. Let Q denote the desired IQ ordering. It is assumed that for each i there is an ordering Ri that is reflexive, transitive, and complete. Then an ZQ function is a functional relationship f such that for any set of m orderings of the individuals R 1, .... R, (one for each quality), one and only one IQ ordering, Q, is determined. The four Arrow conditions can now be imposed upon the IQ function. Thus:
P.J. W.N. Bird f The impossibility of IQ
96
Condition U (Unrestricted Domain). The domain of the function f must include all logically possible combinations of individual ability orderings. Condition P
(Pareto).
For any pair (x, y) in X, x Riy for all i implies x Q y.
Condition I (Independence of Irrelevant Individuals). Let Q and Q’ be two IQ. orderings determined by the IQ function f corresponding respectively to two sets of orderings (RI, .... R,) and (R; , .... Rk). If for all pairs of individuals (x, y) in a subset S of X, x Ri y impliesx R:y for all i, then C(S, R) and C(S, R’) are the same, where C(S, R) is the choice set corresponding to the relation R. There is no ability i such that for every Condition D (No Dominant Ability). element in the domain off, for all (x, y) in X, x Ri y implies that x Qy. Such a formulation parallels exactly Sen’s (1970) statement problem. It thus follows directly that:
of the social choice
There is no IQ function that can satisfy simultaneously conditions U, P, I and D. The relevance of this IQ Impossibility Theorem depends upon two conditions. The first, and the crucial, condition is that in any assessment of ability, the tester can apply to subjects only an ordinal and not a cardinal measure. The second is that any meaningful ordinal measure of general intelligence must necessarily involve, explicitly or implicitly, constructing a general ordering from a number of particular orderings. Both these propositions are valid. To illustrate the first condition, consider three ways in which an apparently cardinal measure of a particular ability can be constructed. The first is that used by psychologists in reporting test scores. The rankings of a standardization sample are mapped directly to a normal distribution, with mean of 100 and standard deviation of 15. The numbers so obtained are far from objective measures of intelligence; rather they are statistical artefacts. A similar procedure is implicit in the grading in universities of students’ examination scripts. A second road to cardinality is provided by the multiple-item test. The cardinal score is the aggregate of correct answers. But cardinalization is obtained at the expense of Condition U: the domain of the IQ function is restricted to that small subset of abilities by which individuals can be assessed on a possess/do not possess basis. A third procedure is to cardinalise with some mechanical proxy, such as the time taken to solve a problem. i Again, the illusion of cardinality is false: the set of such times can be no more than a monotonic transformation of the ranking of individuals according to the ability. 1 I am indebted for this suggestion to an anonymous reviewer of an earlier draft.
P.J. W.N. Bird / The impossibility of IQ
97
We have considered here just three examples, but the point concerning the first condition is a general one. For any particular ability, no more than an ordinal ranking of individuals can be achieved. The second condition for the theorem’s applicability is that any final IQ ordering must represent a distillation of individual ability orderings. This is less contentious; it seems open to dispute only if some concept of general intelligence can be defined that is observable and testable independently of abilities in specific areas. An advantage of applying Arrow’s Theorem to IQ measurement is that it provides a framework for the common criticisms of IQ testing. Consider the frequent argumenf that IQ tests discriminate unfairly against those from minority cultures. The argument arises from the use of cardinalization to combine test orderings in different abilities. Each test has to be standardized. The resultant overall ranking of a group being tested thus depends on the performance in the test of the standardization sample - who generally are members from the majority culture. Condition I is infringed: the members of the standardization sample constitute ‘Irrelevant Individuals’ . ‘No Dominant Ability’ is the other condition often broken in practice. Selectivity in education is an example. Frequently, children are chosen on the basis of a single dominant component, verbal ability. To argue that a selective educational system cannot do justice to children’s varied abilities is no more than to emphasise the importance of the Condition D. Our arguments have implications both for psychology and for economics. For the psychologist, we have contributed a formal framework in which intelligence testing can be discussed. The criticisms are not new; the problems of culture-bias, of dependence on a standardization sample, and of the narrow nature of educational selectivity are freely admitted both by advocates and by opponents of testing. But, more generally, a critique has been produced which shows how inevitable it is that any testing process will be unsatisfactory in at least one respect. For the economist, the conclusion is stark. IQ is an unsatisfactory variable: it has no role to play in serious economic analysis.
References Becker, Gary, Sen, Amartya,
1964, Human capital (National Bureau of Economic Research, New York). 1970, Collective choice and social welfare (Oliver and Boyd, London).