- Email: [email protected]
Time capsules are not a panacea: A reply to Vernon
INTELLIGENCE 10, 277-279 (1986)
COMMENTARY
Time Capsules Are Not a Panacea: A Reply to Vernon ROBERT J. STERNBER(; Yale University
Vernon (1986) and I do not disagree on much. The main point of my critiquc--that there is much more to intelligence than mental speed--holds.
Vernon (1986) and 1 do not disagree on much. Most importantly, the title of his reply, " H e who doesn't believe in speed should beware of hasty judgments." shows that, whereas Vernon and 1 may disagree on the merits of my critique, wc agree that speed and intelligence are by no means synonymous. "Hasty judgments" can be quite unintelligent. 1 should point out, though, that I have never said that 1 believe mental speed bears no relationship to intelligence. To the contrary, 1 believe that it does bear some relation, and have done reaction-time research myself motivated by this belief. The important issue is to unravel the complex nature of the relationship. We must recognize that at some times speed is intelligent and at some times it is not. 1 do not believe that the views of Vernon and others regarding mental speed are wholly wrong, but rather that they vastly oversimplify the relationship between intelligence and mental speed. Vernon's elaborate analysis of the statistical significance of the correlations obtained by Vernon, Nador, and Kantor (1985) was unnecessary, from my point of view. 1 never questioned whether or not the individual correlational differences were statistically significant. As we all know, differences between correlation coefficients have a very large standard error associated with them, and it is quite difficult to get significant differences without very high degrees of statistical powcr. As Vernon (1986) points out, with enough p6wer, differences that were not significant in the Vernon et al. (1985) article might well have been. I simply made the point that 20 out of 21 differences were all in the same direction, and that when more than 95% of a set of sample outcomes go in the same direction, it is difficult to attribute the set of results to chance, even if each of the outcomes is small in its magnitude. This point has been made before in many other contexts. For example, E. Tulving (personal communication, Fall, 1972)
Preparation of this article was supported by Contract N00014-85-K-0589 from the Office of Naval Research and Contract MDA 903-85-K-0305 from the Army Research Institute. Correspondence and requests for reprints should be sent to Robert J. Sternberg. Department of Psychology. Yale University, Box I IA Yale Station. New Haven. CT 06520. 277
278
STERNBERG
has pointed out that, although the effect of negative transfer in part-whole free recall is often not statistically reliable in a given study, its rcplicability over studies renders it a phenomenon to be reckoned with. Similarly, the fact that all multiple regressions involving timed scores yielded higher multiple correlations than any of those multiple regressions involving untimed scores makes my point, regardless of whether or not individual correlational differences are significant. Perhaps it is a matter of taste, but 1 have never been impressed by multiple regressions that contain one statistically significant independent variable and a whole bunch of statistically trivial independent variables along with it. The multiple correlation adds nothing to the simple correlation except j u n k - - a n d a more impressive-looking coefficient, it is possible for the overall multiple correlation to be statistically significant, because it trades on the strength of association in the single predictor that matters. But it is hard to see why a multiple correlation is called for if only a single predictor yields a statistically significant regression weight. Vernon's paragraph (1986, p. 273) on speed of processing and its possible status as a factor underlying intelligencc totally baffles me. First he criticizes me for stating that he believes speed of processing is a factor underlying intelligence. He notes he made this statement only conditionally. Then he goes on to say that, despite the fact that he made thc statement conditionally, he does in fact believe it to be the case. i have trouble understanding the message here, if there is one. Vernon does not like my analogy of the power of a brain to the power of a car engine and prefers the usual analogy to a computer. He argues that computer speed is a good indication of computer power. 1 am probably less enchanted with the computer metaphor than is Vernon. but even if one accepts it in part, one should be wary of equating speed with powcr. Faster computers are more powerful, in a sense, because they can get through brute-force operations more rapidly than slower computers. For example, in chess-playing programs, a faster computer can check several million future board positions for their viability in a lesser amount of time than can a slower computer. But intelligent chess players do not calculate moves by brute force, in any case. They consider only a few possible moves, but generally very good-quality moves. The real intelligence of a computer will be shown not when it can calculate the outcomes of several million moves (n a short span of time, but when it can figure out which few moves are even worth considering in the first place. Vernon is obviously more impressed than I am by the validity of tests of general cognitive ability for predicting job performance. To describe such tests as having " v e r y high validity" (1986, p. 274) seems to me a euphemism, even given the Hunter-Schmidt results. I don't know that it is worth arguing over what constitutes " v e r y high validity." 1 do know that I would not want to select intelligent scientists, business executivcs, lawyers, or doctors on the basis of their speed in reaction-time tasks. To conclude, we need not argue over whether mental speed bears some relationship to intelligence. It does. That has been
TIME CAPSULES
279
shown again and again. But the time has come to go bcyond studies that show this basic fact again and again and to figure out exactly what the nature of the relationship is, under what circumstances it holds, and how mental speed fits in with other aspects of cognitive functioning. The hub of my argument is contained in Vernon's own title. Except on stripped-down laboratory tasks, "'hasty judgments" are not always intelligent ones. If we want to understand what makes judgments intelligent ones, we need to go beyond studying their haste.
REFERENCES Vernon, P.A. (1986). He who doesn't believe in speed should beware of hasty judgments: A reply to Sternberg. Intelligence, I0, 271-275. Vernon. P.A., Nador, S., & Kantor, L.(1985L Reaction times and ~,pced-of-processing:Their relationship to timed and untimed measures of intelligence. Intelligence, 9, 357-374.
COMMENTARY
Time Capsules Are Not a Panacea: A Reply to Vernon ROBERT J. STERNBER(; Yale University
Vernon (1986) and I do not disagree on much. The main point of my critiquc--that there is much more to intelligence than mental speed--holds.
Vernon (1986) and 1 do not disagree on much. Most importantly, the title of his reply, " H e who doesn't believe in speed should beware of hasty judgments." shows that, whereas Vernon and 1 may disagree on the merits of my critique, wc agree that speed and intelligence are by no means synonymous. "Hasty judgments" can be quite unintelligent. 1 should point out, though, that I have never said that 1 believe mental speed bears no relationship to intelligence. To the contrary, 1 believe that it does bear some relation, and have done reaction-time research myself motivated by this belief. The important issue is to unravel the complex nature of the relationship. We must recognize that at some times speed is intelligent and at some times it is not. 1 do not believe that the views of Vernon and others regarding mental speed are wholly wrong, but rather that they vastly oversimplify the relationship between intelligence and mental speed. Vernon's elaborate analysis of the statistical significance of the correlations obtained by Vernon, Nador, and Kantor (1985) was unnecessary, from my point of view. 1 never questioned whether or not the individual correlational differences were statistically significant. As we all know, differences between correlation coefficients have a very large standard error associated with them, and it is quite difficult to get significant differences without very high degrees of statistical powcr. As Vernon (1986) points out, with enough p6wer, differences that were not significant in the Vernon et al. (1985) article might well have been. I simply made the point that 20 out of 21 differences were all in the same direction, and that when more than 95% of a set of sample outcomes go in the same direction, it is difficult to attribute the set of results to chance, even if each of the outcomes is small in its magnitude. This point has been made before in many other contexts. For example, E. Tulving (personal communication, Fall, 1972)
Preparation of this article was supported by Contract N00014-85-K-0589 from the Office of Naval Research and Contract MDA 903-85-K-0305 from the Army Research Institute. Correspondence and requests for reprints should be sent to Robert J. Sternberg. Department of Psychology. Yale University, Box I IA Yale Station. New Haven. CT 06520. 277
278
STERNBERG
has pointed out that, although the effect of negative transfer in part-whole free recall is often not statistically reliable in a given study, its rcplicability over studies renders it a phenomenon to be reckoned with. Similarly, the fact that all multiple regressions involving timed scores yielded higher multiple correlations than any of those multiple regressions involving untimed scores makes my point, regardless of whether or not individual correlational differences are significant. Perhaps it is a matter of taste, but 1 have never been impressed by multiple regressions that contain one statistically significant independent variable and a whole bunch of statistically trivial independent variables along with it. The multiple correlation adds nothing to the simple correlation except j u n k - - a n d a more impressive-looking coefficient, it is possible for the overall multiple correlation to be statistically significant, because it trades on the strength of association in the single predictor that matters. But it is hard to see why a multiple correlation is called for if only a single predictor yields a statistically significant regression weight. Vernon's paragraph (1986, p. 273) on speed of processing and its possible status as a factor underlying intelligencc totally baffles me. First he criticizes me for stating that he believes speed of processing is a factor underlying intelligence. He notes he made this statement only conditionally. Then he goes on to say that, despite the fact that he made thc statement conditionally, he does in fact believe it to be the case. i have trouble understanding the message here, if there is one. Vernon does not like my analogy of the power of a brain to the power of a car engine and prefers the usual analogy to a computer. He argues that computer speed is a good indication of computer power. 1 am probably less enchanted with the computer metaphor than is Vernon. but even if one accepts it in part, one should be wary of equating speed with powcr. Faster computers are more powerful, in a sense, because they can get through brute-force operations more rapidly than slower computers. For example, in chess-playing programs, a faster computer can check several million future board positions for their viability in a lesser amount of time than can a slower computer. But intelligent chess players do not calculate moves by brute force, in any case. They consider only a few possible moves, but generally very good-quality moves. The real intelligence of a computer will be shown not when it can calculate the outcomes of several million moves (n a short span of time, but when it can figure out which few moves are even worth considering in the first place. Vernon is obviously more impressed than I am by the validity of tests of general cognitive ability for predicting job performance. To describe such tests as having " v e r y high validity" (1986, p. 274) seems to me a euphemism, even given the Hunter-Schmidt results. I don't know that it is worth arguing over what constitutes " v e r y high validity." 1 do know that I would not want to select intelligent scientists, business executivcs, lawyers, or doctors on the basis of their speed in reaction-time tasks. To conclude, we need not argue over whether mental speed bears some relationship to intelligence. It does. That has been
TIME CAPSULES
279
shown again and again. But the time has come to go bcyond studies that show this basic fact again and again and to figure out exactly what the nature of the relationship is, under what circumstances it holds, and how mental speed fits in with other aspects of cognitive functioning. The hub of my argument is contained in Vernon's own title. Except on stripped-down laboratory tasks, "'hasty judgments" are not always intelligent ones. If we want to understand what makes judgments intelligent ones, we need to go beyond studying their haste.
REFERENCES Vernon, P.A. (1986). He who doesn't believe in speed should beware of hasty judgments: A reply to Sternberg. Intelligence, I0, 271-275. Vernon. P.A., Nador, S., & Kantor, L.(1985L Reaction times and ~,pced-of-processing:Their relationship to timed and untimed measures of intelligence. Intelligence, 9, 357-374.