Expanding the frontier of human cognitive abilities: so much more than (plain) g!

Learning and Individual Differences 13 (2002) 127 – 158

Expanding the frontier of human cognitive abilities: so much more than (plain) g! David B. Bowman, Pippa M. Markham, Richard D. Roberts* School of Psychology, University of Sydney, Sydney, NSW 2006, Australia Received 9 September 2002; accepted 9 September 2002

Abstract Commentators have argued that general intelligence (or g) is the most important predictor of occupational, educational, and (even) life success (e.g., The g factor by Jensen). Studies implementing advances in information technology, statistical analysis, and emergent scientific trends suggest that the concept of g is not as efficacious as its proponents indicate. Findings moving us beyond g are presently reviewed, including extensions of traditional, structural models of intelligence. Moreover, above g there appear factors, both cognitive and noncognitive, that support intelligent behavior. A range of these constructs, including conceptual models of emotional intelligence (EI), tacit knowledge, and metacognition, are discussed. While these models have problems, there appears much more scope to intelligence research (with all its implications to applied domains) than the restricted perspective offered by proponents of general intelligence. D 2002 Elsevier Science Inc. All rights reserved. Keywords: General intelligence; Success; Cognitive; Noncognitive

1. Introduction The science of human cognitive abilities is now over a century old. During this time, there have been many iconoclastic struggles, perhaps represented best in the disjunction between the so-called British and American schools in the middle half of the twentieth century. It is * Corresponding author. Tel.: +61-2-9351-5696; fax: +61-2-9351-2603. E-mail address: [email protected] (R.D. Roberts). URL: http://www.psych.usyd.edu.au/difference5/index2.html. 1041-6080/02/$ – see front matter D 2002 Elsevier Science Inc. All rights reserved. PII: S 1 0 4 1 - 6 0 8 0 ( 0 2 ) 0 0 0 7 6 - 6

128

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

our contention that a new schism is now evident, one that is arguably more divisive. On the one hand, exists an academic movement, resting on the assumption that the realm of human cognitive abilities has been fully circumscribed using the major tools available to the psychometrician (i.e., psychological tests and multivariate statistics). These researchers, convinced that general intelligence alone is scientifically meaningful, have moved on to attempt explanation, of this hypothetical entity, using computational, genetic, and neuroscientific models. On the other hand, there exist researchers who believe that there is a frontier of human cognitive abilities for the psychometrician to discover. Proponents of this perspective argue that there is still much work to be done in understanding individual differences in human cognitive abilities, in charting the so-called cognitive sphere. In this paper, a series of arguments supporting the latter viewpoint are presented. While there is no intention to rub salt into what may become an open laceration, a comprehensive science of intelligence demands that thorough investigation be given to emergent concepts. The arguments presently offered do not detract from the cross-disciplinarian approaches offered by the former perspective, which may provide a proper explanatory model of intellectual functioning, broadly defined. Nevertheless, it appears self-evident that the possibilities afforded by advances in cognitive science, neuroscience, and behavioral genetics await the development (and implementation) of taxonomic models that more fully capture the complexities of human individual differences. With this overarching goal in mind, the current paper will first critique the concept of general intelligence, which some have seen as inhibitory to the study of human cognitive abilities (see e.g., Hearnshaw, 1951; Roberts, Pallier, & Goff, 1999; Stankov, 2002). Recent developments in an alternative structural theory, which posits the existence of broad cognitive abilities, are then considered. These developments suggest that the role given previously to sensory processes in extant models of human intelligence has been inadequate. We conclude with discussion of new constructs—emotional intelligence (EI), tacit knowledge, and metacognitive processes. It is suggested that each hold promise for enriching our understanding of individual differences in human cognitive abilities, particularly if certain problems (which we suggest solutions to) are remedied.

2. Psychometric g: exposition and critique The concept of general intelligence owes its origins to Spearman (1904, 1923, 1927), who proposed that there are two factors underlying mental test performance—a general factor ( g) and specific factors (s). Since specific factors are unique to performance on any cognitive test, whereas the general factor permeates performance on all intellectual tasks, Spearman postulated that g alone is of psychological significance.1 Within this account, individual differences in general intelligence are the result of differences in the magnitude of mental

1

Technically speaking (and it is sometimes dismissed nontrivially), because specific abilities are unique to any given test, Spearman’s model implicitly evokes many intelligence factors (Carroll, 1993).

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

129

energy invested in any given task. Throughout his writings, Spearman suggests that such differences may be best understood in terms of the variation in people’s abilities to use three ‘‘qualitative principles of cognition’’: the apprehension of experience, eduction of relations, and the eduction of correlates.2 These three principles constituted an attempt to provide a cognitive model of intelligence, long before cognitive psychology had actually become a legitimate scientific discipline. Hence, Spearman (1923) speculated that the ability to educe relations and correlates represented a form of mental processing with identifiable neurophysiological substrates that were largely, inherited. These three laws of ‘‘noegenesis’’ provide important guidelines for test construction. For instance, the Raven’s Progressive Matrices and Esoteric Analogies were constructed on the premise that the eduction of relations and correlates is central to intelligent behavior. Features of general factor theory are endorsed implicitly or explicitly inside several models of human intelligence. In the case of implicit attempts at retaining psychometric g, the terminology has been replaced with concepts such as ‘‘executive functioning’’ (see e.g., Belmont, Butterfield, & Ferretti, 1982). Detterman (1982) argues that such constructs, nonetheless, assume that a unitary process circumscribes performance on all cognitive tasks. A more explicit attempt to retain the essential aspects of g has been that made by Jensen and colleagues. These researchers have retained much of Spearman’s terminology as well as the spirit of his many research proposals (see e.g., Jensen, 1985, 1992a, 1993, 1998; Jensen & Weng, 1994). 2.1. Jensen and the g factor Jensen (1970, 1974) originally endorsed a theory in which a distinction was made between Level I and Level II abilities. In this theory, Level I abilities require minimal mental transformation and manipulation when compared to Level II abilities. The usefulness of this theory appears questionable (see e.g., Horn & Stankov, 1982; Stankov, 1987; Stankov, Horn, & Roy, 1980). Jensen has since abandoned this model in favor of a Spearmanian approach, which incorporates advances in cognitive psychology, factor analysis, and neuroscience. Jensen (1985, 1992a, 1992b, 1993, see also Jensen & Weng, 1994) provides several reasons for postulating a theory of g, including: 1. The existence of positive manifold: A particularly robust finding in intelligence research acknowledges that intelligence tests correlate in a lawful fashion. ‘‘The fact that, in large unrestricted samples of the population, the correlations are virtually always positive’’ means ‘‘that the tests all measure some common source of variance in addition to whatever 2 Carroll (1993) provides the following, useful summary definitions of the major principles encapsulated in the terms eduction of relations and education of correlates. ‘‘The word eduction means the drawing out of some logical abstraction or consequence from two or more stimuli. Relations are abstractions like ‘similarity’ and ‘comparison’; correlates are the particular attributes of stimuli that are seen as identical, similar, compared, or related in some way’’ (Carroll, 1993, p. 53).

130

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

else they may measure’’ (Jensen & Weng, 1994, p. 232). Positive manifold is often used as one of the most powerful arguments for the existence of g. 2. The stability of g across test batteries: Jensen (1992a, 1998) claims that however g is extracted from a correlation matrix, the coefficient of congruence between factor solutions remains high (see also Jensen & Weng, 1994). This evidence suggests that whichever method of factor analysis is used, the similarity between the underlying constructs is retained—a nontrivial fact given the range of factor-analytic methodologies that may be employed. 3. The utility of g in the real world is great: Jensen argues that it is psychometric g that is the chief ‘‘active ingredient’’ responsible for cognitive tasks having both practical and concurrent validity in real-life situations (e.g., Jensen, 1992a, 1998; see also Jensen & Weng, 1994). Given the necessity for psychology to demonstrate its practical utility, this is often touted as one of the major benefits of conceiving intelligence as a unitary trait. 4. Psychometric g has meaningful (yet independent) empirical correlates: One of the major features of g, according to Jensen (1992a), is the fact that it correlates with ‘‘a number of variables which themselves have nothing to do with psychometrics or factor analysis’’ (p. 278). Behavioral variables identified by Jensen include movement time, decision time, inspection time, and musical tests. Noncognitive variables include heritability coefficients, inbreeding depression, average evoked potential, the rate of glucose metabolism in the brain, speed of neural and synaptic transmission, and head and brain size. These correlations have often been used evidentially against claims that g is a statistical artifact, whilst at the same time affording this concept greater scientific credibility.

2.2. A critical appraisal of the g factor The question remains as to whether Jensen’s (1992a, 1998) assertions regarding general mental ability are as noncontentious as he would have the reader believe. While a comprehensive critique of Jensen’s postulations concerning g is outside the scope of the present paper, it is difficult to reconcile alleged key points with empirical findings in the contemporary cognitive, neuroscience, genetic, and psychometric literature. Consider for example, each of the following: 1. The ‘‘lawful’’ principles underlying g are problematic: Guttman (1992) provides a disputatious critique of Jensen’s attempts to apply Spearman’s principles (Roskam & Ellis, 1992), which has stimulated considerable debate (see e.g., Gustafsson, 1992a, 1992b; Jensen, 1992b, 1992c; Loehlin, 1992; Scho¨nemann, 1992). In his critique, Guttman provides compelling evidence suggesting that both positive manifold and the invariance of g are equivocal. Positive manifold, in particular, need not mathematically imply g. Moreover, a large number of noncognitive variables (e.g., athleticism, openness, absence of neuroticism, and psychosis) each correlate positively with intelligence tests yet do not represent a functional unity (Roberts et al., 1999). Furthermore, it is no small point that in personality research, no single author would claim the presence of a general personality

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

131

factor, although (providing neuroticism is reversed) positive manifold is generally evident among constructs usually acknowledged as structurally independent. Guttman (1992) also identifies discrepancies between Spearman and Jensen and their research programs. For instance, he notes that Spearman (1927) held little hope for mental speed measures (e.g., inspection time) providing any meaningful account of the general factor. Jensen (1998), on the other hand, sees mental speed as a prime candidate for providing an explanatory model of g. Guttman (1992) also queries the extent to which Spearman suggested that racial differences meaningfully contribute to variance in general intelligence. However, Jensen (1985, 1993, 1998; see also Braden, 1989) has published several papers examining racial differences, explicitly alluding to the ‘‘Spearman hypothesis.’’ 2. The ingredients of g are arbitrary: Horn (1985) argues that the first principal component is no more than a good weighted linear combination of the abilities of a test battery and, as such, does not represent a legitimate test of single factor models. Different collections of tests yield different principal components, since no one intelligence measure provides a representative sample of the known abilities comprising the domain of intelligence. There can be no singular g because its meaning varies across occasions depending on the arbitrary collection of tests chosen by the experimenter (e.g., Horn, 1985; Humphreys, 1979; Stankov, 2002). Researchers have also suggested that the reasons for measures of people’s performance correlating positively on any two tasks may be the result of any number of attributes, in isolation or combination (e.g., Howe, 1990). This somewhat controversial proposition is a particularly difficult problem for any single factor theory of intelligence to counter, especially as the derivation of a general factor initially involves comparing correlation coefficients between each pair of test scores. 3. g makes a ‘‘minor’’ contribution to total test variance: Carroll (1992) argues that the first principal component, at best, may account for no greater than 50% of the common factor variance observed in cognitive test performance. This figure undoubtedly represents an upper limit, since when a large number of measures, representing disparate primary mental abilities, are sampled, this figure seldom exceeds 25% (Stankov, 2002). Even the higher figure may be construed as unsatisfactory given that a substantial percentage of variance— which is neither specific nor error variance—remains unaccounted for in the presence of g (see e.g., Carroll, 1993; Gustafsson, 1992a, 1991b; Roberts et al., 1999; Roberts & Stankov, 1999; Stankov, 2002). Given this line of reasoning, it has been suggested that focusing upon g is more akin to religious dogma than science (see Matthews, Zeidner, & Roberts, in press). 4. The predictive utility of g is equivocal: Findings from diverse areas including cognitive science, developmental psychology, neurology, and behavioral genetics attest to the fact that disparate cognitive abilities have different construct validities (see e.g., Horn, 1989; Horn & Hofer, 1992; Stankov, Boyle, & Cattell, 1995). The predictive validity of g has been questioned, especially, in the cognitive aging literature (e.g., Horn, 1987; Horn & Hofer, 1992; Horn & Noll, 1994; Stankov et al., 1995). Gustafsson (1992a, 1992b) makes a similar point regarding hypothesized racial differences in g, where, if group factors are considered, a noticeably different picture emerges. Furthermore, meta-analyses supporting

132

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

the validity of intelligence tests for job selection assume (perhaps incorrectly) that all tests measure the same general factor. It is actually an open empirical question whether classes of tests defining broad cognitive abilities might have different validity coefficients. While the onus is on critics of psychometric g to show that broad constellations of abilities do a more efficacious job in the prediction of criterion performance, this possibility cannot be ruled-out based on meta-analyses thus far conducted. 5. g defined: too few tests? Jensen (1979, 1992a, 1998) has often asserted that g may only be extracted from a large (in principle, infinite) and varied battery of intelligence tests. However, in practice, the empirical research involving behavioral and noncognitive variables often uses a small number of tests, indeed most commonly a single index (generally, the Raven’s Progressive Matrices Tests; see Roberts & Stankov, 1999). Whether inferences may be drawn to g, from tests assumed to have high loading on the general factor, remains contentious (e.g., Carroll, 1993; Stankov et al., 1995; cf. Detterman, 2002). For example, there is considerable evidence to suggest that the Raven’s Progressive Matrices test is factorially complex—early items tapping into visualization abilities, while latter items assess reasoning and working memory (see e.g., Roberts & Stankov, 1999). 6. The correlates of g remain tenuous: Almost all of the behavioral correlates of g, which Jensen suggests as indicative of the importance of general mental ability, have been questioned in the literature. For example, inspection time research has been criticized on methodological, conceptual, and theoretical grounds (e.g., Levy, 1992; Stankov et al., 1995). Indeed, recent evidence suggests that the relationship between inspection time and g is mediated largely by a variety of psychological processes, including shared visual requirements, strategy effects, and processing speed (see e.g., Nettelbeck, 2001). Similarly, Stankov and Roberts (1997) have questioned the pivotal assumptions under which decision time per se has been linked to g, recently supporting these propositions with data (Roberts & Stankov, 1999). These authors show that measures of mental speed correlate meaningfully with some (but not all) cognitive ability factors—a proposition that is immediately at odds with the concept of a single, general intelligence. Evidence for the noncognitive correlates of psychometric g is equally contentious. In particular, Jensen (1992a) claims that: ‘‘Persons’ rates of glucose metabolism by the brain while taking a highly g-loaded test . . . is correlated (negatively) with the persons’ test scores’’ (p. 281). Jensen bases this proposition on a study that examines individuals neural efficiency (measured using positron emission tomography) while performing the Raven’s Progressive Matrices (see Haier et al., 1988). Problems inferring from a single test to g aside, an attempt at replicating the Haier, Siegel, Tang, Abel, and Buchsbaum (1992) findings was met with mixed results (see Stankov & Dunn, 1993). Moreover, earlier findings, presented by Metz, Yassillo, and Cooper (1987), lend themselves to a somewhat different interpretation. With regard to the genetic correlates of g, different perspectives have been offered in the literature (e.g., Horn, 1998) (i.e., the view espoused by Jensen is certainly not accepted by all). Similarly, correlations between many of the noncognitive variables (e.g., head size) and g seldom exceed .30 (e.g., Jensen & Sinha, 1992). Notably, even this modest correlation may represent an upper limit to the biological concomitants of g (Detterman, 2002). In short, whether these constitute meaningful dimensions of

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

133

individual differences in intelligence remains questionable (e.g., Stankov et al., 1995; Willerman, 1991). 2.3. Concluding comments Each of the preceding arguments questions the contention that psychometric g constitutes the most important individual differences construct in intelligence research. Other commentators have acknowledged a similar point, suggesting that g be considered a diminutive general (Stankov, 2002). Our position is still stronger—it is entirely likely that the focus on general intelligence, while appearing steeped in research, has proven damaging to the field. By choosing one-off tests (allegedly measuring g) and conducting any sort of experiment (be it genetic, psychophysiological, or cognitive), the source of variance may have been incorrectly located (see Carroll, 1993). Furthermore, when researchers make generalizations based on the general factor and extend these ideas to policy informing social, educational, and commercial interests, they may not only be in gross error scientifically but adversely affect members of a particular racial group, gender category, or even community. How damaging might such an approach be? The answer is perhaps best highlighted with an illustrative example. Herrnstein and Murray’s (1994), The Bell Curve, is a monumental, although contentious work, combining a (rather selective) review of the intelligence field with implications for informing public policy on class in the United States. This book argues for the importance of general intelligence in understanding social class in modern societies. The authors imply that individuals born into economically and educationally advantaged family backgrounds also inherit higher g when compared to their lower class counterparts. This differential distribution of intelligence, by sociocultural group, is claimed to determine, to a significant degree, the chances of various social groups to attain educational and occupational success. The approach espoused by the authors conveys a rather pessimistic message for an egalitarian society— there would appear little hope for the future of those individuals destined to be born into lower class families or those coming from minority backgrounds. The conclusions reached in this book stem in large measure from hypotheses concerning the factor structure of the Armed Services Vocational Aptitude Battery (ASVAB), and in particular a g score emanating from a selection of these subtests (the so-called Air Force Qualifying Test; see Roberts et al., 2000). In fact, an entire chapter of The Bell Curve is devoted to discussion of this index, largely because it is on the basis of data collected with the ASVAB (for the 1980 ‘‘Profile of American Youth’’) that pivotal empirical analyses were conducted (see Herrnstein & Murray, 1994, pp. 569–592). Recently, however, Roberts et al. (2000) have demonstrated, in two large-scale studies and using confirmatory factor analytic techniques, that the ASVAB is predominately a measure of acculturated knowledge (Gc, see below) rather than g per se. In sampling a limited universe of cognitive abilities, the whole of The Bell Curve exercise is rendered problematic. The differential crystallized intelligence of the ‘‘underclass’’ has never been in dispute, and it is the failure of intervention strategies for an ability that is highly malleable that should primarily have been examined more fully. The failure of Herrnstein and Murray (1994) to even consider this possibility is symptomatic of the problems that occur when g is given undue emphasis in intelligence theories and is more

134

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

endemic to the field than this (by no means trivial) example typifies (for other exemplars, see e.g., Rushton, 1999).

3. Extending the theory of fluid (Gf) and crystallized (Gc) intelligence Preoccupation with g would perhaps lead to considerable consternation, were it not for the postulation of alternative perspectives, acknowledging the diversity of human cognitive capabilities. Particularly prominent among these approaches is the theory of fluid and crystallized intelligence, which deems there to be enough structure among primary mental abilities to define several distinct broad cognitive abilities. The model derives its name from the two most extensively studied constructs—fluid reasoning (Gf) and acculturated knowledge (Gc). The main distinguishing feature between these constructs is the amount of formal education and knowledge of the dominant culture necessary for performance on measures of these abilities. It is well established that Gf depends to a much smaller extent on formal education experiences than does Gc (e.g., Horn & Hofer, 1992; Horn & Noll, 1994, 1997; Roberts et al., 2000; Stankov et al., 1995). The original conceptualization of Gf–Gc theory inherently contained the possibility that empirical investigation would identify factorially distinct cognitive abilities further to those already recognized. Indeed, in an elaboration of this theory, Horn and Noll (1997) have posited the existence of some seven broad abilities (beyond Gf and Gc)—each of which is conceptually, practically, and empirically defensible. Further factors, tied to memory processes of varying duration, include short-term apprehension and retention (SAR) and fluency of retrieval from long-term storage (TSR). Recent evidence has also identified at least two factors tied to different types of speed in performing tests of varying difficulty, including processing speed (Gs) and correct decision speed (CDS); while another has been linked specifically to mathematical abilities—broad quantitative ability (Gq). Of particular importance to subsequent considerations of the present paper, broad abilities have also been uncovered that are linked to individual differences in visual (Gv) and auditory (Ga) processing. All broad factors of Gf–Gc theory are assumed to share differential relations with a range of psychological, social, and biological factors and to have distinctive cognitive, genetic, and neurophysiological concomitants (see e.g., Horn, 1998; Horn & Noll, 1997). Proponents of Gf–Gc theory generally acknowledge that a proper understanding of all human cognitive abilities is required for a full-blown theory of intelligence to have demonstrable ecological validity. Similarly, an assumption driving this model is that the human organism is complex (i.e., information processing is not delimited simply to auditory and visual channels). However, since the vast majority of cognitive tests are presented to the individual via visual and auditory media, ‘‘several sensory modalities remain underrepresented and, therefore, poorly understood in individual differences research’’ (Roberts, Stankov, Pallier, & Dolph, 1997, p. 112). With these assumptions in mind, and noting that two independent factors have been previously linked to perceptual processes, recent extensions of Gf–Gc theory have attempted to delineate broad abilities tied to further sensory modalities (Danthiir, Pallier, Roberts, & Stankov, 2001; Roberts et al., 1997, 1999;

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

135

Stankov, Seizova-Cajic, & Roberts, 2001). In the passages that follow, we review this research, which is fairly recent in origin. 3.1. Olfaction and cognitive abilities Danthiir et al. (2001) examined the place of olfactory processes within the sphere of human cognitive abilities. This study employed 17 tasks, five of which were presented in the olfactory modality. In addition to an olfactory discrimination test, several putative ‘‘cognitive’’ olfactory measures were constructed to resemble established psychometric measures. Exploratory factor analysis of these data resulted in four factors, three of which were related to extant broad abilities (Danthiir et al., 2001). These were Gf/Gv (i.e., a factor representing an amalgamation of the constructs of fluid reasoning and visual processing), Gc, and SAR. Interestingly, a fourth factor, predominately defined by the cognitive olfactory measures, emerged.3 Each of the olfactory measures employed contained some memory component. In combination with the paucity of differential studies investigating olfactory capacities, this factor was tentatively interpreted as olfactory memory (OM), most likely lying at the first stratum. A number of the ‘‘cognitive’’ olfactory markers employed by Danthiir et al. (2001) demonstrated moderate-sized positive correlations with established psychometric measures, and small loadings from established tests were observed on the OM factor. For example, a smell identification task exhibited salient loading on the (conceptually related) Gc factor, as did an olfactory swaps measure on the Gf/Gv factor. Low positive intercorrelations were also observed between the OM factor and the second-order factors derived by Danthiir et al. (2001). Collectively, this evidence is indicative of the OM factor being both cognitive in nature (i.e., not dependent on simple olfactory acuity) and structurally independent of established cognitive abilities. The fact that this is (to our knowledge) the only differential study into olfactory abilities suggests several avenues for future research. Studies utilizing more extensive batteries of olfactory tests are needed in order to elucidate the nature of the olfactory factor identified by Danthiir et al. (2001). In particular, additional tests need to be constructed in the olfactory modality that might legitimately demarcate primary mental abilities beyond those linked to memory. If this is possible, the position of olfactory processing within the hierarchy of human cognitive abilities may be fully ascertained. Two outcomes appear likely—olfactory processes represent primary mental abilities of already existent broad cognitive abilities—although more likely is the presence of a second stratum olfactory ability conceptually akin to auditory (Ga) and visual (Gv) processing. The extent to which olfactory processes (whether located at the first or second stratum) might add incremental predictive validity to certain job clusters or aid in the identification of clinical syndromes (e.g., loss of smell is an indicator of the onset of Alzheimer’s disease) also deserves attention.

3

The fact that these two broad abilities did not separate in the Danthiir et al. (2001) study is consistent with previous research in which this link has been observed (see our discussion of tactile – kinesthetic processes).

136

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

3.2. Tactile–kinesthetic processes and cognitive abilities In addition to cognitive olfactory abilities, recent studies conducted at the University of Sydney have examined the place of tactile–kinesthetic processes in the structure of human intelligence (Roberts et al., 1997; Stankov et al., 2001). Despite Gardner’s (1983) claims surrounding the existence of bodily kinesthetic intelligence, there have been few studies investigating tactile–kinesthetic processes. The paucity of prior differential research conducted in this domain is perhaps best reflected in the fact that Carroll’s (1993) meta-analysis of some 477 data sets contained only four studies investigating tactile–kinesthetic ability. The aforementioned research program has explicitly attempted to redress this deficiency. Roberts et al. (1997) employed a battery of 27 cognitive ability measures along with a set of eight relatively complex tactile–kinesthetic tasks. The results of this study appear to support the presence of a tactile–kinesthetic ability factor at the first-order of analysis. However, it should be noted that the authors of this study recognized several caveats in drawing strong conclusions from these data. First, the factor identified by Roberts et al. (1997) does not appear to reflect pure tactile–kinesthetic ability. Although this factor was defined largely by tactile– kinesthetic measures, visual perception processes also exhibited salient loadings on this factor. Moreover, visual processes could not be easily separated from tactile measures in this study, even when confirmatory factor analysis was conducted. Second, the tactile–kinesthetic measures employed by Roberts et al. (1997) appear to have strong working memory components. As such, the ability identified in this study may be substantially broader than the tactile–kinesthetic processes defined by traditional measures in this domain (e.g., the Halstead–Reitan Tactual Performance Test). Finally, the tactile–visualization factor identified by Roberts et al. (1997) was closely related to fluid intelligence, suggesting that Gf, Gv, and tactile–kinesthetic processes may be difficult to separate at an empirical level. Subsequently, Stankov et al. (2001), utilizing a more comprehensive battery of measures (8 cognitive ability and 14 tactile – kinesthetic indices), demonstrated that visual – spatial processes are difficult to separate from complex measures of tactile–kinesthetic processing. These findings suggest that existing neuropsychological tests that use relatively complex tactile and kinesthetic tasks may essentially assess Gf and/or Gv abilities albeit in different sensory modalities. Stankov et al. (2001) employed seven tactile–kinesthetic tasks not used by Roberts et al. (1997), which were generally simpler than those used for neuropsychological assessment. Stankov et al. identified separate kinesthetic and tactile factors defined predominately by salient loadings from these ‘‘new’’ tests. Unlike the factor postulated by Roberts et al. (1997), these factors both appear to be structurally independent of the (existing) higher order cognitive abilities (e.g., Gf and Gv). These factors also appear to reflect cognitively simpler processes conceptually akin to the aspects of performance captured by pure perceptual markers of higher stratum abilities (i.e., Gv and Ga). Finally, these abilities appear to share relatively little common variance casting doubt on the presence of a broad ability spanning both the tactile and kinesthetic domains. Until the nature of these tactile and kinesthetic factors can be further elucidated, Stankov et al. (2001) have placed these abilities at the first stratum of an ‘‘extended’’ structural model of

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

137

human intelligence. However, the possibility remains that future research will see an elaboration of this taxonomic model to include separate broad abilities relating to tactile and kinesthetic processes. The fact that these processes are prominently studied in cognitive psychology, learning, neuropsychology, and developmental psychology is nonincidental to previous arguments mounted against the concept of g (for a review of some of this research see Roberts et al., 1997). Individual differences in both tactile and kinesthetic processes are clearly scientifically meaningful, yet g theory would seemingly have us ignore them. Moreover, it seems conceivable that a battery of tests measuring these processes might add incremental predictive validity in job selection, help elucidate factors tied to cognitive aging, and/or lead to fresh understanding of the neuropsychological underpinnings of cognitive abilities (Pallier, Roberts, & Stankov, 2000). Given these possibilities alone, further research into individual differences in tactile and kinesthetic processing is rendered scientifically, socially, and practically important. 3.3. Beyond g through sensory processes In virtually every instance, findings from the preceding studies, we suggest, run counter to theories of g. Consider thus, the following: 1. The first principal component accounts for little common variance when abilities linked to sensory processes are studied: Stankov (2002, Table 2.1) has demonstrated empirically that studies including measures of sensory processes and (traditional) indicators of intelligence, result in a general factor that is diluted—accounting for around 20–25% of the total variance. It is worth noting that as models of intelligence become more conceptually elaborate (i.e., further cognitive abilities are identified) the strength of this principal component (logically at least) could diminish still further. 2. Abilities linked to sensory processes will likely have practical utility, which g theory (in principle) ignores: On an applied level, the study of sensory abilities may lead to the development of test batteries that would appear useful in diagnosis of special populations such as the sight or hearing impaired. Further, it is likely that the processes captured by sensory measures may be useful in the selection of professionals where acute sensory capacities are paramount. For example, wine and culinary expertise likely requires highlevel olfactory and/or gustatory processing, while professions requiring manual dexterity (e.g., dentistry, performing art, and carpentry) involve high-level tactile and/or kinesthetic processing. Carefully conducted validation studies while lacking, might not only improve selection in these fields, but may also point to limitations currently inherent in the way gbased measures address this issue. 3. The psychological models underlying sensory processes are complex and need to be accounted for to provide a tenable model of g: The psychological processes underlying sensory abilities are more complex than simple sensory acuity. Individual differences are not merely evident in the ability to perceive information, but also to manipulate and utilize that information. If (lower order) abilities tied to sensory domains cannot be reduced to simple perceptual processes, then why would higher order cognitive abilities—such as Gf

138

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

and Gc (and of course, g)—be reducible to speed or lower biological mechanisms (like head size)? 4. Unlike g, sensory processes constitute a tractable interface between intelligence and biology: Sensory abilities are intrinsically linked to biology—it is through the senses that we receive our information from the outside world. As such, sensory abilities appear to represent a logical first step in developing models linking cognitive abilities to biology. Arguably, it is only when the mechanisms underlying sensory abilities have been established that various biological models of higher order cognitive abilities can be considered in a meaningful fashion. 3.4. Concluding comments Recent elaborations of Gf–Gc theory suggest that extant structural models of cognitive abilities are likely underdeveloped. Given that the goal of individual differences is to understand ‘‘the diversity of intellect in the people of the planet . . . cognitive processes and operations, mental performances, and creations of knowledge and art’’ (Carroll, 1995, p. 429), assessing processes tied to the tactile, kinesthetic, and olfactory senses is scientifically defensible. The astute reader might also point to the need to study gustatory processes— and this is yet another candidate process we would advocate investigating. Moreover, as our understanding of the human information processing system is expanded, still further ability factors are likely to come to light. For example, recent investigations into long-term working memory may indicate that there is yet another meaningful individual differences construct tied to memory. There is also the possibility (as alluded to in Roberts & Stankov, 1999) that the structure of speed of performance is as elaborate as that underlying level (i.e., intelligence as defined by accuracy scores). The search to completely chart the cognitive sphere should thus be a major undertaking of intelligence research. Equally important as a research enterprise, each broad factor should be linked to cognitive, genetic, biological, and social factors, which, we believe, would give the field of individual differences enhanced scientific status.

4. Extending traditional notions of intelligence Another challenge to psychometric g, representing (to our mind) the direction in which differential psychology should head, is into an area that we will label ‘‘new constructs.’’4 Considerable impetus for the investigation of these new constructs comes from applied 4

As testament to the importance associated with the investigation of new constructs, one prominent psychological test company—Educational Testing Service (ETS)—has recently established a research arm devoted to emergent individual differences constructs, known as the Center for New Constructs. The mission of this center is to research cognitive, personality, and attitudinal factors such as learning ability, critical reasoning in context, creativity, motivation, teamwork, leadership, practical and emotional intelligence, self-concept, selfefficacy, metacognition, and others. Presumably, the lack of reference to traditional academic – intelligence constructs, which ETS has certainly played a major role in the past with the SAT and GRE, is nonincidental.

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

139

psychology. Here, the (combined) validity coefficients of intelligence, personality, and other individual differences variables, for the prediction of educational, workplace, and (even) life success, appear no higher than .60 (see e.g., Hunter & Hunter, 1984; Jensen, 1998; Matthews et al., in press). On the theoretical side, beyond general cognitive ability, or intelligence, per se, there also clearly appear factors, both cognitive and noncognitive, that might be said to support intelligent behavior. A theme of this research focus is that by enhancing these factors one can enhance intellectual performance, moving the domain of individual differences afield from the static viewpoint that intelligence is destiny (see Kyllonen, Roberts, & Stankov, in preparation). For example, the concept of ‘‘emotional intelligence’’ extends the idea of human cognitive abilities by suggesting that social and emotional factors can affect intelligent behavior (Mayer, Salovey, & Caruso, 2000a). Further still, Sternberg (1996) has suggested that creativity and practical intelligence are supplemental to analytic intelligence, in producing intelligent behavior, and has suggested methods for enhancing these factors. One new construct that has received particular attention with regard to enhancement is practical intelligence (or perhaps more correctly, tacit knowledge) (Sternberg & Grigorenko, 2000). There is also an emerging interest in the idea of metacognition governing over a range (and perhaps all) of these constructs, including those underlying general intelligence and/or the broad cognitive abilities previously elucidated. It is to detailed discussion of these new constructs that the rest of the present exposition is devoted. 4.1. Emotional intelligence Inspired by an influx of research, several best-selling trade texts, and frequent media exposure, EI has emerged as a concept that, while unlikely to replace general intelligence, may certainly rival it for sheer popular, economic, and scientific interest. The concept appears to have prospered due to a number of converging societal factors, including cultural trends and orientations that stress the previously neglected role of the emotions (Matthews et al., in press). Popular claims suggest that EI is predictive of important educational and occupational criteria beyond that proportion of variance that general intellectual ability predicts (although this claim clearly lacks empirical grounding) (see Zeidner, Roberts, & Matthews, in press). Thus, the field appears increasingly to have important implications for society, particularly in the impetus to improve emotional functioning in real life. Another reason for widespread acceptance of the EI construct may be attributed to the suggestion that EI gives hope for a more utopian, classless society. This vision for the future stands in contrast to messages contained in The Bell Curve, which as we indicated earlier, suggested a preordained ‘‘cognitive elite.’’ The argument runs something like this; EI, by its very nature, is within everyone’s grasp to learn, cultivate, enrich, change, and/or ultimately enhance. However, there appear a number of conceptual, empirical, and practical problems currently plaguing the emerging field demarcated by the term ‘‘emotional intelligence.’’ These include, problems of definition, determining the correct methods to employ for assessing dimensions of EI, and the relative paucity of published empirical data (although this last issue is changing). It is to a discussion of the current status of EI, with respect to each of these issues, that this paper now turns.

140

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

4.1.1. Defining EI: promises, problems, and remedies It has been asserted that EI represents a conceptually coherent construct covering a wide array of emotional functioning (e.g., Goleman, 1995). However, examination of the literature suggests there is no clear, operational definition of EI. In turn, the multitude of qualities covered by the concept are overwhelming, and arguably suggest that it may be ephemeral (Roberts, 2002). For example, Mayer and Salovey (1997), couch it as a form of cognitive ability (where the processing of emotional information is tantamount), leading them to construe EI as the capacity to reason about emotions (see also Mayer et al., 2000a, 2000b). Goleman (1995), on the other hand, defines EI by exclusion (i.e., it represents all those positive qualities that are not represented by g). Reflecting an ardently prescientific view, he even suggests that: ‘‘there is an old-fashioned word for the body of skills that EI representscharacter’’ (p. 34). By contrast, Bar-On (1997) depicts EI as ‘‘an array of non-cognitive capabilities, competencies, and skills that influence one’s ability to succeed in coping with environmental demands and pressures’’ (p. 14). The preceding definitions raise several concerns about contemporary conceptualizations of EI. Does EI represent an aptitude for handling challenging situations, expression of which may vary according to prevailing situations? Is there a single EI (akin to psychometric g—let us call it emotional g) or rather a collection of EIs? Perhaps instead, EI represents an outcome variable, reflecting the successful resolution of environmental contingencies. Such differing definitions and conceptions have caused, we believe, considerable confusion in the literature. In defense of EI, however, one should bear in mind that, after over a century of research in the field of cognitive abilities, there remains (as the present paper attests) controversy over the precise meaning of intelligence. Note however, that there is certainly consensus concerning the operational means for arriving at intelligent behavior (see Boring, 1923). Moreover, following two very famous conferences, where the concept of cognitive–academic intelligence was debated, general agreement concerning its nature has been reached (see Intelligence and its measurement, 1921; Sternberg & Detterman, 1986). These series of debates suggest that the concept of intelligence is representative of some adaptive function having real-life consequences. Elsewhere, we have suggested that a similar forum, involving luminaries in the fields of emotion and intelligence research, might lead to more general agreement concerning the frames of reference under which EI might be investigated. Presently, the range of definitions has led to a critical impasse. On the one hand, are proponents who, because they acknowledge the personality-like nature of EI, are prepared to assess it using self-report methodologies. On the other hand, proponents considering EI from the ability perspective see self-report methodologies as circumspect; EI should be capable of being objectively determined. While it is too early to dismiss either approach, as the preceding disposition unravels, the reader should perceive that the evidence clearly favors the performance-based framework over the use of self-report methodologies. 4.1.2. Self-reported EI: common (although problematic) measurement Proponents following the self-report approach to EI create tests that ask a person to endorse a series of descriptive statements on some form of rating scale. For example, in the Schutte Self-Report Inventory (Schutte et al., 1998) the individual rates themselves from 1

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

141

(strongly disagree) to 5 (strongly agree) on 33 statements (e.g., ‘‘I know why my emotions change’’). Notably, self-perceptions of EI may be inaccurate, being vulnerable to response— sets, social desirability, deception, and impression management factors, although this is seldom acknowledged. Alternatively, this form of EI may not be consciously accessible. It is also questionable whether items asking participants to self-appraise intellectual ability (e.g., ‘‘I am an extremely intelligent student’’) would make for a valid measure of cognitive– academic intelligence. Indeed, past research has reported rather modest associations (r =.30 or so) between self-rated and actual ability measures (e.g., Paulhus, Lysy, & Yik, 1998). Moreover, tests of EI that assess noncognitive traits (e.g., impulse control) seem to be assessing dimensions of individual differences that relate to established personality constructs rather than intelligence (Davies, Stankov, & Roberts, 1998; Matthews et al., in press). Empirical data pointing to the substantial relationship between self-reported EI and existing personality measures are unequivocal. Thus, a recent study by Dawda and Hart (2000) revealed average correlations approaching .50 between measures of the Big Five personality factors (i.e., neuroticism, extroversion, openness, agreeableness, and conscientiousness) and Bar-On’s EQ-i measure. Noting the relative independence of each of the Big Five Factors (e.g., Costa & McCrae, 1992), these data suggest that the EQ-i is nothing but a proxy measure of a composite of Big Five Personality constructs, weighted most strongly towards low neuroticism. This finding is ubiquitous—virtually all self-reported EI instruments share similar relationships with personality (see Matthews et al., in press). A recent Dutch study, evaluating the relationship between Bar-On’s EQ-i and the General Adult Mental Ability scale (GAMA—a measure of Gf), raises yet another major criticism of self-reported EI (Derksen, Kramer, & Katzko, 2002). Results indicated that the correlations between the EQ-i and the GAMA were very low (i.e., approaching zero). These findings indicate that the two tests are psychometrically independent—that the EQ-i is measuring something other than an ‘‘intelligence.’’ Subsequently, Garcia, Roberts, Zeidner, and Matthews (in preparation) have found near-zero correlation between the Schutte Self-Report Inventory and a number (and range) of measures of Gf and Gc. 4.1.3. Performance-based measures of EI: the way forward? The preceding account raises the possibility that EI is simply an old wine (personality) dressed up in a new bottle (EI). Fortunately, there have been alternative perspectives to the assessment of EI, which gain considerable impetus from the proposition that it meets the standards required of a traditional intelligence. Under this framework, it is claimed that EI needs to meet certain conceptual, psychometric, and developmental criteria (Mayer & Cobb, 2000; Mayer & Salovey, 1993, 1997; Mayer et al., 2000a, 2000b; Salovey, Bedell, Detweiler, & Mayer, 2000). It is both to an exposition of this approach and these criteria that discussion will now focus. The first criterion is that the concept in question be capable of being operationalized as a set of abilities (in this case, emotion-related capabilities) that have clearly defined performance components. Thus, EI should be capable of reflecting cognitive performance rather than nonintellective attainments or preferred ways of behaving (Mayer, Caruso, & Salovey, 1999). It must therefore be possible to categorize answers to stimuli assessing various facets of

142

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

feelings, as correct or incorrect (Mayer & Salovey, 1997). Often, however, the emotionally intelligent response to a real-life problem is unclear or depends on mitigating circumstances. Therefore, the question remains as to how to decide whether a response to a test item is emotionally intelligent or not. To resolve this difficulty, proponents of the performance-based approach to EI employ three methods: expert scoring, target judgment, and group consensus. These methods, however, have problematic aspects for the instantiation of EI. In expert scoring, specialists determine the best answer to each question. However, no criteria exist for deciding who is an expert in the domain of the emotions. Target scoring involves the creator of the stimuli determining the correct answer. Problems with this method include the target not being able to accurately express the emotion that they are feeling or perhaps becoming prosocial when making their reports. The third method, consensus scoring, allocates a score to each option according to the percentage of people choosing that option. This method effectively scores an option as indexing greater or lesser levels of EI rather than simply scoring a response as right or wrong. Consensus scoring, by its very nature, excludes identification of difficult items on which, say, 10% of the most able individuals pick the correct answer, and the consensus answer is incorrect. Thus, consensus scoring is likely to lead to special problems at the top end of the scale, especially in distinguishing the ‘‘emotional genius’’ from the normally functioning, emotionally intelligent person. Consequently, consensual-scored, performancebased measures of EI may be more effective in screening for ‘‘emotional stupidity’’ than in discriminating levels of EI at the upper end of the range (see also MacCann, Roberts, Matthews, & Zeidner, submitted for publication). These facts notwithstanding, arguably the most important criterion related to psychometrics is the establishing the construct validity of EI and in particular the extent to which it overlaps with other intelligence(s). Mayer, Caruso, and Salovey (1999) found that a performance-based measure of EI was sufficiently differentiated from verbal abilities to provide unique variance but also sufficiently correlated to indicate that concepts underlying the measure formed an ‘‘intelligence.’’ Somewhat curiously, the verbal intelligence measure used in this study (i.e., the Army Alpha) is seldom employed in contemporary investigations of cognitive ability. Moreover, another study, using an oft-used measure of cognitive abilities came up with a notably different finding. Thus, Ciarrochi, Chan, and Caputi (2000) found near-zero (and sometimes negative) correlations between various factors of performancebased EI and the Ravens Standard Progressive Matrices Test. This curious result may simply be an aberration caused by small sample size or may reflect the fact that EI shares more in common with acculturated abilities than fluid intelligence. Consistent with these interpretations, Roberts, Zeidner, and Matthews (2001), studying a sample of 720 Air Force enlistees, found moderate positive correlations between Gc and factors of performance-based EI. Another necessary condition for EI to qualify as a form of intelligence relates to developmental criterion and empirical demonstration that EI increases with age and experience (Mayer et al., 1999). Consequently, age differences in EI have been heralded as evidence for the construct validity of EI. Mayer et al. (2000a) report that differences in mean EI scores observed for adolescents and adults serve as evidence supporting this developmental criterion. Note, however, that the above study was based on a cross-

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

143

sectional design and thus allows interpretation only in terms of age group—not developmental—differences. Moreover, contrary, to what has been claimed by EI researchers, it is a misconception that intelligence increases developmentally. Some classes of broad cognitive abilities (e.g., Gf) decline (see e.g., Carroll, 1993; Horn & Hofer, 1992), while others (e.g., Gc) improve. Following this logic, it seems plausible to suggest that different components of EI might have different developmental trajectories and that each should be examined in a carefully designed cross-sequential longitudinal study (see Schaie, 2001). 4.1.4. Concluding comments Research into EI is likely to continue unabated for some time, although currently the directions in which this research is preceding appear rather disjointed. Appreciating this stateof-affairs, Matthews et al. (in press) have recently suggested guidelines for how this research program might proceed more scientifically. They advocate each of the following: (1) development of a process-based theory, (2) the construction of reliable and valid measurement methodologies, and (3) demonstration of the practical applications of EI. Matthews et al. (in press) also suggest that it is a mistake to conduct investigations into EI without considering the extensive theoretical and applied research already dealing with emotional aptitudes and competencies. Thus, it needs to be demonstrated that tests of EI measure something new; that EI is distinct from existing dimensions of individual differences. Similarly, a theoretical account of EI must differentiate the biological and cognitive processes supporting emotional competence from those processes that are known to underpin existing personality, emotional, and intelligence dimensions. Finally, claims concerning the importance of EI in applied domains hinge on the demonstration that is distinct from concepts, procedures, and techniques that are more fully understood. Future research into EI should seek to address, in a systematic manner, all of these aspects of the concept and in so doing, give it greater scientific credibility. 4.2. Practical intelligence and tacit knowledge Sternberg (1985) has also emphasized a departure from traditional conceptualizations of intelligence. On this basis, he goes ‘‘beyond IQ’’ to emphasize different aspects of intellectual functioning. Perhaps the most prominent of these concepts is practical intelligence. While there have been difficulties arriving at a tight definition of this construct (a case reminiscent of EI), it is generally agreed that practical intelligence entails the ability of an individual to deal with the problems and situations of everyday life (Wagner, 2000). Sternberg and colleagues suggest that while academic intelligence is useful in the academic classroom, practical intelligence contributes in a meaningful fashion to life success (e.g., Sternberg, 1993; Sternberg et al., 2000; Sternberg, Wagner, Williams, & Horvath, 1995; Sternberg, Wagner, & Okagaki, 1993). Clearly, from our review thus far, traditional approaches to the measurement of intelligence (be it g or broad cognitive abilities) have not adequately assessed the concept of practical intelligence, making it another viable construct that extends the boundaries of the science of individual differences.

144

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

An important type of practical intelligence, which has received considerable impetus, is the concept of tacit knowledge. Sternberg et al. (1995) define tacit knowledge as ‘‘actionorientated knowledge, acquired without direct help from others, that allows individuals to achieve goals they personally value’’ (p. 916). Tacit knowledge is reflected in a given situation and involves taking actions that were not necessarily taught inside a formal learning system. Because tacit knowledge is often acquired with little (or no) environmental support, it tends to be difficult to articulate, and is often not given enough emphasis related to its importance for practical success (Sternberg et al., 1993). There are a number of research findings highlighting the potential importance of tacit knowledge and its independence from psychometric g. For example, tacit knowledge tests have been found to have low (sometimes even negative) correlations with measures of traditional academic intelligence (and especially Gc). These tests also appear unrelated to personality constructs (with the possible exception of measures of Agreeableness), at least for certain types of occupations (see e.g., Wagner & Sternberg, 1985). Tacit knowledge measures also appear to have demonstrable utility. For example, in a study of business managers, tacit knowledge scores correlated in the range of .20–.40 with criteria such as salary, whether or not the manager worked for a Fortune 500 company, and years within management (Sternberg et al., 2000; Wagner & Sternberg, 1985). Because generally low (or near zero) correlations are observed with cognitive ability tests, tacit knowledge measures also appear to possess incremental predictive validity, over and above measures of academic intelligence (Hedlund & Sternberg, 2000). While Sternberg and his colleagues have undertaken much of the differential research into tacit knowledge, the concept was actually introduced into the literature by the philosopher, Polanyi (1966). Polyani contrasts tacit knowledge with all forms of knowledge learned explicitly, pointing to the fact that it likely explains why scientists, for example, are able to make advances in their chosen field. The concept of tacit knowledge also shares parallels with the notion of implicit learning, which has been defined as ‘‘the acquisition of knowledge that takes place largely independent of conscious attempts to learn and largely in the absence of explicit knowledge of what was acquired’’ (Reber, 1993, p. 5). As such, cognitive psychologists, differential psychologists, educational psychologists, and industrial–organizational psychologists have taken up the construct of tacit knowledge, alike. In the passages that follow, we begin with discussion of various attempts to assess tacit knowledge, using paradigms having their origin in industrial–organizational psychology. The exposition then addresses findings from disparate domains of applied psychology, which have demonstrated that these measures have some predictive validity, although there are clearly some aspects that also require further investigation. We conclude with a series of suggestions for improved assessment of the concept, using procedures that are currently being developed in our laboratory. 4.2.1. Measuring tacit knowledge The specific series of items that comprise tacit knowledge tests draw on a methodology that has been established within industrial and organizational psychology. The most common design of these items is in the form of a situational judgment test (SJT). The SJT technique

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

145

presents an individual with a description of a problematic or critical situation, followed by a number of possible problem-solving responses to this situation.5 The situations are primarily generated employing another established industrial–organizational technique: the critical incident approach. Critical incidents (or problematic situations) are produced from observations, surveys, and interviews of task specialists to specify the nature of the competencies that appear to be pivotal for success in a particular task domain. Using a rigorous item development process that borrows on these techniques, Sternberg and colleagues have developed a number of specific inventories to assess performance in a range of applied settings. Among the most fully developed of these measures are the Tacit Knowledge Inventory for Managers (TKIM), the Tacit Knowledge Inventory for Sales, and the Tacit Knowledge Inventory for Military Leaders (TKML) (see Sternberg et al., 2000a, 2000b, for actual sample items). However, tacit knowledge tests have also been developed for college students (Sternberg et al., 2000), academic psychologists (Wagner & Sternberg, 1985), Kenyan school children (Sternberg et al., 2001), and even immigrants (Nevo & Chawarski, 1997). Despite the fact that the content of these various tests varies considerably (as might be expected given the domain-specificity of tacit knowledge), the general structure of these tests is remarkably similar. A scenario is given to the participant (e.g., You have been asked to give a talk to students on tips for good writing in a psychological report). This scenario is followed by instructions to the participant to rate the quality of various courses of action associated with this hypothetical situation (e.g., Advise the students to carefully consider the audience for whom they are writing). The participant is required to make a decision, for each and every course of action (often 10 are given), on a seven-point Likert-scale (e.g., 1 = extremely uncharacteristic to 7 = extremely characteristic). Currently, the scoring of the tacit knowledge items from each of these tests has been yoked to the concept of expert knowledge, being such that experts are expected (due to the procedural nature of implicit learning) to be more competent within their specialized domain. By implication, competence in the tacit domain is defined in contrast to traditional measures of performance, which have been explicitly taught. Thus, where traditional measures of performance use a veridical scoring protocol, tacit knowledge tests use expert scoring (i.e., scores are benchmarked on the performance of members in higher positions within the organization). Due to the scoring techniques employed (expert scoring), if an individual (or group of individuals) is recognized to be successful in a certain environment, they are assumed to be competent in the tacit domain. Their responses to a nonveridical series of items are taken to define successful tacit knowledge within that specific area (e.g., managerial decision making). All other individuals are scored using the expert response as the template. In general, it must be said of the various tacit knowledge measures so far developed that the reliability is modest (often measures of internal consistency are no larger than .60) (see

5

The use of a situational judgment test to assess any competency raises certain methodological issues. These tests fall into an area of tasks defined as ‘‘low-fidelity’’ simulations (Motowildo, Dunnette, & Carter, 1990). We note, as we explicate later on in the current paper, that Markham and Roberts (in preparation) have developed tacit knowledge tests that are more conceptually sound, since they more closely resemble in-basket tests.

146

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

e.g., Sternberg et al., 2001). The magnitude (and congruence) of these coefficients indicates that further test development is likely required (see Carroll, 1993), a point that must be taken very seriously if tacit knowledge inventories are ever to be used for important selection decisions. Equally vexing, information on the test–retest reliability of tacit knowledge measures is sparse. There would appear a need to more carefully document these and other psychometric properties (e.g., factor structure), of each respective tacit knowledge test, according to established standards and principles (Gottfredson, 2002).6 4.2.2. Empirical findings Although a large number of measures of tacit knowledge have been constructed, the number of peer-reviewed publications examining them is relatively sparse—Sternberg and colleagues tending instead to report their findings in books, book chapters, or unpublished technical reports and manuscripts (Gottfredson, 2002). Despite this and preceding criticisms, we contend that the early research with these instruments suggests that they represent a form of cognitive ability, relatively independent of existent cognitive ability constructs, with demonstrable practical utility. In the passages that follow, we review some of these findings as they pertain to three specific domains: organizational, educational, and military psychology. 4.2.3. Organizational settings Tacit knowledge has been most extensively assessed in the area of industrial and organizational psychology. The main research focus has been to examine tacit knowledge in relation to job performance, with a wealth of criterion variables serving as predictors. For example, global tacit knowledge scores correlate meaningfully with salary (r =.46), level of position (r =.36), companies worked for (r =.35), work simulations (r =.61), and salary increase (r =.48) in a variety of management contexts (see, e.g., Sternberg et al., 2000; Wagner & Sternberg, 1985). In studies examining salespersons, tacit knowledge has also been shown to be related to criteria such as number of quality awards (r =.35), sales volume (r =.37), and premiums (r =.29). In many instances, the correlations with ability measures were also obtained, with the tacit knowledge measures having at least as high validity coefficients as measures of g. Equally, because the correlations between ability and tacit knowledge measures were low, it appears they possess incremental predictive validity. The research thus far conducted by Sternberg and colleagues is not without problems, since the sample sizes reported have generally been quite small (generally around 50 participants) 6

In a disputatious critique, Gottfredson (2002) uses these and other limitations to argue that tacit knowledge is ephemeral. To some extent, this critique relies on a series of ‘‘straw-persons’’ for impact. For example, it is claimed (in something of a linguistic sleight of hand, involving an obscure earlier quote and hypothesized developmental similarities) that Sternberg et al. equate tacit knowledge with Gc. This claim lacks substance in light of Sternberg et al. (2000), who actually call for further studies examining the developmental trajectory of tacit knowledge. Gottfredson (2002) goes on to state that tacit knowledge is simply the ‘‘motivated and sensitive application of whatever level of g we individually possess’’ (p. 50). This claim reinforces points made throughout the current paper concerning disjunction in the field of individual differences. It also indicates a dangerous precedent which at all costs should be avoided in a science—an attempt to stifle a new construct in its infancy, which in no way necessarily replaces the old, but rather supplements it.

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

147

(Gottfredson, 2002). The measures of intelligence employed are certainly nonoptimal tests. For example, in one study of leadership in managers, Sternberg et al. (2000) measured intelligence using the Shipley Institute of Living Scale (SILS), a short-screening test, which was originally devised as an index of intellectual deterioration (Shipley, 1983). While Sternberg et al. (2000) found a nonsignificant correlation (r =.14) between tacit knowledge and SILS, because of its origins, the SILS has a notably low ceiling, such that it does not result in sufficient spread among high performing individuals (Matthews et al., in press). Given this problem in range restriction and the small sample size, little can be concluded from this investigation concerning the overlap between tacit knowledge and measures of intelligence. Future research should remedy this situation by employing tests known to be more suitable for assessing a wide range of talent, extending beyond g to assess the various cognitive abilities comprising Gf–Gc theory. 4.2.4. Educational settings Another area where tacit knowledge has been assessed is in the domain of education and more specifically, attempting to discover those aspects of college success that cannot be learned from textbooks (see Sternberg et al., 2000). The methodologies used to assess tacit knowledge in this instance appear less fully developed than those used in organizational settings, since they have so far focused on a single methodology (developed by Sternberg and colleagues, which borrows on the procedures described above). Nevertheless, this approach has led to some promising results. For example, a measure of academic achievement (a composite of high school and college GPA, SAT, and college board achievement test scores) was correlated with several tacit knowledge items, ranging from r =.45 (not playing a sport or exercise) to r =.23 (not attending optional weekly review sessions). Similarly, tacit knowledge items were correlated with academic adjustment (a composite of scales assessing happiness, self-perceived success both with tacit knowledge and college success, and rated closeness of college to the individuals ideal college), ranging from r =.42 (being positive) to r =.27 (being flexible). It remains difficult, however, to fully substantiate the veracity of these claims, since the study upon which this research was based (Williams & Sternberg), while being cited twice, has not been published (see Sternberg et al., 1993, 2000). Even so, the tacit knowledge items assessed here appear to capture aspects of personality as much as anything, and it is an open empirical question, whether they are simply proxies of the Big Five personality constructs. 4.2.5. Military settings Tacit knowledge measures have also been devised for military leaders and recruits. In one study, Eddy (1988, cited in Sternberg et al., 2000) used the TKIM and ASVAB scores to determine relations between these measures. The sample comprised 631 Air Force enlistees. Eddy (1988) found low to modest correlations between TKIM and all 10 subtests of the ASVAB, with some of these correlations even negative in sign. Given the lawfulness of positive manifold, this result is somewhat striking—it is conceivable tacit knowledge is not therefore a ‘‘strict’’ ability construct. Nevertheless, we might also question the relevance of the instrument used here. The TKIM seems a notably obtuse scale to assess tacit knowledge

148

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

in enlistees. Nevertheless, one striking feature of the study was minimal correlation among age, sex, and tacit knowledge, a finding indicating that the TKIM may reduce the effects of adverse impact, especially given correlations among ASVAB scores, race, and sex (as indeed Herrnstein & Murray, 1994, found) were significant. In three studies examining the performance of officers on the TKML, Hedlund, Horvarth, Forysthe, et al. (1998) found significant convergent validity for the TKML and measures of leadership effectiveness. The authors also report low correlations (sometimes negative in sign) between the TKML and measures of cognitive ability. However, Gottfredson (2002) has raised some problems with the conclusions reached in the report, since the findings do not replicate well across studies. Even so, Hedlund et al. (1998) do provide some evidence that tacit knowledge has incremental predictive validity for various criterion related to success in the military. 4.2.6. Concluding comments The assessment of tacit knowledge appears to have certain promise in providing incremental predictive validity (over and above cognitive ability) for success and achievement in various real-world environments. However, there is clearly a need to determine whether or not tacit knowledge measures, as currently defined, will add incremental validity when constructs such as personality, interests, and integrity are also assessed against the criterion of success. Moreover, as proponents of this concept themselves suggest, it is not certain to what extent tacit knowledge, social, and EI measures are structurally independent (see Hedlund & Sternberg, 2000). This research question would appear worthy of detailed consideration inside the context of a large-scale multivariate investigation, with suitable predictor variables. Equally important would appear the need to examine structural relationships between measures of tacit knowledge and implicit learning paradigms of the type designed by Reber (1993) (Mackintosh, 1998). Conceptually at least, tacit knowledge shares parallels with implicit learning, since the latter has been defined as ‘‘the acquisition of knowledge that takes place largely independent of conscious attempts to learn and largely in the absence of explicit knowledge of what was acquired’’ (Reber, 1993, p. 5). This proposition is made more attractive because the correlation between implicit learning (measured using cognitive paradigms) and general intelligence tends to mirror the correlations found by Sternberg et al. using the methodologies outlined previously (i.e., near-zero relationships) (see Reber, Walkenfeld, & Hernstadt, 1991). Notwithstanding, we also contend that the paradigms currently available to assess tacit knowledge could be greatly improved. To date, tacit knowledge measures still depend on subjective responses to case-based scenarios that are scored by using correspondence to the expert as criteria. Recently, however, Markham and Roberts (in preparation) have developed more objective forms of tacit knowledge items that have responses that can be veridically determined using standards available from empirical research, professional guidelines, and the like. Furthermore, in an appreciation of other issues raised in these passages, performance in these ‘‘new’’ tacit knowledge tests are currently being analyzed with respect to measures of implicit learning, Gf, Gc, personality, and tacit knowledge (as measured using the quasi-

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

149

objective protocols developed by Sternberg et al.). Predictor variables have also been collected on a sample size that is highly adequate for multivariate analyses (N = 146). 4.3. Metacognitive abilities Another avenue of research emerging as a new construct to be examined by differential psychologists concerns metacognition or ‘‘knowing about knowing’’ (Metcalfe & Shimamura, 1994). Within the literature, a distinction is usually made between knowledge about cognition and processes of metacognitive control—typically those of planning, monitoring, and evaluation (e.g., Schraw & Moshman, 1995). Metacognitive abilities are widely regarded as integral to effective learning (e.g., Flavell, 1987) and to strategy selection in performance on cognitive tasks (e.g., Schraw, 1997). Indeed, Sternberg (1997) claims that these higher order, executive processes are central to all forms of intellectual functioning. Other researchers, however, propose a more peripheral role for metacognition, suggesting that it may simply constitute a form of cognitive ability (rather than being the essence of intelligence) (Stankov, 2000; Stankov & Dolph, 2000). 4.3.1. The ‘‘calibration paradigm’’ and measures of ‘‘realism’’ In recent decades, there has been considerable research into the efficacy of individuals’ confidence ratings in predicting their performance on cognitive tests (see e.g., Ferrell, 1994; Gigerenzer, Hoffrage, & Kleinbo¨lting, 1991; Juslin, 1993; Lichtenstein & Fischhoff, 1977). This domain of investigation derives from the literature on probabilistic decision-making and has been conducted largely within the framework of experimental cognitive psychology. The focus of these studies is usually on the ‘‘realism’’ or ‘‘calibration’’ of confidence ratings (i.e., the correspondence between subjective probability judgments and relative frequencies) (Juslin, 1993). In a typical paradigm, participants are presented with items in a cognitive test and asked to provide a response. Following each response, participants then provide a confidence rating (usually expressed as a percentage) indicating how ‘‘sure’’ they are that the answer is correct. Confidence judgments are generally considered to reflect an individual’s capacity for self-monitoring (i.e., one’s ‘‘online’’ awareness of understanding and task performance) (Schraw & Moshman, 1995). Participants are considered to be well calibrated if these subjective probabilities are realized in terms of corresponding relative frequencies (Juslin, 1993). If the mean confidence rating across all items is significantly higher (or lower) than the mean percentage correct, then participants are considered to be poorly calibrated. Several indices can be extracted from accuracy and confidence scores to evaluate the realism of confidence judgments (see Keren, 1991; Lichtenstein, Fischhoff, & Phillips, 1982, for a review). However, Stankov and Crawford (1996) suggest that the overconfidence parameter (i.e., ‘‘bias’’ score) alone has satisfactory reliability. The bias score is simply the difference between the mean confidence rating (over all items in the test) and the percentage of correctly solved items. A positive difference indicates overconfidence, while a negative difference demonstrates underconfidence. The realism of confidence judgments improves as the bias score approaches zero.

150

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

4.3.2. Major empirical findings from calibration studies Several significant and replicable findings have emerged from the calibration literature. The first is a pervasive tendency towards overconfidence on (although not restricted to) tests of general knowledge (Bjo¨rkman, 1994; Gigerenzer et al., 1991; Juslin, 1993; Lichtenstein & Fischhoff, 1977). Furthermore, it has been demonstrated that the degree of overconfidence observed is mediated by the difficulty of the task. Thus, overconfidence is typically observed on difficult tasks (low mean percentage correct) and is eliminated or even reversed on easy tasks (high mean percentage correct) (e.g., Juslin & Olsson, 1997; Kahneman & Tversky, 1996). This finding has been labeled both the difficulty effect (Lichtenstein & Fischhoff, 1977) and the hard/easy effect (Gigerenzer et al., 1991). The results of calibration studies within the sensory domain are more equivocal. While Baranski and Petrusic (1995) and Petrusic and Baranski (1997) claim to have identified the same hard/easy effect in both general knowledge and perceptual discrimination tasks, Juslin and colleagues (Bjo¨rkman, Juslin, & Winman, 1993; Juslin & Olsson, 1997; Winman & Juslin, 1993) report a pervasive tendency towards underconfidence in perceptual tasks (cf. Stankov, 2000). These results have produced a point of disjunction between theorists proposing a single mechanism (usually difficulty) for findings in both domains (e.g., Baranski & Petrusic, 1995; Ferrell, 1994) and those that suggest two accounts are necessary (e.g., Juslin & Olsson, 1997). Moreover, these findings have some significance for differential psychologists. Different trends in over/underconfidence scores on general knowledge and perceptual tasks seem to depend on the structural distinction between acculturated knowledge (Gc) and visual processing (Gv). In light of comments made previously, it is our contention that future research in the calibration paradigm might move beyond visual processes to explore the ‘‘new’’ sensory domains of olfaction, tactile, and kinesthetic processes. 4.3.3. Self-confidence: a trait overarching intelligence and personality? Calibration research conducted within an individual differences framework has consistently identified a general self-confidence factor. This construct is defined by confidence ratings from diverse cognitive tests whose accuracy scores define separate factors within Gf– Gc theory (Kleitman & Stankov, 2001; Pallier et al., 2002; Stankov, 1998; Stankov & Crawford, 1996, 1997). There are several implications associated with this finding. First, it appears that confidence judgments across cognitive domains are based on a common underlying process; and, that the differences between perceptual and general knowledge tasks evident in measures of realism are not apparent factorially (Stankov & Crawford, 1996). Stankov (1999) has proposed the self-confidence factor to be an aspect of, or perhaps identical to, self-monitoring. As the trait is neither entirely personality-like nor cognitive in nature it may be viewed as being either ‘‘close to human abilities or as part of the interface between abilities and personality’’ (Stankov, 1999, p. 324). Although metacognitive abilities such as planning and evaluation have been postulated in addition to self-monitoring, at present there is only limited evidence to suggest that these theoretically distinct constructs are independent dimensions of individual differences (Stankov & Dolph, 2000). As such, several studies have been conducted utilizing post-test performance estimates (PTPEs), or frequency judgments, which are considered to reflect the

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

151

evaluative component of metacognition (Kleitman & Stankov, 2001; Stankov & Crawford, 1996). PTPEs are obtained by asking participants to provide an estimate of the percentage of items they have solved correctly immediately after completing a cognitive test. A bias score analogous to those described above for item-by-item confidence judgments can be calculated for PTPEs by taking the difference between the estimated percentage of correct items and the mean percentage correct. While confidence judgments on general knowledge tasks display the aforementioned tendency towards overconfidence, PTPEs typically exhibit good calibration or underevaluation (e.g., Gigerenzer et al., 1991; Stankov & Crawford, 1996). This finding is referred to in the literature as the confidence/frequency effect (Gigerenzer et al., 1991). According to Stankov (2000), the different degrees of realism in measures of selfconfidence and self-evaluation typified by the confidence/frequency effect provide partial evidence for the independence of these constructs. The identification of separate factors corresponding to monitoring and evaluation using factor analytic techniques has also been posited as necessary in demonstrating that metacognitive processes are structurally independent dimensions of individual differences. Stankov (2000) has provided evidence to support this latter condition, although this work clearly requires replication. 4.3.4. Future directions in confidence research Despite extensive research, few meaningful associations have been identified between the self-confidence factor and other seemingly related constructs. Further studies ‘‘mining the noman’s land’’ between personality and cognitive abilities are necessary in order to elucidate the nature of the metacognitive factors previously identified. One area of inquiry that seems to be particularly pertinent to self-confidence research concerns self-concept. Self-concept refers to an individual’s perceptions of her (or his) self, formed through experiences with and interpretations of their environment (Hattie, 1992; Marsh, 1990). Presumably, individuals with greater belief in their cognitive abilities are likely to show greater levels of confidence in their responses on cognitive tasks. To this end, Stankov and Crawford (1997) have identified significant (albeit rather weak) correlations between mathematics self-concept and confidence ratings on the Raven’s Progressive Matrices Test (r =.20); and between English self-concept and confidence indices from a vocabulary test (r =.32). The fact that these correlations are not overly large appears less surprising when consideration is given to the fact that general measures from each domain may have only minimal conceptual overlap. In keeping with modern conceptions of self-concept as multidimensional and hierarchical (see Marsh, 1990), it is likely that the association between measures of self-concept specific to particular cognitive domains and confidence measures from corresponding markers of these abilities will be substantially higher (see Kleitman & Stankov, submitted for publication). On a more applied level, the predictive utility of self-confidence measures within an organizational (or industrial) setting remains untested, although it is conceivable that these measures would have substantial predictive power—particularly in roles that require decision making under conditions of uncertainty. In the sphere of education, inquiry into the development of metacognitive abilities over time may be beneficial to informing both learning and

152

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

teaching practices. For example, conceivably self-confidence measures might be used to provide ancillary information in standardized tests that have been used to select students for special education, college instruction, or various forms of intervention. Self-confidence measures might also be implemented with all of the tests of the various ‘‘new’’ constructs discussed in this paper to provide improved understanding of their underlying psychological mechanisms.

5. Summary and conclusions We have argued that the study of human cognitive abilities lies at something of a crossroads. On the one hand, there are researchers who believe the discipline has uncovered the essence of individual differences—general intelligence—and that all research efforts should be expended to provide a proper explanatory model of this construct and its antecedents. On the other hand, there are exponents whose primary aim is to uncover and/ or develop taxonomic models encapsulating the myriad of constructs (some of them presently perhaps even unknown) that might rightfully represent the ways in which humans differ in cognitive performance and its various manifestations. Even if g does exist, it is worth noting that explanatory models of each (and every) broad cognitive ability (and we include new constructs among these) may be required before the more ambitious undertaking of developing a scientific model of general intelligence is attempted. This suggestion is by no means trivial—even the staunchest advocates of g cannot rule out the possibility that it represents an amalgam of all second-order constructs. This point gains currency when one considers that studies designed to investigate the cognitive and biological correlates of g are constrained by the tests serving as proxies for the general factor (i.e., adequate sampling of the universe of tests constituting g is practically impossible). Notably, this suggestion renders the disjunction between g-centered and taxonomic approaches less divisive. There is yet another point that makes understanding taxonomic models of human cognitive abilities and a focus on ‘‘new’’ constructs an important endeavor. It is our contention that the capabilities indicating human intelligence change over time as a function, in particular, of technological and cultural evolution (see e.g., Horn & Noll, 1994; Roberts et al., 2000). New capabilities appear with every innovation (e.g., computer proficiency), while competencies that were once very important (e.g., spelling ability), in the face of these technologies, have taken on a less important role. For instance, knowing how to perform factor analysis using vector algebra (an attribute once viewed as highly intelligent) brings no rewards to the modern psychometrician and the word processor’s spell-checking tool has made lexical ability less important than it once was to an individual’s attainments. In light of the dynamic nature of cognitive abilities, it is conceivable that the taxonomic models of the next century may even look different to the one’s of this century. This of course begs several questions, which research should serve to elucidate. Is the search for g illusory in the face of technological changes (a proposition that could be dismissed if g is taken to represent the transfer of abilities to emerging domains)? What constructs might remain factorially stable in the light of the dynamic nature of cognitive abilities? What broad abilities are likely to

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

153

become obsolete or change radically in the face of these changes? The emergence of so-called wearable intelligence renders these types of questions of more immediate concern to differential psychology than is perhaps self-evident (see Kyllonen et al., in preparation). In this paper, we have argued that the science of human cognitive abilities should focus on extending structural models and exploring new constructs. A complete model would attempt to circumscribe all of the constructs presently reviewed—at present an undertaking that no one researcher has attempted. We believe, however, that this issue is in urgent need of attention. We have also suggested that giving exclusive focus to psychometric g may be damaging to the field. Indeed, even if g is proven to be the most important construct in individual differences, it will only be when the cognitive domain is fully chartered that a falsifiable scientific model of it is possible.

References Baranski, J. V., & Petrusic, W. M. (1995). On the calibration of knowledge and perception. Canadian Journal of Experimental Psychology, 49, 397 – 407. Bar-On, R. (1997). The emotional intelligence inventory (EQ-i): technical manual. Toronto: Multi-Health Systems. Belmont, J. M., Butterfield, E. C., & Ferretti, R. P. (1982). To secure transfer of training instruct self-management skills. In D. K. Detterman, & R. J. Sternberg (Eds.), How and how much can intelligence be increased? (pp. 147 – 154). Norwood, NJ: Ablex. Bjo¨rkman, M. (1994). Internal cue theory: calibration and resolution of confidence in general knowledge. Organizational Behavior and Human Decision Processes, 58, 386 – 405. Bjo¨rkman, M., Juslin, P., & Winman, A. (1993). Realism of confidence in sensory discrimination: the underconfidence phenomenon. Perception and Psychophysics, 54, 75 – 81. Boring, E. G. (1923). Intelligence as the tests test it. The New Republic, 35, 35 – 37. Braden, J. P. (1989). Fact or artifact? An empirical test of Spearman’s hypothesis. Intelligence, 13, 149 – 155. Carroll, J. B. (1992). Cognitive abilities: the state of the art. Psychological Science, 3, 266 – 270. Carroll, J. B. (1993). Human cognitive abilities: a survey of factor-analytic studies ( pp. 429 – 452). New York: Cambridge University Press. Carroll, J. B. (1995). On methodology in the study of cognitive abilities. Multivariate Behavioral Research, 30, 429 – 452. Ciarrochi, J., Chan, A., & Caputi, P. (2000). A critical evaluation of the emotional intelligence construct. Personality and Individual Differences, 28, 539 – 561. Costa Jr., P. T., & McCrae, R. R. (1992). NEO PI-R professional manual. Odessa, FL: Psychological Assessment Resources. Danthiir, V., Pallier, G., Roberts, R. D., & Stankov, L. (2001). What the nose knows: olfaction within the structure of human cognitive abilities. Intelligence, 30, 337 – 361. Davies, M., Stankov, L., & Roberts, R. D. (1998). Emotional intelligence: in search of an elusive construct. Journal of Personality and Social Psychology, 75, 989 – 1015. Dawda, D., & Hart, S. D. (2000). Assessing emotional intelligence: reliability and validity of the Bar-On Emotional Quotient Inventory (EQ-i) in university students. Personality and Individual Differences, 28, 797 – 812. Derksen, J., Kramer, I., & Katzko, M. (2002). Does a self-report measure for emotional intelligence assess something different than general intelligence? Personality and Individual Differences, 32, 37 – 48. Detterman, D. K. (1982). Does g exist? Intelligence, 6, 99 – 108. Detterman, D. K. (2002). General intelligence: cognitive and biological explanations. In R. J. Sternberg, & E. Grigorenko (Eds.), General factor of intelligence: fact or fiction? (pp. 223 – 244). Los Angeles, CA: Erlbaum.

154

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

Ferrell, W. R. (1994). Calibration of sensory and cognitive judgments: a single model for both. Scandinavian Journal of Psychology, 35, 297 – 314. Flavell, J. H. (1987). Speculations about the nature and development of metacognition. In F. E. Weinert, & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 21 – 29). Hillsdale, NJ: Erlbaum. Garcia, A., Roberts, R. D., Zeidner, M., & Matthews, G. (in preparation). Performance-based emotional intelligence assessed in an Uruguayan population: further problems. Gardner, H. (1983). Frames of mind: the theory of multiple intelligences. New York: Basic Books. Gigerenzer, G., Hoffrage, U., & Kleinbo¨lting, H. (1991). Probabilistic mental models: a Brunswikian theory of confidence. Psychological Review, 98, 506 – 528. Goleman, D. (1995). Emotional intelligence. New York: Bantam Books. Gottfredson, L. S. (2002). Dissecting practical intelligence theory: its claims and evidence. Intelligence, 30, 1 – 55. Gustafsson, J. -E. (1992a). The relevance of factor analysis for the study of group differences. Multivariate Behavioral Research, 27, 239 – 248. Gustafsson, J. -E. (1992b). The ‘‘Spearman hypothesis’’ is false. Multivariate Behavioral Research, 27, 265 – 267. Guttman, L. (1992). The irrelevance of factor analysis for the study of group differences. Multivariate Behavioral Research, 27, 175 – 204. Haier, R. J., Siegel Jr., B. V., Nuechterlein, K. H., Hazlett, E., Wu, J. C., Paek, J., Browning, H. L., & Buchsbaum, M. S. (1988). Cortical glucose metabolic rate correlates of abstract reasoning and attention studied with positron emission tomography. Intelligence, 12, 199 – 217. Haier, R. J., Siegel Jr., B. V., Tang, C., Abel, L., & Buchsbaum, M. S. (1992). Intelligence and changes in regional cerebral glucose metabolic rate following learning. Intelligence, 16, 415 – 426. Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum. Hearnshaw, L. S. (1951). Exploring the intellect. British Journal of Psychology, 42, 315 – 321. Hedlund, J., Horvarth, J. A., Forysthe, G. B., Snook, S., Williams, W. M., Bullis, R. C., Dennis, M., & Sternberg, R. J. (1998). Tacit knowledge in military leadership: evidence of construct validity. Technical Report 1080. Alexandria, VA: U.S. Army Research Institute for the Behavioral Sciences. Hedlund, J., & Sternberg, R. J. (2000). Too many intelligences? Integrating social, emotional, and practical intelligence. In R. Bar-On, & J. D. A. Parker (Eds.), The handbook of emotional intelligence (pp. 171 – 191). San Francisco, CA: Jossey-Bass. Herrnstein, R. J., & Murray, C. (1994). The bell curve: intelligence and class structure in American life. New York: Free Press. Horn, J. L. (1985). Remodeling old models of intelligence. In B. B. Wolman (Ed.), Handbook of intelligence: theories, measurements, and applications (pp. 267 – 300). New York: Wiley. Horn, J. L. (1987). A context for understanding information processing studies of human abilities. In P. A. Vernon (Ed.), Speed of information-processing and intelligence (pp. 201 – 238). Norwood, NJ: Ablex. Horn, J. L. (1989). Cognitive diversity: a framework of learning. In P. L. Ackerman, R. J. Sternberg, & R. Glaser (Eds.), Learning and individual differences: advances in theory and research. A series of books in psychology (pp. 61 – 116). New York: W.H. Freeman. Horn, J. L. (1998). A basis for research on age differences in cognitive capabilities. In J. J. McArdle, & R. W. Woodcock (Eds.), Human cognitive abilities in theory and practice (pp. 57 – 91). Mahwah, NJ: Erlbaum. Horn, J. L., & Hofer, S. M. (1992). Major abilities and development in the adult period. In R. J. Sternberg, & C. Berg (Eds.), Intellectual development (pp. 44 – 99). New York: Cambridge University Press. Horn, J. L., & Noll, J. (1994). System for understanding cognitive capabilities: a theory and the evidence on which it is based. In D. K. Detterman (Ed.), Current topics in human intelligence, vol. IV (pp. 151 – 203). New York: Springer-Verlag. Horn, J. L., & Noll, J. (1997). Human cognitive capabilities: Gf – Gc theory. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: theories, tests, and issues (pp. 53 – 91). New York: Guilford Press. Horn, J. L., & Stankov, L. (1982). Auditory and visual factors of intelligence. Intelligence, 6, 165 – 185.

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

155

Howe, M. J. A. (1990). Does intelligence exist? Psychologist, 3, 490 – 493. Humphreys, L. G. (1979). The construct of general intelligence. Intelligence, 3, 105 – 120. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72 – 98. Intelligence and its measurement: a symposium (1921). Journal of Educational Psychology, 12, 123 – 147, 195 – 216, 271 – 275. Jensen, A. R. (1970). Hierarchical theories of mental ability. In B. Dockrell (Ed.), On intelligence (pp. 119 – 190). Toronto: Ontario Institute of Education. Jensen, A. R. (1974). Interaction of Level I and Level II abilities with race and socio-economic status. Journal of Educational Psychology, 66, 99 – 111. Jensen, A. R. (1979). g: outmoded theory or unconquered frontier? Creative Science and Technology, 2, 16 – 29. Jensen, A. R. (1985). The nature of black – white differences on various psychometric tests: Spearman’s hypothesis. Behavioral and Brain Sciences, 8, 193 – 263. Jensen, A. R. (1992a). Understanding g in terms of information processing. Educational Psychology Review, 4, 271 – 308. Jensen, A. R. (1992b). Spearman’s hypothesis: methodology and evidence. Multivariate Behavioral Research, 27, 225 – 234. Jensen, A. R. (1992c). More on psychometric g and ‘‘Spearman’s hypothesis’’. Multivariate Behavioral Research, 27, 257 – 260. Jensen, A. R. (1993). Spearman’s hypothesis tested with chronometric information processing tasks. Intelligence, 17, 47 – 77. Jensen, A. R. (1998). The g factor: the science of mental ability. Westport, CT: Praeger. Jensen, A. R., & Sinha, S. N. (1992). Physical correlates of human intelligence. In P. A. Vernon (Ed.), Biological approaches to the study of human intelligence (pp. 139 – 242). Norwood, NJ: Ablex. Jensen, A. R., & Weng, L. -J. (1994). What is a good g? Intelligence, 18, 231 – 258. Juslin, P. (1993). An explanation of the hard-easy effect in studies of realism of confidence in one’s general knowledge. European Journal of Cognitive Psychology, 5, 55 – 71. Juslin, P., & Olsson, H. (1997). Thurstonian and Brunswikian origins of uncertainty in judgment: a sampling model of confidence in sensory discrimination. Psychological Review, 104, 344 – 366. Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 103, 582 – 591. Keren, G. (1991). Calibration and probability judgments: conceptual and methodological issues. Acta Psychologica, 77, 217 – 273. Kleitman, S., & Stankov, L. (2001). Ecological and person-oriented aspects of metacognitive processes in testtaking. Applied Cognitive Psychology, 15, 321 – 341. Kleitman, S., & Stankov, L. (submitted for publication). The trait of self-confidence and its associations with metacognitive beliefs and cognitive strategies: an individual differences perspective. Personality and Individual Differences. Kyllonen, P. C., Roberts, R. D. & Stankov, L. (Eds.) (in preparation). Extending intelligence: enhancement and new constructs. Mahwah, NJ: Erlbaum. Levy, P. (1992). Inspection time and its relation to intelligence: issues of measurement and meaning. Personality and Individual Differences, 13, 987 – 1002. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159 – 183. Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: the state of the art in 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: heuristics and biases (pp. 306 – 334). Cambridge, UK: Cambridge University Press. Loehlin, J. C. (1992). Guttman on factor analysis and group differences: a comment. Multivariate Behavioral Research, 27, 235 – 238. MacCann, C., Roberts, R. D., Matthews, G., & Zeidner, M. (submitted for publication). Effects of empirical option

156

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

weighting on the reliability and validity of performance-based emotional intelligence (EI) tests. Personality and Individual Differences. Mackintosh, N. J. (1998). IQ and human intelligence. Oxford, UK: Oxford University Press. Markham, P. M., & Roberts, R. D. (in preparation). Can tacit knowledge and implicit learning predict college success? A multivariate investigation. Marsh, H. W. (1990). A multidimensional, hierarchical model of self-concept: theoretical and empirical justification. Educational Psychology Review, 2, 77 – 172. Matthews, G., Zeidner, M., & Roberts, R. D. (in press). Emotional intelligence: science and myth. Boston: MIT Press. Mayer, J. D., Caruso, D., & Salovey, P. (1999). Emotional intelligence meets traditional standards for an in telligence. Intelligence, 27, 267 – 298. Mayer, J. D., & Cobb, C. D. (2000). Educational policy on emotional intelligence: does it make sense? Educational Psychology Review, 12, 163 – 183. Mayer, J. D., & Salovey, P. (1993). The intelligence of emotional intelligence. Intelligence, 17, 433 – 442. Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey, & D. J. Sluyter (Eds.), Emotional development and emotional intelligence: educational implications (pp. 3 – 31). New York: Basic Books. Mayer, J. D., Salovey, P., & Caruso, D. (2000a). Models of emotional intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence (pp. 396 – 420). New York: Cambridge University Press. Mayer, J. D., Salovey, P. R., & Caruso, D. R. (2000b). Emotional intelligence as Zeitgeist, as personality, and as a mental ability. In R. Bar-On, & J. D. A. Parker (Eds.), The handbook of emotional intelligence (pp. 99 – 117). San Francisco, CA: Jossey-Bass. Metcalfe, J., & Shimamura, A. P. (Eds.) (1994). Metacognition: knowing about knowing. Cambridge, MA: MIT Press. Metz, J. T., Yassillo, N. J., & Cooper, M. (1987). Relationship between cognitive functioning and cerebral metabolism. Journal of Cerebral Blood Flow and Metabolism, 7(Suppl. 1), S305. Motowildo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: the low-fidelity simulation. Journal of Applied Psychology, 75, 640 – 647. Nettelbeck, T. (2001). Correlation between inspection time and psychometric abilities: a personal interpretation. Intelligence, 29, 459 – 474. Nevo, B., & Chawarski, M. C. (1997). Individual differences in practical intelligence and success in immigration. Intelligence, 25, 83 – 92. Pallier, G., Roberts, R. D., & Stankov, L. (2000). Biological versus psychometric intelligence: Halstead (1947) Re-visited. Archives of Clinical Neuropsychology, 15, 205 – 226. Pallier, G., Wilkinson, R., Danthiir, V., Kleitman, S., Knezevic, G., Stankov, L., & Roberts, R. D. (2002). The role of question format and individual differences in the realism of confidence judgments. Journal of General Psychology, 129, 257 – 259. Paulhus, D. L., Lysy, D. C., & Yik, M. S. (1998). Self-report measures of intelligence: are they useful as proxy IQ tests? Journal of Personality, 66, 525 – 554. Petrusic, W. M., & Baranski, J. V. (1997). Context, feedback, and the calibration and resolution of confidence in perceptual judgments. American Journal of Psychology, 110, 543 – 572. Polanyi, M. (1966). The tacit dimensions. Garden City, NY: Doubleday. Reber, A. S. (1993). Implicit learning and tacit knowledge: an essay on the cognitive unconscious. Oxford, UK: Oxford University Press. Reber, A. S., Walkenfield, F. F., & Hernstadt, R. (1991). Implicit learning and explicit learning: Individual differences and IQ. Journal of Experimental Psychology: Learning, Memory and Cognition, 17, 888 – 896. Roberts, R. D. (2002). Review of Bar-On and Parker’s (Editors). The Handbook of Emotional Intelligence. Intelligence, 30, 209 – 210. Roberts, R. D., Goff, G. N., Anjoul, F., Kyllonen, P. C., Pallier, G., & Stankov, L. (2000). The Armed Services Vocational Aptitude Battery: not much more than acculturated learning (Gc)!? Learning and Individual Differences, 12, 81 – 103.

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

157

Roberts, R. D., Pallier, G., & Goff, G. N. (1999). Sensory processes within the structure of human cognitive abilities. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: process, trait, and content determinants (pp. 339 – 370). Washington, DC: American Psychological Association. Roberts, R. D., & Stankov, L. (1999). Individual differences in speed of mental processing and human cognitive abilities: towards a taxonomic model. Learning and Individual Differences, 11, 1 – 120. Roberts, R. D., Stankov, L., Pallier, G., & Dolph, B. (1997). Charting the cognitive sphere: tactile and kinesthetic performance within the structure of intelligence. Intelligence, 25, 111 – 148. Roberts, R. D., Zeidner, M., & Matthews, G. (2001). Does emotional intelligence meet traditional standards for an ‘intelligence’? Some new data and conclusions. Emotion, 1, 196 – 231. Roskam, E. E., & Ellis, J. (1992). Commentary on Guttman: the irrelevance of factor analysis for the study of group differences. Multivariate Behavioral Research, 27, 205 – 218. Rushton, J. P. (1999). Race, evolution, and behavior. New Brunswick, NJ: Transaction Publishers. Salovey, P., Bedell, B. T., Detweiler, J. B., & Mayer, J. D. (2000). Current directions in emotional intelligence research. In M. Lewis, & J. M. Haviland-Jones (Eds.), Handbook of emotions (pp. 504 – 520). New York: Guilford Press. Schaie, K. W. (2001). Emotional intelligence: psychometric status and developmental characteristics—comment on Roberts, Zeidner, and Matthews. Emotion, 1, 243 – 248. Scho¨nemann, P. H. (1992). Extension of Guttman’s result from g to PC1. Multivariate Behavioral Research, 27, 219 – 224. Schraw, G. (1997). The effect of generalized metacognitive knowledge on test performance and confidence judgments. Journal of Experimental Education, 65, 135 – 146. Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7, 351 – 371. Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golden, C. J., & Dornheim, L. (1998). Development and validation of a measure of emotional intelligence. Personality and Individual Differences, 25, 167 – 177. Shipley, W. C. (1983). Shipley Institute of Living Scale. Los Angeles, CA: Western Psychological Services. Spearman, C. (1904). General intelligence, objectively determined and measured. American Journal of Psychology, 15, 201 – 293. Spearman, C. (1923). The nature of intelligence and the principles of cognition. London: Macmillan. Spearman, C. (1927). The abilities of man. New York: Macmillan. Stankov, L. (1987). Level I/Level II: a theory ready to be archived. In S. Modgil, & C. Modgil (Eds.), Arthur Jensen: consensus and controversy (pp. 25 – 37). London: Falmers Press. Stankov, L. (1998). Calibration curves, scatterplots and the distinction between general knowledge and perceptual tests. Learning and Individual Differences, 8, 28 – 51. Stankov, L. (1999). Mining on the ‘‘no man’s land’’ between intelligence and personality. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: process, trait, and content determinants (pp. 314 – 337). Washington, DC: American Psychological Association. Stankov, L. (2000). Complexity, metacognition, and fluid intelligence. Intelligence, 28, 121 – 143. Stankov, L. (2002). ‘g’: a diminutive general. In R. J. Sternberg, & E. Grigorenko (Eds.), General factor of intelligence: fact or fiction? (pp. 19 – 37). Los Angeles, CA: Erlbaum. Stankov, L., Boyle, G. J., & Cattell, R. B. (1995). Models and paradigms in intelligence research. In D. Saklofske, & M. Zeidner (Eds.), International handbook of personality and intelligence (pp. 15 – 43). New York: Plenum. Stankov, L., & Crawford, J. (1996). Confidence judgments in studies of individual differences. Personality and Individual Differences, 21, 971 – 986. Stankov, L., & Crawford, J. (1997). Self-confidence and performance on cognitive tests. Intelligence, 25, 93 – 109. Stankov, L., & Dolph, B. (2000). Metacognitive aspects of test-taking and intelligence. Psychologische Beitra¨ge, Band, 42, 213 – 227.

158

D.B. Bowman et al. / Learning and Individual Differences 13 (2002) 127–158

Stankov, L., & Dunn, S. (1993). Physical substrata of mental energy: the number of neurons and efficient cerebral metabolic processes. Learning and Individual Differences, 5, 241 – 257. Stankov, L., Horn, J. L., & Roy, T. (1980). On the relationship between Gf/Gc theory and Jensen’s Level I/Level II theory. Journal of Educational Psychology, 72(1), 21 – 44. Stankov, L., & Roberts, R. D. (1997). Mental speed is not the ‘basic’ process of intelligence. Personality and Individual Differences, 22, 69 – 85. Stankov, L., Seizova-Cajic, T., & Roberts, R. D. (2001). Tactile and kinesthetic perceptual processes within the taxonomy of human cognitive abilities. Intelligence, 29, 1 – 29. Sternberg, R. J. (1985). Beyond IQ: a triarchic theory of human intelligence. Cambridge: Cambridge University Press. Sternberg, R. J. (1993). Sternberg triarchic abilities test. Unpublished manuscript. Sternberg, R. J. (1996). Successful intelligence. New York: Simon & Schuster. Sternberg, R. J. (1997). The triarchic theory of intelligence. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: theories, tests, and issues ( pp. 92 – 104). New York: Guilford Press. Sternberg, R. J., & Detterman, D. K. (Eds.) (1986). What is intelligence? Contemporary viewpoints on its nature and definition. Norwood, NJ: Ablex. Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., Snook, S. A., & Grigorenko, E. L. (2000). Practical intelligence in everyday life. Cambridge, UK: Cambridge University Press. Sternberg, R. J., & Grigorenko, E. L. (2000). Practical intelligence and its development. In R. Bar-On, & J. D. A. Parker (Eds.), The handbook of emotional intelligence (pp. 215 – 243). San Francisco, CA: Jossey-Bass. Sternberg, R. J., Nokes, C., Geissler, P. W., Prince, R., Okatcha, F., Bundy, D. A., & Grigorenko, E. L. (2001). The relationship between academic and practical intelligence: a case study in Kenya. Intelligence, 29, 401 – 418. Sternberg, R. J., Wagner, R. K., & Okagaki, L. (1993). Practical intelligence: the nature and role of tacit knowledge in work and at school. In J. M. Puckett, & H. W. Reese (Eds.), Mechanisms of everyday cognition (pp. 205 – 227). Hillsdale, NJ: Erlbaum. Sternberg, R. J., Wagner, K., Williams, W. M., & Horvath, J. A. (1995). Testing common sense. American Psychologist, 50, 912 – 927. Wagner, R. K. (2000). Practical intelligence. In R. J. Sternberg (Ed.), Handbook of human intelligence (2nd ed.) (pp. 380 – 395). New York: Cambridge University Press. Wagner, R. K., & Sternberg, R. J. (1985). Practical intelligence in real world pursuits. Journal of Personality and Social Psychology, 49, 436 – 458. Willerman, L. (1991). Commentary on Rushton’s Mongoloid – Caucasoid differences in brain size. Intelligence, 15, 361 – 364. Winman, A., & Juslin, P. (1993). Calibration of sensory and cognitive judgments: two different accounts. Scandinavian Journal of Psychology, 34, 135 – 148. Zeidner, M., Roberts, R. D., & Matthews, G. (in press). Can emotional intelligence be schooled? A critical review. Educational Psychology.