- Email: [email protected]
Development and Validation of an International English Big-Five Mini-Markers
Personality and Individual Differences 45 (2008) 542–548
Contents lists available at ScienceDirect
Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid
Development and Validation of an International English Big-Five Mini-Markers q Edmund R. Thompson * School of Management, University of Bath, Bath BA2 7AY, United Kingdom
a r t i c l e
i n f o
Article history: Received 11 November 2007 Received in revised form 12 May 2008 Accepted 12 June 2008
Keywords: International English Mini-Markers Big-five Scale development and validation Saucier
a b s t r a c t Examination of Saucier (1994) big-five Mini-Markers using a multinational sample (N = 491) found its psychometric properties suboptimal. Using further multinational samples, through a qualitative study (N = 23) appraising items and then a series of quantitative development and validation studies (total N = 3,068), a revised marker set was derived. This new International English Mini-Markers (a) produced better factor structures, higher scale internal consistency reliabilities, and greater orthogonality than the original set of items, (b) prove to have temporal stability, and (c) acceptable convergent validity. Ó 2008 Elsevier Ltd. All rights reserved.
1. Introduction Personality dimensions have long attracted research attention (Eysenck, 1991). The five-factor model of personality (Goldberg 1990; McCrae and Costa; 1987) has met with particularly wide application in the personality field, plus across disciplines as diverse as aviation (Grant et al., 2007), politics (Schoen & Schumann, 2007), and entrepreneurship (Zhao & Seibert 2006). The model’s wide disciplinary application has prompted the development of several brief big-five measures comprising fewer than 50 singleitem adjectives or only a very few statement-based items specifically for use in applied research settings where respondent time or instrument space are constrained (e.g. Gosling, Rentfrow, & Swann, 2003; Langford, 2003; Saucier, 1994; Woods & Hampson, 2005). While, as Church (2001) notes, some debate has existed about the uniform replicability of the five-factor model in emically developed measures across some cultures such as China (Cheung et al., 2001), the model’s general cross-cultural applicability has, nevertheless, stimulated translations of relatively long big-five measures into Chinese (Trull & Geary, 1997), Croatian (Mlacic & Goldberg, 2007), Italian (Terracciano, 2003), Spanish (Garcia, Aluja, & Garcia, 2004), and other languages. Such translations tend to support the broad cross-cultural applicability of the five-factor model. For example, Hendriks, Hofstee, and De Raad (1999) developed interactively in Dutch, English and German a 100-statement big-five
q Support for this research was provided by the Japan Society for the Promotion of Science, Grant No. 16330082. * Tel.: +44 1225 386742; fax: +44 1225 386473. E-mail address: [email protected]
0191-8869/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.paid.2008.06.013
measure that confirmed the five-factor structure in these languages and 11 others into which it was subsequently translated, including Chinese and Japanese. More recently, Schmitt, Allik, McCrae, and Benet-Martínez (2007) found robust support for the five-factor model across world regions in a study covering 56 nations using a 44-statement (173-word) big-five measure. Some brief big-five measures have also been translated, although as yet only into German (Muck, Hell, & Gosling, 2007; Rammstedt & John 2007) and Swedish (Hochwalder, 2006). The relatively low number of languages into which any single brief big-five measure has so far been translated has meant that some forms of applied cross-cultural research demanding brief measures have been constrained. Most particularly, applied research in cross-cultural settings where multiple national backgrounds can (a) be anticipated, are (b) of specific interest, but (c) cannot necessarily be known precisely in advance is currently impossible, except, as Thompson (2007) suggests, using English measures. Such research settings include international government bodies and firms, plus, of course, many educational institutions, where populations increasingly comprise individuals from numerous countries who, despite not always being native English-speakers, are obliged to operate organizationally in English. Because English big-five measures have been emically developed, predominantly among native English-speakers in North America, research is needed to assess their psychometric properties among non-native English-speakers before they can with confidence be used in international research settings. No such research has yet been undertaken. The six studies reported in this paper, first, examine the psychometric performance of Saucier’s (1994) big-five Mini-Markers, in an English-using, multinational sample, then, having found its psychometric properties to be suboptimal, develop and validate an International English Mini-Markers.
543
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
The Mini-Markers, a short-form of Goldberg’s (1992) unipolar lexical big-five measure, is selected for assessment because it has proven to be one of the most psychometrically reliable (Mooradian & Nezlek, 1996) and frequently used brief big-five measures, being employed widely in personality research (Diefendorff & Richard, 2003; McCullough, Emmons, & Tsang, 2002) and in applied settings across several disciplines, including health (Austin, Saklofske, & Egan, 2005), business (Erdheim, Wang, & Zickar 2006), and education (Moon & Illingworth, 2005). Comprising just 40 singleadjective personality descriptors originally selected for their psychometric qualities (Saucier, 1994), the Mini-Markers can be hypothesized to lend itself without translation to use with multinational samples that use English. 2. Study one: Mini-Markers assessment with multinational sample 2.1. Participants Participants comprised 491 family and friends of executive MBA students at an English-based international university in East Asia who had volunteered to assist with research, plus had identified themselves as proficient at English. The sample therefore represents the relatively affluent and educated strata of individuals who might reasonably be encountered in numerous international but Englishspeaking research settings (see Table 1 for demographic details). 2.2. Measure The Mini-Markers was used as Saucier (1994) specifies, except, following Hampson and Goldberg (2006), the original 9-point interval measure anchored on accurate and inaccurate was reduced to 5 points to maximize brevity. This was incorporated into an online instrument addressing career motivations sent individually to participants.
2.3. Analyses and results Following Saucier’s (1994) analytical procedure in his development of the Mini-Markers, a principal component analysis with a varimax rotation specifying five components failed to produce the expected factor structure, with Conscientiousness being the only sub-scale forming a distinct, complete and interpretable component (Table 2). Seven items had loadings below the lowest (.44) reported by Saucier’s (1994, p. 512), and 18 items had loadings failing to meet Saucier’s (1994, p. 509) item-purity criterion of having a highest loading at least double its loading on any other component. Of the 22 items that did meet this criterion, only seven could be said to load on the expected component. Analyses specifying different extraction and rotation procedures failed to produce an improved or acceptable five-factor solution. The mean inter-scale correlation of .31 was considerably higher than Saucier’s (1994, p. 512–514) reported mean inter-scale correlation of .11, indicating considerable lack of orthogonality of sub-scales. However, scale internal consistency reliabilities were adequate, with the exception of Extraversion (Table 3). 2.4. Discussion Given the poor factor structure and high inter-scale correlations, but acceptable internal consistencies for four of the subscales, method variance was suspected. Examination of responses to other measures administered alongside the Mini-Markers, such as Rosenberg’s (1965) self-esteem scale, did not indicate careless or non-diligent responding, suggesting that the suboptimal psychometric performance of the Mini-Markers with a multinational sample was due to an interaction of the nature of the multinational sample and the Mini-Markers. A number of items could be identified as potentially problematic by their low component loadings or high cross-loadings, including energetic, relaxed, touchy, withdrawn, complex, sloppy,
Table 1 Sample characteristics of each study Study 1
Study 3
Study 4
Study 5
491 259
650 346
459 216
1927 704
Age (percent) 18–24 25–29 30 or older
30 33 37
15 48 37
25 36 39
55 25 20
Percent already with undergraduate degree
78
86
83
63
Occupation (percent) Private sector employees Public sector employees Self-employed Postgraduate student Undergraduate student Homemaker, retired or unemployed
33 17 4 17 10 19
49 22 5 11 5 9
34 18 5 19 9 15
26 9 9 8 24 24
408 38
629 31
378 23
1327 68
Burma (54) China (26) Indonesia (27) Mexico (44) Thailand (79) Vietnam (35)
China (50) Indonesia (58) Japan (42) Malaysia (78) Mongolia (39) Philippines (109) Singapore (26) Vietnam (90)
China (22) Indonesia (53) Japan (35) Malaysia (25) Peru (20) Singapore (29) Taiwan (21) Thailand (29) Vietnam (49)
Burma (83) China (884) Hungary (82) India (28) Indonesia (96) Japan (40) Malaysia (37) Philippines (67) Taiwan (32) Thailand (48) Vietnam (32)
N Males
Non-native English-speakers Number of non-native English-speaking countries sample drawn from Non-native English-speaking countries with 20 plus respondents (N)
Note: Countries with fewer than 20 respondents are omitted for space considerations.
544
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
Table 2 Principal component analysis of Saucier’s Mini-Markers with multinational samplea Itemsb A. Kind IO. Imaginative A. Warm IO. Creative A. Sympathetic IO. Deep IO. Intellectual A. Cooperative E. Energetic IO. Philosophical ES. Relaxed A. Harsh ES. Fretful ES. Temperamental A. Rude ES. Envious ES. Jealous ES. Moody A. Cold ES. Touchy E. Withdrawn IO. Complex C. Organized C. Disorganized C. Systematic C. Efficient C. Inefficient C. Sloppy C. Practical C. Careless E. Quiet E. Shy E. Talkative E. Extraverted E. Bashful E. Bold ES. Unenvious A. Unsympathetic IO. Unintellectual IO. Uncreative % of variance explained
Component 1 .69 .64 .62 .62 .61 .60 .58 .55 .52 .48 .38 .21
.29 .12 .10 .27 .20 .29 .14
Component 2
.42 .10 .15 .26 .23 .18 .14 .47 .39 .45 20.37
Component 4
Component 5
.23 .17 .21 .24 .13 .14 .35 .15 .38 .12 .14 .66 .65 .61 .60 .59 .57 .55 .51 .45 .35 .33 .23
.28 .39 .24
Component 3
.32 .44 .33 .12 .30 .10 .26 .12 .30 .26 .19 .16 9.09
.16
.10 .15 .12
.23 .30 .16 .23
.24 .17
.17 .12 .31 .25 .19 .20 .80 .74 .71 .63 .54 .49 .42 .40 .18
.10 .15 .12 .29 .22 6.03
.21 .16 .31 .10 .32
.16 .35 .20 .11 .19
.15 .20 .18 .18
.73 .69 .61 .60 .42 .31 .10 .16 .21 5.75
.23
.25 .12
.17 .15 .20 .71 .59 .57 .48 4.54
N = 491. a Item loadings exceeding .35 are emboldened, those below .10 are omitted. b Letters before item indicate component in Saucier’s (1994) Mini-Markers: E = Extraversion, A = Agreeableness, C = Conscientiousness, ES = Emotional Stability, IO = Intellect or Openness.
Two focus groups were conducted to investigate Mini-Marker items’ clarity, ease of understanding and singularity of meaning. Together, focus groups comprised 13 executive-MBA students and 5 undergraduates, average age 26, half being male, at the same East Asian international university, who came variously from America, Burma, China, Hungary, Indonesia, Mexico, Mongolia, the Philippines, Thailand, Tonga, and Vietnam.
stand rather than multifaceted or multilayered in the sense of having several meritorious qualities indicative of intelligence and intellect, the implicit intention of the Mini-Markers. Relaxed tended to be associated with a transient state of being without external stress or pressure rather than with a permanent internal trait of easygoingness. Together, the lack of familiarity with, or differing comprehensions of, these items could partially account for the method variance and poor factor structure found with the multinational sample. Additionally, nearly all focus group participants remarked on how the alphabetical ordering of adjectives resulted in a slightly confusing block of four items beginning with the un negation (uncreative, unenvious, unintellectual, unsympathetic). Being towards the end of the scale, this block of items may also have increased method variance due to respondent fatigue, as evidenced by the items loading together as the last component to be rotated and by their generally high cross-loadings.
3.2. Results
4. Study three: first scale revision
Certain items were not uniformly familiar to non-native Englishspeakers, either due to relative rarity (bashful, bold, fretful, temperamental, withdrawn) or their colloquial nature (sloppy, touchy). Complex, and relaxed were familiar, but had ambiguous meanings. Complex was taken to mean something like complicated to under-
4.1. Participants
practical, careless, bashful, bold, unsympathetic, unintellectual, and uncreative. However, rather than rely solely on a quantitative approach to identifying and changing problematic items, the MiniMarkers was first assessed qualitatively. 3. Study two: qualitative item appraisal 3.1. Participants
Participants, 650 family and friend volunteers of a different set of MBA students at the same East Asian international university (see Table 1 for demographics), completed instruments online.
545
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548 Table 3 Scale Cronbach’s alphas and mean loadings/crossloadings Saucier’s (1994, p. 512–513) reported results
Multinational Assessment
First revision
Cronbach’s alpha Conscientiousness Extraversion Intellect/Openness Emotional Stability Agreeableness
.83 .83 .78 .78 .81
.83 .69 .79 .71 .79
.84 .80 .71 .57 .80
Mean scale loading/cross-loading Conscientiousness Extraversion Intellect/Openness Emotional Stability Agreeableness
.66/.06 .67/.07 .57/.09 .60/.07 .62/.08
– – – – –
.61/.15 .62/.09 .55/.14 .48/.12 .60/.13
Second revision
Final revision All
Native English
Non-native English
.88 .87 .78 .67 .78
.87 .87 .85 .79 .81
.90 .92 .84 .84 .86
.86 .85 .84 .77 .80
.71/.08 .67/.09 .60/.10 .51/.13 .60/.13
.70/.09 .70/.08 .67/.10 .62/.09 .63/.11
.76/.06 .79/.06 .68/.09 .68/.07 .69/.12
.69/09 .68/.08 .66/.09 .60/.09 .61/.10
Note: Mean scale loadings and cross-loadings are not calculated for the multinational assessment of Saucier’s original items due to the uninterpretable 5-component rotated solution produced.
4.2. Measure To help determine which Mini-Marker items to replace, data from the initial Mini-Marker assessment study were used to determine each sub-scale’s items contribution to internal consistency with a view to eliminating those that (a) contributed least, (b) loaded poorly in the whole scale principal component analysis, and (c) were identified in focus groups as potentially problematic. These criteria suggested the replacement in the Extraversion subscale of bashful, bold, and withdrawn, in the Intellect or Openness subscale of complex, in the Emotional Stability sub-scale of relaxed, temperamental, and touchy, in the Conscientiousness sub-scale of sloppy, and no items in the Agreeableness sub-scale. Potential replacement items were selected from Goldberg’s (1992) original 100 unipolar markers, from which the Mini-Markers is derived, or from the best five-factor solution Saucier (1997) obtained from the 500 most familiar English person descriptors. Potentially suitable replacement items were then rated for ease and consistency of understanding by the same individuals who had participated in the focus groups and then a decision made to use them based also on the need to minimize negations and to retain the overall balance of negatively and positively keyed items. As neither replacement source offered an acceptable replacement for sloppy, a new adjective, untidy, was used. To ameliorate the problem identified in focus groups of the alphabetical item listing leading to confusion, and in an effort to maximize orthogonality, items were ordered non-alphabetically in sub-scale blocks. Final item order was determined by an effort to obtain a placement balance between negatively and positively keyed items and adequate distancing of items consisting of an adjective and its negation (e.g. envious and unenvious). The items, in order, used for each sub-scale were: Extraversion, shy, talkative, energetic, quiet, extraverted, outgoing, reserved, untalkative; Intellect or Openness, creative, intellectual, simple, imaginative, philosophical, deep, uncreative, unintellectual; Emotional Stability, envious, carefree, fretful, jealous, easy-going, moody, emotional, unenvious; Conscientiousness, efficient, disorganized, careless, untidy, practical, inefficient, systematic, organized; Agreeableness, cold, sympathetic, harsh, cooperative, kind, warm, rude, unsympathetic. To increase clarity and brevity, question instructions were altered slightly and shortened (see Appendix). 4.3. Analyses and results Again following Saucier’s (1994) exploratory principal component analysis procedures, a slightly improved, more interpretable
solution than that for the original Mini-Markers was produced, with better orthogonality, as evidenced by a mean inter-scale correlation of .27 for the revised items compared with .31 for the multinational sample assessment of the original Mini-Marker items. The Agreeableness items continued to perform reasonably well (Table 3), loading cleanly on one component, although cold was revealed to detract from internal consistency. The Extraversion subscale also loaded cleanly on one component. Overall, however, three replacement items, carefree, easy-going, and simple prove to have highest loadings of just .33 or lower, and eight items failed to have highest loadings at least double their highest cross-loading. The Conscientiousness sub-scale performed reasonably well except for its item practical that detracted from internal consistency and cross-loaded highly on Intellect or Openness. The Intellect or Openness sub-scale itself performed slightly worse than the original, with simple proving to detract substantially from internal consistency. The replacement items of the Emotional Stability sub-scale performed substantially worse than the originals, with carefree and easy-going detracting substantially from internal consistency. 5. Study four: second scale revision 5.1. Participants Participants comprised 459 family and friend volunteers of master degree students studying international relations at the same East Asian international university (Table 1) who completed instruments online. 5.2. Measure The three selection criteria used for the first scale revision suggested that all Extraversion sub-scale items and their order be retained. For Agreeableness, the criteria suggested replacement of cold and unsympathetic. Again using Goldberg (1992) and Saucier (1997) as replacement sources, cold was replaced with unkind, and unsympathetic with inconsiderate, and the sub-scale reordered thus: kind, sympathetic, harsh, cooperative, unkind, warm, rude, inconsiderate. The Agreeableness sub-scale was retained, except the item practical which was replaced by neat. The Intellect or Openness items simple and unintellectual were replaced by artistic and unintelligent. Imaginative was then changed to unimaginative to improve negative and positive item balance, and items then reordered: creative, intellectual, unimaginative, artistic, unintelligent, philosophical, deep, uncreative. For Emotional Stability, carefree was
546
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
students at the same East Asian international university (see Table 1 for demographics). Instruments were completed online.
replaced by secure, fretful by anxious, and easy-going by unemotional, and the order changed to: envious, emotional, anxious, secure, jealous, unenvious, moody, unemotional.
6.2. Measure 5.3. Analyses and results Using the three item-evaluation criteria mentioned above, the Extraversion, Conscientiousness, and Agreeableness sub-scales were retained unaltered, but unintelligent in the Intellect or Openness sub-scale was altered to intelligent, and in the Emotional Stability sub-scale, secure was replaced by unworried, and unemotional was replaced by unanxious, once again using Goldberg (1992) and Saucier (1997) as sources (see Appendix for final item order).
Once again following Saucier’s (1994) principal component analysis procedures, an improved, more interpretable solution was produced with a mean inter-scale correlation of .20, indicating improved orthogonality, and adequate scale internal reliability consistencies (Table 3). Each sub-scale’s items loaded most highly on its own component, except secure. Nevertheless, the Intellect or Openness item unintelligent had relatively poor component- and cross-loadings, as did the Emotional Stability items secure and unemotional, with both detracting from consistency.
6.3. Results Using the full sample, Saucier’s (1994) principal component analysis procedure produced a readily interpretable solution (see Table 4) with all items loading most highly on their intended sub-scale. Mean component loadings and cross-loadings were comparable with those Saucier (1994, p. 512–514) reports (Table 3), and there were no cross-loadings above .35, and only 2 out of 160 cross-loadings fell marginally short of Saucier’s stringent
6. Study five: final scale revision 6.1. Participants Participants, 1,927 family and friend volunteers of two undergraduate management classes’ students and a further set of MBA
Table 4 Principal component analysis of the International English Mini-Markers items for full, native English-speaker and non-native English-speaker samplesa,b Items
Conscientiousness Full
Organized Disorganized Systematic Untidy Inefficient Neat Efficient Careless Talkative Untalkative Quiet Outgoing Extroverted Shy Reserved Energetic Creative Uncreative Intellectual Intelligent Philosophical Unimaginative Deep Artistic Anxious Unanxious Jealous Moody Emotional Unworried Envious Unenvious Unkind Kind Sympathetic Rude Warm Inconsiderate Harsh Cooperative a
.82 .80 .74 .70 .64 .64 .63 -.59
Eng. .84 .86 .67 .77 .70 .80 .76 .67
.10
Extraversion Other .82 .79 .75 .68 .64 .59 .61 .60
.11
.19
.13
.19
.25 .25 .11
.18 .18 .10
.26 .27 .12
.16
.13 .10
.17
Full
Eng.
Full .14
Eng. .13 .10
.18 .18
.12
.17 .79 .78 .78 .72 .69 .68 .63 .56 .13 .18 .11 .14 .20
.76 .78 .84 .84 .84 .82 .83 .58
.14 .20
.13 .10 .14
Intellect/ Openness Other
.12 .16
Emotional Stability Other
.18
.20
.25
.15
.24
.80 .79 .76 .70 .64 .64 .57 .56 .14 .20 .10 .13
.10 .10 .10
.17 .80 .75 .71 .70 .61 .71 .63 .55
.19 .11 .11
.10 .11 .21
.15
.11 .14 .14
.15
.23
.16
.25
.25
.15
.27
.12
.22
.10 .10 .14
.30
.12 .19 .17 .20
.17
.14 .28 .25
.17 .16
.12 .12
.19
.11
.15
.14 .17
.19
.17
.12
.13
.14
Eng.
Other
.16
.10 .13
.14 .16 .13
.16 .17 .16
.13 .12
.20
.18
.18
.14
.15
.12
.12 .17 .11
.26 .77 .68 .69 .67 .66 .59 .60 .61
.14 .11
.16
.13
.15 .12
.11
.13
Full
.10
.19
.20
Agreeableness Other
.19
.20
.26 .78 .70 .69 .69 .66 .62 .61 .60
Eng.
.14
.20
.17
Full
.11 .10
.17 .11 .17 .19
.17 .11
.23 .16 .15
.11 .16 .10
.26 .81 .78 .71 .65 .59 .67 .68 .53
.75 .65 .59 .59 .61 .59 .53 .49
.13 .22
.15 .26
.13 .20
.14 .23
.23 .33
.10 .20
.76 .67 .62 .61 .61 .60 .57 .51 .10
.12
.32 .13 .10 .33 .20 .74 .70 .66 .62 .60 .58 .58 .56
.16 .20 .10
.12 .12 .12 .17
.29 .16 .10 .26 .13 .76 .74 .71 .70 .68 .66 .62 .68
.17
.32 .11 .12 .35 .21 .73 .69 .63 .61 .57 .56 .57 .54
Items appear in order of principal component analysis for full sample. Full sample (Full) N = 1,927, native English-speaker (Eng.) sample N = 300, non-native Englishspeaker (Other) sample N = 1,627. b Item loadings exceeding .35 are emboldened, those below .10 are omitted.
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
criterion of no cross-loading more than half an item’s subscale loading. Analyses using a random split of the full sample produced similar results, suggesting cross-sample stability. Separate analyses of participants from native English- and nonnative English-speaking countries produced similarly interpretable solutions (Table 4) and adequate internal consistency reliabilities for each set of countries’ participants (Table 3), indicating an acceptable level of cross-national equivalence. The good orthogonality suggested by high sub-scale loadings and low cross-loadings was supported by relatively low mean inter-scale correlations of .24 for the full sample, and .24 and .13, respectively, for the nonnative English-speaking and native English-speaking countries. 7. Study six: temporal stability and convergent validity 7.1. Temporal stability Three months after the initial test with the final scale revision, a retest was completed by a randomly selected sub-sample of 85 of the initial test participants. Correlations between test and retest data were .83 for Extraversion, .70 for Intellect or Openness, .73 for Emotional Stability, .81 for Conscientiousness, and .79 for Agreeableness, indicating ‘exemplary’ medium-run temporal stability by the criteria suggested by Robinson, Shaver, and Wrightsman (1991, p. 13). As Saucier (1994) does not offer any data on temporal stability, comparison with his original scale is not possible. 7.2. Convergent validity To test convergent validity, Goldberg’s (1992, Appendix B) bipolar five-factor inventory was administered in the same instrument completed by the retest sample above. Correlations were: between Extraversion and Goldberg’s equivalent sub-scale, .69; between Intellect or Openness and its equivalent, .78; between Agreeableness and its equivalent, .70; between Conscientiousness and its equivalent, .69; between Emotional Stability and its equivalent, .68. Hence, convergent validity was found acceptable. 8. Limitations and conclusion 8.1. Limitations While the International English Mini-Markers exhibits across both native and non-native English-speakers stable and interpretable factor structure, good orthogonality, adequate sub-scale internal consistency reliabilities, convergent validity and temporal stability, limitations need to be acknowledged. First, the aggregation of respondents into native and non-native English speakers is broad and potentially masks country-level differences in the psychometric performance of the International English Mini-Markers that might arise from differing prevalences of English usage, a matter that further research will need to examine. Second, although the convenience samples here used might reasonably be regarded as reflecting to some extent the kind of individuals that might be found in many international research settings, they may not be representative of English-speaking but non-native English-speakers more widely. Again, further research will be required to examine the psychometric properties of the International English MiniMarkers both for more broadly based international, English-speaking samples and for specific sub-samples. Third, while testing for cross-cultural structural invariance of relatively complex measures is difficult using structural equation methods, future research might usefully seek to establish more firmly the cross-cultural stability of the International English Mini-Markers using procrustes rotation procedures.
547
8.2. Conclusion The growth of research settings with numerous respondents from diverse and not-necessarily-known-in-advance, non-native English-speaking countries but who, nevertheless, operate in English, together with a relative lack of translations of brief big-five measures, have conspired to make necessary studies that systematically test, develop and validate brief English measures with multinational, English-using populations. Because no other studies appear yet to have systematically tested the psychometric properties of English big-five measures, neither brief nor longer measures, with non-native English-speakers, the research here reported together with the International English Mini-Markers it develops and validates contribute to filling a void in the literature for a psychometrically robust and practical brief big-five measure for English-using but not necessarily native English-speaking research populations. Acknowledgements Two anonymous reviewers and Colin Cooper are thanked for their useful comments and suggestions. Appendix. The International English Mini-Markers Please use the below list of common human traits to describe yourself as accurately as possible. Describe yourself as you really are compared to other people you know of the same age and sex, not as you wish to be. So, generally, is it accurate or inaccurate that you are: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34.
Shy Talkative Energetic Quiet Extraverted Outgoing Reserved Untalkative Creative Intellectual Unimaginative Artistic Intelligent Philosophical Deep Uncreative Envious Emotional Anxious Unworried Jealous Unenvious Moody Unanxious Efficient Disorganized Careless Untidy Neat Inefficient Systematic Organized Kind Sympathetic
548
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
35. 36. 37. 38. 39. 40.
Harsh Cooperative Unkind Warm Rude Inconsiderate
Interval measure: Inaccurate (1) (2) (3) (4) (5) Accurate. References Austin, E. J., Saklofske, D. H., & Egan, V. (2005). Personality, well-being and health correlates of trait emotional intelligence. Personality and Individual Differences, 38(3), 547–558. Cheung, F. M., Leung, K., Zhang, J. X., Sun, H. F., Gan, Y. Q., & Song, W. Z. (2001). Indigenous Chinese personality constructs – Is the five-factor model complete? Journal of Cross-Cultural Psychology, 32(4), 407–433. Church, A. T. (2001). Personality measurement in cross-cultural perspective. Journal of Personality, 69(6), 979–1006. Diefendorff, J. M., & Richard, E. M. (2003). Antecedents and consequences of emotional display rule perceptions. Journal of Applied Psychology, 88(2), 284–294. Erdheim, J., Wang, M., & Zickar, M. J. (2006). Linking the big-five personality constructs to organizational commitment. Personality and Individual Differences, 41(5), 959–970. Eysenck, H. J. (1991). Dimensions of personality – 16, 5 or 3 – criteria for a taxonomic paradigm. Personality and Individual Differences, 12(8), 773–790. Garcia, O., Aluja, A., & Garcia, L. F. (2004). Psychometric properties of Goldberg’s 50 personality markers for the big five model: a study in the Spanish language. European Journal of Psychological Assessment, 20(4), 310–319. Goldberg, L. R. (1990). An alternative description of personality – the big-five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. Goldberg, L. R. (1992). The development of markers for the big-five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the bigfive personality domains. Journal of Research in Personality, 37(6), 504–528. Grant, I., Erksen, H. R., Marquis, P., Orre, I. J., Palinkas, L. A., Suedfeld, P., et al. (2007). Psychological selection of antarctic personnel: the ‘‘SOAP” instrument. Aviation Space and Environmental Medicine, 78(8), 793–800. Hampson, S. E., & Goldberg, L. R. (2006). A first large cohort study of personality trait stability over the 40 years between elementary school and midlife. Journal of Personality and Social Psychology, 91(4), 763–779. Hendriks, A. A. J., Hofstee, W. K. B., & De Raad, B. (1999). The five-factor personality inventory. Personality and Individual Differences, 27(2), 307–325. Hochwalder, J. (2006). A psychometric assessment of a Swedish translation of Shafer’s personality scale. Scandinavian Journal of Psychology, 47(6), 523–530. Langford, P. H. (2003). A one-minute measure of the big-five? Evaluating and abridging Shafer’s (1999a) big-five markers. Personality and Individual Differences, 35(5), 1127–1140.
McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1), 81–90. McCullough, M. E., Emmons, R. A., & Tsang, J. A. (2002). The grateful disposition: a conceptual and empirical topography. Journal of Personality and Social Psychology, 82(1), 112–127. Mlacic, B., & Goldberg, L. R. (2007). An analysis of a cross-cultural personality inventory: the IPIP big-five factor markers in Croatia. Journal of Personality Assessment, 88(2), 168–177. Moon, S. M., & Illingworth, A. J. (2005). Exploring the dynamic nature of procrastination: a latent growth curve analysis of academic procrastination. Personality and Individual Differences, 38(2), 297–309. Mooradian, T. A., & Nezlek, J. B. (1996). Comparing the NEO-FFI and Saucier’s MiniMarkers as measures of the big-five. Personality and Individual Differences, 21(2), 213–215. Muck, P. M., Hell, B., & Gosling, S. D. (2007). Construct validation of a short fivefactor model instrument - A self-peer study on the German adaptation of the Ten-Item Personality Inventory (TIPI-G). European Journal of Psychological Assessment, 23(3), 166–175. Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute or less: a 10-item short version of the big-five Inventory in English and German. Journal of Research in Personality, 41(1), 203–212. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Criteria for Scale Selection and Evaluation. In J. P. Robinson, P. R. Shaverl, & L. S. Wrightsman (Eds.), Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press. Rosenberg, M. (1965). Society and the Adolescent Self-Image. Princeton, NJ: Princeton University Press. Saucier, G. (1994). Mini-Markers – a brief version of Goldberg’s unipolar big-five markers. Journal of Personality Assessment, 63(3), 506–516. Saucier, G. (1997). Effects of variable selection on the factor structure of person descriptors. Journal of Personality and Social Psychology, 73(6), 1296–1312. Schmitt, D. P., Allik, J., McCrae, R. R., & Benet-Martínez, V. (2007). The geographic distribution of big five personality traits: patterns and profiles of human selfdescription across 56 nations. Journal of Cross-Cultural Psychology, 3(38), 173–212. Schoen, H., & Schumann, S. (2007). Personality traits, partisan attitudes, and voting behavior. Evidence from Germany. Political Psychology, 28(4), 471–498. Terracciano, A. (2003). The Italian version of the NEO PI-R: conceptual and empirical support for the use of targeted rotation. Personality and Individual Differences, 35(8), 1859–1872. Thompson, E. R. (2007). Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). Journal of Cross-Cultural Psychology, 38(2), 227–242. Trull, T. J., & Geary, D. C. (1997). Comparison of the big-five factor structure across samples of Chinese and American adults. Journal of Personality Assessment, 69(2), 324–341. Woods, S. A., & Hampson, S. E. (2005). Measuring the big-five with single items using a bipolar response scale. European Journal of Personality, 19(5), 373–390. Zhao, H., & Seibert, S. E. (2006). The big five personality dimensions and entrepreneurial status: a meta-analytical review. Journal of Applied Psychology, 91(2), 259–271.
Contents lists available at ScienceDirect
Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid
Development and Validation of an International English Big-Five Mini-Markers q Edmund R. Thompson * School of Management, University of Bath, Bath BA2 7AY, United Kingdom
a r t i c l e
i n f o
Article history: Received 11 November 2007 Received in revised form 12 May 2008 Accepted 12 June 2008
Keywords: International English Mini-Markers Big-five Scale development and validation Saucier
a b s t r a c t Examination of Saucier (1994) big-five Mini-Markers using a multinational sample (N = 491) found its psychometric properties suboptimal. Using further multinational samples, through a qualitative study (N = 23) appraising items and then a series of quantitative development and validation studies (total N = 3,068), a revised marker set was derived. This new International English Mini-Markers (a) produced better factor structures, higher scale internal consistency reliabilities, and greater orthogonality than the original set of items, (b) prove to have temporal stability, and (c) acceptable convergent validity. Ó 2008 Elsevier Ltd. All rights reserved.
1. Introduction Personality dimensions have long attracted research attention (Eysenck, 1991). The five-factor model of personality (Goldberg 1990; McCrae and Costa; 1987) has met with particularly wide application in the personality field, plus across disciplines as diverse as aviation (Grant et al., 2007), politics (Schoen & Schumann, 2007), and entrepreneurship (Zhao & Seibert 2006). The model’s wide disciplinary application has prompted the development of several brief big-five measures comprising fewer than 50 singleitem adjectives or only a very few statement-based items specifically for use in applied research settings where respondent time or instrument space are constrained (e.g. Gosling, Rentfrow, & Swann, 2003; Langford, 2003; Saucier, 1994; Woods & Hampson, 2005). While, as Church (2001) notes, some debate has existed about the uniform replicability of the five-factor model in emically developed measures across some cultures such as China (Cheung et al., 2001), the model’s general cross-cultural applicability has, nevertheless, stimulated translations of relatively long big-five measures into Chinese (Trull & Geary, 1997), Croatian (Mlacic & Goldberg, 2007), Italian (Terracciano, 2003), Spanish (Garcia, Aluja, & Garcia, 2004), and other languages. Such translations tend to support the broad cross-cultural applicability of the five-factor model. For example, Hendriks, Hofstee, and De Raad (1999) developed interactively in Dutch, English and German a 100-statement big-five
q Support for this research was provided by the Japan Society for the Promotion of Science, Grant No. 16330082. * Tel.: +44 1225 386742; fax: +44 1225 386473. E-mail address: [email protected]
0191-8869/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.paid.2008.06.013
measure that confirmed the five-factor structure in these languages and 11 others into which it was subsequently translated, including Chinese and Japanese. More recently, Schmitt, Allik, McCrae, and Benet-Martínez (2007) found robust support for the five-factor model across world regions in a study covering 56 nations using a 44-statement (173-word) big-five measure. Some brief big-five measures have also been translated, although as yet only into German (Muck, Hell, & Gosling, 2007; Rammstedt & John 2007) and Swedish (Hochwalder, 2006). The relatively low number of languages into which any single brief big-five measure has so far been translated has meant that some forms of applied cross-cultural research demanding brief measures have been constrained. Most particularly, applied research in cross-cultural settings where multiple national backgrounds can (a) be anticipated, are (b) of specific interest, but (c) cannot necessarily be known precisely in advance is currently impossible, except, as Thompson (2007) suggests, using English measures. Such research settings include international government bodies and firms, plus, of course, many educational institutions, where populations increasingly comprise individuals from numerous countries who, despite not always being native English-speakers, are obliged to operate organizationally in English. Because English big-five measures have been emically developed, predominantly among native English-speakers in North America, research is needed to assess their psychometric properties among non-native English-speakers before they can with confidence be used in international research settings. No such research has yet been undertaken. The six studies reported in this paper, first, examine the psychometric performance of Saucier’s (1994) big-five Mini-Markers, in an English-using, multinational sample, then, having found its psychometric properties to be suboptimal, develop and validate an International English Mini-Markers.
543
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
The Mini-Markers, a short-form of Goldberg’s (1992) unipolar lexical big-five measure, is selected for assessment because it has proven to be one of the most psychometrically reliable (Mooradian & Nezlek, 1996) and frequently used brief big-five measures, being employed widely in personality research (Diefendorff & Richard, 2003; McCullough, Emmons, & Tsang, 2002) and in applied settings across several disciplines, including health (Austin, Saklofske, & Egan, 2005), business (Erdheim, Wang, & Zickar 2006), and education (Moon & Illingworth, 2005). Comprising just 40 singleadjective personality descriptors originally selected for their psychometric qualities (Saucier, 1994), the Mini-Markers can be hypothesized to lend itself without translation to use with multinational samples that use English. 2. Study one: Mini-Markers assessment with multinational sample 2.1. Participants Participants comprised 491 family and friends of executive MBA students at an English-based international university in East Asia who had volunteered to assist with research, plus had identified themselves as proficient at English. The sample therefore represents the relatively affluent and educated strata of individuals who might reasonably be encountered in numerous international but Englishspeaking research settings (see Table 1 for demographic details). 2.2. Measure The Mini-Markers was used as Saucier (1994) specifies, except, following Hampson and Goldberg (2006), the original 9-point interval measure anchored on accurate and inaccurate was reduced to 5 points to maximize brevity. This was incorporated into an online instrument addressing career motivations sent individually to participants.
2.3. Analyses and results Following Saucier’s (1994) analytical procedure in his development of the Mini-Markers, a principal component analysis with a varimax rotation specifying five components failed to produce the expected factor structure, with Conscientiousness being the only sub-scale forming a distinct, complete and interpretable component (Table 2). Seven items had loadings below the lowest (.44) reported by Saucier’s (1994, p. 512), and 18 items had loadings failing to meet Saucier’s (1994, p. 509) item-purity criterion of having a highest loading at least double its loading on any other component. Of the 22 items that did meet this criterion, only seven could be said to load on the expected component. Analyses specifying different extraction and rotation procedures failed to produce an improved or acceptable five-factor solution. The mean inter-scale correlation of .31 was considerably higher than Saucier’s (1994, p. 512–514) reported mean inter-scale correlation of .11, indicating considerable lack of orthogonality of sub-scales. However, scale internal consistency reliabilities were adequate, with the exception of Extraversion (Table 3). 2.4. Discussion Given the poor factor structure and high inter-scale correlations, but acceptable internal consistencies for four of the subscales, method variance was suspected. Examination of responses to other measures administered alongside the Mini-Markers, such as Rosenberg’s (1965) self-esteem scale, did not indicate careless or non-diligent responding, suggesting that the suboptimal psychometric performance of the Mini-Markers with a multinational sample was due to an interaction of the nature of the multinational sample and the Mini-Markers. A number of items could be identified as potentially problematic by their low component loadings or high cross-loadings, including energetic, relaxed, touchy, withdrawn, complex, sloppy,
Table 1 Sample characteristics of each study Study 1
Study 3
Study 4
Study 5
491 259
650 346
459 216
1927 704
Age (percent) 18–24 25–29 30 or older
30 33 37
15 48 37
25 36 39
55 25 20
Percent already with undergraduate degree
78
86
83
63
Occupation (percent) Private sector employees Public sector employees Self-employed Postgraduate student Undergraduate student Homemaker, retired or unemployed
33 17 4 17 10 19
49 22 5 11 5 9
34 18 5 19 9 15
26 9 9 8 24 24
408 38
629 31
378 23
1327 68
Burma (54) China (26) Indonesia (27) Mexico (44) Thailand (79) Vietnam (35)
China (50) Indonesia (58) Japan (42) Malaysia (78) Mongolia (39) Philippines (109) Singapore (26) Vietnam (90)
China (22) Indonesia (53) Japan (35) Malaysia (25) Peru (20) Singapore (29) Taiwan (21) Thailand (29) Vietnam (49)
Burma (83) China (884) Hungary (82) India (28) Indonesia (96) Japan (40) Malaysia (37) Philippines (67) Taiwan (32) Thailand (48) Vietnam (32)
N Males
Non-native English-speakers Number of non-native English-speaking countries sample drawn from Non-native English-speaking countries with 20 plus respondents (N)
Note: Countries with fewer than 20 respondents are omitted for space considerations.
544
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
Table 2 Principal component analysis of Saucier’s Mini-Markers with multinational samplea Itemsb A. Kind IO. Imaginative A. Warm IO. Creative A. Sympathetic IO. Deep IO. Intellectual A. Cooperative E. Energetic IO. Philosophical ES. Relaxed A. Harsh ES. Fretful ES. Temperamental A. Rude ES. Envious ES. Jealous ES. Moody A. Cold ES. Touchy E. Withdrawn IO. Complex C. Organized C. Disorganized C. Systematic C. Efficient C. Inefficient C. Sloppy C. Practical C. Careless E. Quiet E. Shy E. Talkative E. Extraverted E. Bashful E. Bold ES. Unenvious A. Unsympathetic IO. Unintellectual IO. Uncreative % of variance explained
Component 1 .69 .64 .62 .62 .61 .60 .58 .55 .52 .48 .38 .21
.29 .12 .10 .27 .20 .29 .14
Component 2
.42 .10 .15 .26 .23 .18 .14 .47 .39 .45 20.37
Component 4
Component 5
.23 .17 .21 .24 .13 .14 .35 .15 .38 .12 .14 .66 .65 .61 .60 .59 .57 .55 .51 .45 .35 .33 .23
.28 .39 .24
Component 3
.32 .44 .33 .12 .30 .10 .26 .12 .30 .26 .19 .16 9.09
.16
.10 .15 .12
.23 .30 .16 .23
.24 .17
.17 .12 .31 .25 .19 .20 .80 .74 .71 .63 .54 .49 .42 .40 .18
.10 .15 .12 .29 .22 6.03
.21 .16 .31 .10 .32
.16 .35 .20 .11 .19
.15 .20 .18 .18
.73 .69 .61 .60 .42 .31 .10 .16 .21 5.75
.23
.25 .12
.17 .15 .20 .71 .59 .57 .48 4.54
N = 491. a Item loadings exceeding .35 are emboldened, those below .10 are omitted. b Letters before item indicate component in Saucier’s (1994) Mini-Markers: E = Extraversion, A = Agreeableness, C = Conscientiousness, ES = Emotional Stability, IO = Intellect or Openness.
Two focus groups were conducted to investigate Mini-Marker items’ clarity, ease of understanding and singularity of meaning. Together, focus groups comprised 13 executive-MBA students and 5 undergraduates, average age 26, half being male, at the same East Asian international university, who came variously from America, Burma, China, Hungary, Indonesia, Mexico, Mongolia, the Philippines, Thailand, Tonga, and Vietnam.
stand rather than multifaceted or multilayered in the sense of having several meritorious qualities indicative of intelligence and intellect, the implicit intention of the Mini-Markers. Relaxed tended to be associated with a transient state of being without external stress or pressure rather than with a permanent internal trait of easygoingness. Together, the lack of familiarity with, or differing comprehensions of, these items could partially account for the method variance and poor factor structure found with the multinational sample. Additionally, nearly all focus group participants remarked on how the alphabetical ordering of adjectives resulted in a slightly confusing block of four items beginning with the un negation (uncreative, unenvious, unintellectual, unsympathetic). Being towards the end of the scale, this block of items may also have increased method variance due to respondent fatigue, as evidenced by the items loading together as the last component to be rotated and by their generally high cross-loadings.
3.2. Results
4. Study three: first scale revision
Certain items were not uniformly familiar to non-native Englishspeakers, either due to relative rarity (bashful, bold, fretful, temperamental, withdrawn) or their colloquial nature (sloppy, touchy). Complex, and relaxed were familiar, but had ambiguous meanings. Complex was taken to mean something like complicated to under-
4.1. Participants
practical, careless, bashful, bold, unsympathetic, unintellectual, and uncreative. However, rather than rely solely on a quantitative approach to identifying and changing problematic items, the MiniMarkers was first assessed qualitatively. 3. Study two: qualitative item appraisal 3.1. Participants
Participants, 650 family and friend volunteers of a different set of MBA students at the same East Asian international university (see Table 1 for demographics), completed instruments online.
545
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548 Table 3 Scale Cronbach’s alphas and mean loadings/crossloadings Saucier’s (1994, p. 512–513) reported results
Multinational Assessment
First revision
Cronbach’s alpha Conscientiousness Extraversion Intellect/Openness Emotional Stability Agreeableness
.83 .83 .78 .78 .81
.83 .69 .79 .71 .79
.84 .80 .71 .57 .80
Mean scale loading/cross-loading Conscientiousness Extraversion Intellect/Openness Emotional Stability Agreeableness
.66/.06 .67/.07 .57/.09 .60/.07 .62/.08
– – – – –
.61/.15 .62/.09 .55/.14 .48/.12 .60/.13
Second revision
Final revision All
Native English
Non-native English
.88 .87 .78 .67 .78
.87 .87 .85 .79 .81
.90 .92 .84 .84 .86
.86 .85 .84 .77 .80
.71/.08 .67/.09 .60/.10 .51/.13 .60/.13
.70/.09 .70/.08 .67/.10 .62/.09 .63/.11
.76/.06 .79/.06 .68/.09 .68/.07 .69/.12
.69/09 .68/.08 .66/.09 .60/.09 .61/.10
Note: Mean scale loadings and cross-loadings are not calculated for the multinational assessment of Saucier’s original items due to the uninterpretable 5-component rotated solution produced.
4.2. Measure To help determine which Mini-Marker items to replace, data from the initial Mini-Marker assessment study were used to determine each sub-scale’s items contribution to internal consistency with a view to eliminating those that (a) contributed least, (b) loaded poorly in the whole scale principal component analysis, and (c) were identified in focus groups as potentially problematic. These criteria suggested the replacement in the Extraversion subscale of bashful, bold, and withdrawn, in the Intellect or Openness subscale of complex, in the Emotional Stability sub-scale of relaxed, temperamental, and touchy, in the Conscientiousness sub-scale of sloppy, and no items in the Agreeableness sub-scale. Potential replacement items were selected from Goldberg’s (1992) original 100 unipolar markers, from which the Mini-Markers is derived, or from the best five-factor solution Saucier (1997) obtained from the 500 most familiar English person descriptors. Potentially suitable replacement items were then rated for ease and consistency of understanding by the same individuals who had participated in the focus groups and then a decision made to use them based also on the need to minimize negations and to retain the overall balance of negatively and positively keyed items. As neither replacement source offered an acceptable replacement for sloppy, a new adjective, untidy, was used. To ameliorate the problem identified in focus groups of the alphabetical item listing leading to confusion, and in an effort to maximize orthogonality, items were ordered non-alphabetically in sub-scale blocks. Final item order was determined by an effort to obtain a placement balance between negatively and positively keyed items and adequate distancing of items consisting of an adjective and its negation (e.g. envious and unenvious). The items, in order, used for each sub-scale were: Extraversion, shy, talkative, energetic, quiet, extraverted, outgoing, reserved, untalkative; Intellect or Openness, creative, intellectual, simple, imaginative, philosophical, deep, uncreative, unintellectual; Emotional Stability, envious, carefree, fretful, jealous, easy-going, moody, emotional, unenvious; Conscientiousness, efficient, disorganized, careless, untidy, practical, inefficient, systematic, organized; Agreeableness, cold, sympathetic, harsh, cooperative, kind, warm, rude, unsympathetic. To increase clarity and brevity, question instructions were altered slightly and shortened (see Appendix). 4.3. Analyses and results Again following Saucier’s (1994) exploratory principal component analysis procedures, a slightly improved, more interpretable
solution than that for the original Mini-Markers was produced, with better orthogonality, as evidenced by a mean inter-scale correlation of .27 for the revised items compared with .31 for the multinational sample assessment of the original Mini-Marker items. The Agreeableness items continued to perform reasonably well (Table 3), loading cleanly on one component, although cold was revealed to detract from internal consistency. The Extraversion subscale also loaded cleanly on one component. Overall, however, three replacement items, carefree, easy-going, and simple prove to have highest loadings of just .33 or lower, and eight items failed to have highest loadings at least double their highest cross-loading. The Conscientiousness sub-scale performed reasonably well except for its item practical that detracted from internal consistency and cross-loaded highly on Intellect or Openness. The Intellect or Openness sub-scale itself performed slightly worse than the original, with simple proving to detract substantially from internal consistency. The replacement items of the Emotional Stability sub-scale performed substantially worse than the originals, with carefree and easy-going detracting substantially from internal consistency. 5. Study four: second scale revision 5.1. Participants Participants comprised 459 family and friend volunteers of master degree students studying international relations at the same East Asian international university (Table 1) who completed instruments online. 5.2. Measure The three selection criteria used for the first scale revision suggested that all Extraversion sub-scale items and their order be retained. For Agreeableness, the criteria suggested replacement of cold and unsympathetic. Again using Goldberg (1992) and Saucier (1997) as replacement sources, cold was replaced with unkind, and unsympathetic with inconsiderate, and the sub-scale reordered thus: kind, sympathetic, harsh, cooperative, unkind, warm, rude, inconsiderate. The Agreeableness sub-scale was retained, except the item practical which was replaced by neat. The Intellect or Openness items simple and unintellectual were replaced by artistic and unintelligent. Imaginative was then changed to unimaginative to improve negative and positive item balance, and items then reordered: creative, intellectual, unimaginative, artistic, unintelligent, philosophical, deep, uncreative. For Emotional Stability, carefree was
546
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
students at the same East Asian international university (see Table 1 for demographics). Instruments were completed online.
replaced by secure, fretful by anxious, and easy-going by unemotional, and the order changed to: envious, emotional, anxious, secure, jealous, unenvious, moody, unemotional.
6.2. Measure 5.3. Analyses and results Using the three item-evaluation criteria mentioned above, the Extraversion, Conscientiousness, and Agreeableness sub-scales were retained unaltered, but unintelligent in the Intellect or Openness sub-scale was altered to intelligent, and in the Emotional Stability sub-scale, secure was replaced by unworried, and unemotional was replaced by unanxious, once again using Goldberg (1992) and Saucier (1997) as sources (see Appendix for final item order).
Once again following Saucier’s (1994) principal component analysis procedures, an improved, more interpretable solution was produced with a mean inter-scale correlation of .20, indicating improved orthogonality, and adequate scale internal reliability consistencies (Table 3). Each sub-scale’s items loaded most highly on its own component, except secure. Nevertheless, the Intellect or Openness item unintelligent had relatively poor component- and cross-loadings, as did the Emotional Stability items secure and unemotional, with both detracting from consistency.
6.3. Results Using the full sample, Saucier’s (1994) principal component analysis procedure produced a readily interpretable solution (see Table 4) with all items loading most highly on their intended sub-scale. Mean component loadings and cross-loadings were comparable with those Saucier (1994, p. 512–514) reports (Table 3), and there were no cross-loadings above .35, and only 2 out of 160 cross-loadings fell marginally short of Saucier’s stringent
6. Study five: final scale revision 6.1. Participants Participants, 1,927 family and friend volunteers of two undergraduate management classes’ students and a further set of MBA
Table 4 Principal component analysis of the International English Mini-Markers items for full, native English-speaker and non-native English-speaker samplesa,b Items
Conscientiousness Full
Organized Disorganized Systematic Untidy Inefficient Neat Efficient Careless Talkative Untalkative Quiet Outgoing Extroverted Shy Reserved Energetic Creative Uncreative Intellectual Intelligent Philosophical Unimaginative Deep Artistic Anxious Unanxious Jealous Moody Emotional Unworried Envious Unenvious Unkind Kind Sympathetic Rude Warm Inconsiderate Harsh Cooperative a
.82 .80 .74 .70 .64 .64 .63 -.59
Eng. .84 .86 .67 .77 .70 .80 .76 .67
.10
Extraversion Other .82 .79 .75 .68 .64 .59 .61 .60
.11
.19
.13
.19
.25 .25 .11
.18 .18 .10
.26 .27 .12
.16
.13 .10
.17
Full
Eng.
Full .14
Eng. .13 .10
.18 .18
.12
.17 .79 .78 .78 .72 .69 .68 .63 .56 .13 .18 .11 .14 .20
.76 .78 .84 .84 .84 .82 .83 .58
.14 .20
.13 .10 .14
Intellect/ Openness Other
.12 .16
Emotional Stability Other
.18
.20
.25
.15
.24
.80 .79 .76 .70 .64 .64 .57 .56 .14 .20 .10 .13
.10 .10 .10
.17 .80 .75 .71 .70 .61 .71 .63 .55
.19 .11 .11
.10 .11 .21
.15
.11 .14 .14
.15
.23
.16
.25
.25
.15
.27
.12
.22
.10 .10 .14
.30
.12 .19 .17 .20
.17
.14 .28 .25
.17 .16
.12 .12
.19
.11
.15
.14 .17
.19
.17
.12
.13
.14
Eng.
Other
.16
.10 .13
.14 .16 .13
.16 .17 .16
.13 .12
.20
.18
.18
.14
.15
.12
.12 .17 .11
.26 .77 .68 .69 .67 .66 .59 .60 .61
.14 .11
.16
.13
.15 .12
.11
.13
Full
.10
.19
.20
Agreeableness Other
.19
.20
.26 .78 .70 .69 .69 .66 .62 .61 .60
Eng.
.14
.20
.17
Full
.11 .10
.17 .11 .17 .19
.17 .11
.23 .16 .15
.11 .16 .10
.26 .81 .78 .71 .65 .59 .67 .68 .53
.75 .65 .59 .59 .61 .59 .53 .49
.13 .22
.15 .26
.13 .20
.14 .23
.23 .33
.10 .20
.76 .67 .62 .61 .61 .60 .57 .51 .10
.12
.32 .13 .10 .33 .20 .74 .70 .66 .62 .60 .58 .58 .56
.16 .20 .10
.12 .12 .12 .17
.29 .16 .10 .26 .13 .76 .74 .71 .70 .68 .66 .62 .68
.17
.32 .11 .12 .35 .21 .73 .69 .63 .61 .57 .56 .57 .54
Items appear in order of principal component analysis for full sample. Full sample (Full) N = 1,927, native English-speaker (Eng.) sample N = 300, non-native Englishspeaker (Other) sample N = 1,627. b Item loadings exceeding .35 are emboldened, those below .10 are omitted.
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
criterion of no cross-loading more than half an item’s subscale loading. Analyses using a random split of the full sample produced similar results, suggesting cross-sample stability. Separate analyses of participants from native English- and nonnative English-speaking countries produced similarly interpretable solutions (Table 4) and adequate internal consistency reliabilities for each set of countries’ participants (Table 3), indicating an acceptable level of cross-national equivalence. The good orthogonality suggested by high sub-scale loadings and low cross-loadings was supported by relatively low mean inter-scale correlations of .24 for the full sample, and .24 and .13, respectively, for the nonnative English-speaking and native English-speaking countries. 7. Study six: temporal stability and convergent validity 7.1. Temporal stability Three months after the initial test with the final scale revision, a retest was completed by a randomly selected sub-sample of 85 of the initial test participants. Correlations between test and retest data were .83 for Extraversion, .70 for Intellect or Openness, .73 for Emotional Stability, .81 for Conscientiousness, and .79 for Agreeableness, indicating ‘exemplary’ medium-run temporal stability by the criteria suggested by Robinson, Shaver, and Wrightsman (1991, p. 13). As Saucier (1994) does not offer any data on temporal stability, comparison with his original scale is not possible. 7.2. Convergent validity To test convergent validity, Goldberg’s (1992, Appendix B) bipolar five-factor inventory was administered in the same instrument completed by the retest sample above. Correlations were: between Extraversion and Goldberg’s equivalent sub-scale, .69; between Intellect or Openness and its equivalent, .78; between Agreeableness and its equivalent, .70; between Conscientiousness and its equivalent, .69; between Emotional Stability and its equivalent, .68. Hence, convergent validity was found acceptable. 8. Limitations and conclusion 8.1. Limitations While the International English Mini-Markers exhibits across both native and non-native English-speakers stable and interpretable factor structure, good orthogonality, adequate sub-scale internal consistency reliabilities, convergent validity and temporal stability, limitations need to be acknowledged. First, the aggregation of respondents into native and non-native English speakers is broad and potentially masks country-level differences in the psychometric performance of the International English Mini-Markers that might arise from differing prevalences of English usage, a matter that further research will need to examine. Second, although the convenience samples here used might reasonably be regarded as reflecting to some extent the kind of individuals that might be found in many international research settings, they may not be representative of English-speaking but non-native English-speakers more widely. Again, further research will be required to examine the psychometric properties of the International English MiniMarkers both for more broadly based international, English-speaking samples and for specific sub-samples. Third, while testing for cross-cultural structural invariance of relatively complex measures is difficult using structural equation methods, future research might usefully seek to establish more firmly the cross-cultural stability of the International English Mini-Markers using procrustes rotation procedures.
547
8.2. Conclusion The growth of research settings with numerous respondents from diverse and not-necessarily-known-in-advance, non-native English-speaking countries but who, nevertheless, operate in English, together with a relative lack of translations of brief big-five measures, have conspired to make necessary studies that systematically test, develop and validate brief English measures with multinational, English-using populations. Because no other studies appear yet to have systematically tested the psychometric properties of English big-five measures, neither brief nor longer measures, with non-native English-speakers, the research here reported together with the International English Mini-Markers it develops and validates contribute to filling a void in the literature for a psychometrically robust and practical brief big-five measure for English-using but not necessarily native English-speaking research populations. Acknowledgements Two anonymous reviewers and Colin Cooper are thanked for their useful comments and suggestions. Appendix. The International English Mini-Markers Please use the below list of common human traits to describe yourself as accurately as possible. Describe yourself as you really are compared to other people you know of the same age and sex, not as you wish to be. So, generally, is it accurate or inaccurate that you are: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34.
Shy Talkative Energetic Quiet Extraverted Outgoing Reserved Untalkative Creative Intellectual Unimaginative Artistic Intelligent Philosophical Deep Uncreative Envious Emotional Anxious Unworried Jealous Unenvious Moody Unanxious Efficient Disorganized Careless Untidy Neat Inefficient Systematic Organized Kind Sympathetic
548
E.R. Thompson / Personality and Individual Differences 45 (2008) 542–548
35. 36. 37. 38. 39. 40.
Harsh Cooperative Unkind Warm Rude Inconsiderate
Interval measure: Inaccurate (1) (2) (3) (4) (5) Accurate. References Austin, E. J., Saklofske, D. H., & Egan, V. (2005). Personality, well-being and health correlates of trait emotional intelligence. Personality and Individual Differences, 38(3), 547–558. Cheung, F. M., Leung, K., Zhang, J. X., Sun, H. F., Gan, Y. Q., & Song, W. Z. (2001). Indigenous Chinese personality constructs – Is the five-factor model complete? Journal of Cross-Cultural Psychology, 32(4), 407–433. Church, A. T. (2001). Personality measurement in cross-cultural perspective. Journal of Personality, 69(6), 979–1006. Diefendorff, J. M., & Richard, E. M. (2003). Antecedents and consequences of emotional display rule perceptions. Journal of Applied Psychology, 88(2), 284–294. Erdheim, J., Wang, M., & Zickar, M. J. (2006). Linking the big-five personality constructs to organizational commitment. Personality and Individual Differences, 41(5), 959–970. Eysenck, H. J. (1991). Dimensions of personality – 16, 5 or 3 – criteria for a taxonomic paradigm. Personality and Individual Differences, 12(8), 773–790. Garcia, O., Aluja, A., & Garcia, L. F. (2004). Psychometric properties of Goldberg’s 50 personality markers for the big five model: a study in the Spanish language. European Journal of Psychological Assessment, 20(4), 310–319. Goldberg, L. R. (1990). An alternative description of personality – the big-five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. Goldberg, L. R. (1992). The development of markers for the big-five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the bigfive personality domains. Journal of Research in Personality, 37(6), 504–528. Grant, I., Erksen, H. R., Marquis, P., Orre, I. J., Palinkas, L. A., Suedfeld, P., et al. (2007). Psychological selection of antarctic personnel: the ‘‘SOAP” instrument. Aviation Space and Environmental Medicine, 78(8), 793–800. Hampson, S. E., & Goldberg, L. R. (2006). A first large cohort study of personality trait stability over the 40 years between elementary school and midlife. Journal of Personality and Social Psychology, 91(4), 763–779. Hendriks, A. A. J., Hofstee, W. K. B., & De Raad, B. (1999). The five-factor personality inventory. Personality and Individual Differences, 27(2), 307–325. Hochwalder, J. (2006). A psychometric assessment of a Swedish translation of Shafer’s personality scale. Scandinavian Journal of Psychology, 47(6), 523–530. Langford, P. H. (2003). A one-minute measure of the big-five? Evaluating and abridging Shafer’s (1999a) big-five markers. Personality and Individual Differences, 35(5), 1127–1140.
McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1), 81–90. McCullough, M. E., Emmons, R. A., & Tsang, J. A. (2002). The grateful disposition: a conceptual and empirical topography. Journal of Personality and Social Psychology, 82(1), 112–127. Mlacic, B., & Goldberg, L. R. (2007). An analysis of a cross-cultural personality inventory: the IPIP big-five factor markers in Croatia. Journal of Personality Assessment, 88(2), 168–177. Moon, S. M., & Illingworth, A. J. (2005). Exploring the dynamic nature of procrastination: a latent growth curve analysis of academic procrastination. Personality and Individual Differences, 38(2), 297–309. Mooradian, T. A., & Nezlek, J. B. (1996). Comparing the NEO-FFI and Saucier’s MiniMarkers as measures of the big-five. Personality and Individual Differences, 21(2), 213–215. Muck, P. M., Hell, B., & Gosling, S. D. (2007). Construct validation of a short fivefactor model instrument - A self-peer study on the German adaptation of the Ten-Item Personality Inventory (TIPI-G). European Journal of Psychological Assessment, 23(3), 166–175. Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute or less: a 10-item short version of the big-five Inventory in English and German. Journal of Research in Personality, 41(1), 203–212. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Criteria for Scale Selection and Evaluation. In J. P. Robinson, P. R. Shaverl, & L. S. Wrightsman (Eds.), Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press. Rosenberg, M. (1965). Society and the Adolescent Self-Image. Princeton, NJ: Princeton University Press. Saucier, G. (1994). Mini-Markers – a brief version of Goldberg’s unipolar big-five markers. Journal of Personality Assessment, 63(3), 506–516. Saucier, G. (1997). Effects of variable selection on the factor structure of person descriptors. Journal of Personality and Social Psychology, 73(6), 1296–1312. Schmitt, D. P., Allik, J., McCrae, R. R., & Benet-Martínez, V. (2007). The geographic distribution of big five personality traits: patterns and profiles of human selfdescription across 56 nations. Journal of Cross-Cultural Psychology, 3(38), 173–212. Schoen, H., & Schumann, S. (2007). Personality traits, partisan attitudes, and voting behavior. Evidence from Germany. Political Psychology, 28(4), 471–498. Terracciano, A. (2003). The Italian version of the NEO PI-R: conceptual and empirical support for the use of targeted rotation. Personality and Individual Differences, 35(8), 1859–1872. Thompson, E. R. (2007). Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). Journal of Cross-Cultural Psychology, 38(2), 227–242. Trull, T. J., & Geary, D. C. (1997). Comparison of the big-five factor structure across samples of Chinese and American adults. Journal of Personality Assessment, 69(2), 324–341. Woods, S. A., & Hampson, S. E. (2005). Measuring the big-five with single items using a bipolar response scale. European Journal of Personality, 19(5), 373–390. Zhao, H., & Seibert, S. E. (2006). The big five personality dimensions and entrepreneurial status: a meta-analytical review. Journal of Applied Psychology, 91(2), 259–271.