- Email: [email protected]
On-line questionnaire completion time and personality test scores
Personality and Individual Differences 54 (2013) 716–720
Contents lists available at SciVerse ScienceDirect
Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid
On-line questionnaire completion time and personality test scores Adrian Furnham a,⇑, Gillian Hyde b, Geoff Trickey b a b
Department of Clinical, Educational, and Health Psychology, University College London, London WC1H 0AP, United Kingdom PCL, Tunbridge-Wells TN4 8AS, United Kingdom
a r t i c l e
i n f o
Article history: Received 16 July 2012 Received in revised form 23 November 2012 Accepted 28 November 2012 Available online 3 January 2013 Keywords: Response time Questionnaires Bright and dark side traits
a b s t r a c t Participants completed two well established questionnaires on line (HPI: Hogan Personality Inventory; and the HDS: Hogan Developmental Survey). Time taken to complete each study was correlated with scale scores from both questionnaires including the occupational scales derived from the HPI. Those who scored higher on Adjustment (Stability), and Prudence (Conscientiousness) but lower on Learning Approach took longer to complete the test. Those who scored higher on Stress Tolerance and Reliability took significantly longer than those with low scores on these measures. With only the exception of Diligent and Dutiful all correlations between Dark Side variables and time taken were negative, particularly Leisurely, Excitable and Imaginative. Regression showed that up to 6% of the time taken variance could be accounted for. Implications for measurement were considered. Ó 2012 Elsevier Ltd. All rights reserved.
1. Introduction Until relatively recently most people completed questionnaires of all types by pen, pencil and paper. However, there has been a shift to online assessment which offers certain economic and computational advantages (Merten & Ruch, 1996; Merten & Siebert, 1997). It can also offer the opportunity to measure how long people take to complete a questionnaire. Completion time has been of interest to pollsters. They are primarily interested in the demographics of people and how the nature of the questions affects response time. Bassili and Fletcher (1991) found that response time to a telephone administered questionnaire was a function of the type of question asked. People responded fastest to simple, then complex factual questions. They took longer to answer social judgement questions and even longer for value conflict questions. Malhotra (2008) examined education differences in completion time to web surveys. He found low education respondents who filled out the questionnaire most quickly were most prone to primacy effects when faced with uni-polar rating scales. Psychological studies on correlates of response speed to questionnaires however preceded electronic administrations. Thus Molto, Segarra, and Avila (1993) divided students into two groups (fasts and slows) by the time they took to complete the Eysenck Personality Scale (EPQ). As predicted the fast group (both males and females analysed separately) scored higher on impulsivity and extraversion. That is, it was argued that the way they filled in the questionnaire reflected their underlying personality traits. ⇑ Corresponding author. E-mail address: [email protected] (A. Furnham). 0191-8869/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.paid.2012.11.030
There is a considerable and important literature on mental speed and processing reviewed by Jensen (2006). As all intelligence tests are timed, speed of response is fundamental as a criterion in that area. However personality researchers have shown much less interest in time taken to complete preference tests. In a series of studies Furnham and colleagues studied correlates of the time taken to complete the 420 item Eysenck Personality Profiler (EPP). Furnham, Forde, and Cotter (1998a) looked at the time taken to complete the Eysenck Personality Profile in over 900 New Zealand based employees. They found associations between time taken and both scores at the domain and facet level. The amount of total time taken to complete the EPP was significantly correlated with Extraversion and Psychoticism: Introverts and those scoring low on Psychoticism took longer. Two primary factors from Extraversion were correlated with time taken – Inhibited and Submissive people took longer. Predictably, the Obsessive primary factor from the Neuroticism superfactor correlated with time taken – the more Obsessive one is, the longer one takes. Five of the seven primary factors from the Psychoticism superfactor were correlated with time taken, one negatively. They showed that the more Careful, Controlled, Responsible, Unadventurous and Practical subjects were, the significantly longer they took to complete the questionnaire. In a second study Furnham, Forde, and Cotter (1998b) found the time taken to complete the EPP test correlated negatively with two brief intelligence tests. Baddeley Reasoning Test r = .35, Wonderlic r = .18; N = 233). Furnham, Forde, and Ferarri (1999) found a significant negative correlation (r = .12) between time taken on the test and a preference for Motivator as opposed to Hygiene factors in a work motivation questionnaire. Finally Furnham, Jackson, Forde, and Cotter (2001) examined the relationship between time taken to complete the EPP and scores on two other personality
717
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
tests. They found time taken correlated positively with the Feelings (r = .15) and Perceiving (r = .13) dimension of the MBTI and with three dimensions of the Learning Styles Questionnaire. It is possible that time taken could be used as a proxy measure for various traits if, and only if, the results were clear and highly significant. Thus if well adjusted or prudent people took significantly more time than others this could be used as additional ‘‘unobtrusive measure’’ of personality, little affected by social desirability. It is therefore the size and direction of the correlation results between time taken and traits that is the focus of this paper. This study has various advantages over previous studies. First, it had a large population (N around 5000) of working adults. Second, we have scores on two tests to check test-taking style. Third, participants were unaware that they were being timed while they took the test. 2. Method
g. The Learning Approach scale reflects the degree to which a person enjoys academic activities and values education as an end in itself. High scorers tend to enjoy education and training. Low scorers are less interested in formal learning and more interested in hands-on learning on the job. The manual reports alphas for all the scales over .80 and test– retest reliability varying between .69 and .87. The measure has six established criterion related scales, called the occupational scales: Service orientation (being attentive, pleasant, and courteous to clients and customers), Stress tolerance (being able to handle stress; low scores are associated with absenteeism and health problems), Reliability (integrity: high scores and organisational delinquency: low scores, Clerical potential (the ability to follow directions, pay attention to details, communicate clearly), Sales potential (energy, social skill, and the ability to solve problems for clients), and Managerial potential (leadership ability, planning and decision making skills) (Furnham, Trickey, & Hyde, 2012).
2.1. Participants In this study there were 4943 participants of which 2827 were male and 2116 female. They ranged in age from 19 to 70 with a mean of 40.59 years (SD = 8.76). Nearly 80% (79.8%) described themselves as white British. All had completed secondary school and over half were graduates. All were in full time work.
3. Materials 1. Hogan Personality Inventory (HPI) (Hogan & Hogan, 1997a) – This 206 item measure has two types of scales: seven personality traits and six criterion scores. The seven personality traits assessed are: adjustment, ambition, sociability, interpersonal sensitivity, prudence, inquisitive, Learning Approach. a. The Adjustment scale reflects the degree to which a person is steady in the face of pressure, or conversely, sensitive and self-critical. High scorers seem confident, resilient, and optimistic. Low scorers seem self-critical, tense, irritable, and negative. b. The Ambition scale evaluates the degree to which a person seems leader like, seeks status, and values achievement. High scorers seem competitive and eager to advance. Low scorers seem unassertive and less interested in advancement. c. The Sociability scale assesses the degree to which a person needs and/or enjoys social interaction. High scorers seem outgoing, colourful and impulsive, and they dislike working by themselves. Low scorers seem reserved and quiet; they avoid calling attention to themselves and do not mind working alone. d. The Interpersonal Sensitivity scale reflects social sensitivity, tact, and perceptiveness. High scorers seem friendly, warm, and popular. Low scorers seem independent, frank, and direct. e. The Prudence scale concerns self-control and conscientiousness. High scorers seem organised, dependable, and thorough; they follow rules and are easy to supervise. Low scorers seem impulsive and flexible. They tend to resist rules and close supervision; however, they may be creative and spontaneous. f. The Inquisitive scale reflects the degree to which a person seems creative, adventurous, and analytical. High scorers seem imaginative, quick-witted and visionary; they may be easily bored and not pay attention to details. Low scorers tend to be practical, focused, and able to concentrate for long periods.
2. Hogan Development Survey (Hogan & Hogan, 1997b) – This survey has 154 items, scored for 11 scales, each grouping 14 items. Respondents are requested to answer ’True’ or ’False’ to the items. The scales are excitable, sceptical, cautious, reserved, leisurely, bold, mischievous, colourful, imaginative, diligent, dutiful. The HDS has been cross-validated with the MMPI personality disorder scales as well as ‘‘normal traits’’ (Furnham & Crump, 2005). The eleven traits are fully described in Table 1. The manual reports internal reliabilities for the scales varying from .50 to .78 with an average of .67. The 3 month test–retest reliability ranged from .58 to .87. Excitable Concerns being overly enthusiastic about people or projects, and then becoming disappointed with them. They seem to lack persistence Sceptical Concerns being socially insightful, but cynical and overly sensitive to criticism. They seem to lack trust Cautious Concerns being overly worried about being criticised. They seem resistant to change and reluctant to take chances Reserved Concerns lacking interest in or awareness of the feelings of others. They seem to be a poor communicator Leisurely Concerns being independent, ignoring others’ requests, and becoming irritable if they persist. They seem stubborn, uncooperative, and a procrastinator (continued on next page)
Table 1 Regression results for the HPI. r
Beta
Social desirability Age Sex Adjustment Ambition Sociability Agreeability Prudence Inquisitiveness Learning Approach
.10** .02 .03 .02 .12** .00 .16***
F(10,4538) = 31.85, p < .001, Adj. R2 = .06. ** p < .01. *** p < .001.
t
05 07 00
2.97** 4.94*** 0.18
09 01 06 02 09 08 21
4.29*** 0.76 3.21*** 1.08 5.08*** 4.73*** 13.16***
718
Bold
Mischievous
Colourful
Imaginative
Diligent
Dutiful
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
Concerns having inflated views of one’s competence and worth. They seem unable to admit to mistakes or learn from experience Concerns being charming, risk-taking, and excitement-seeking. They seem to have trouble maintaining commitments and learning from experience Concerns being dramatic, engaging, and attention-seeking. They seem preoccupied with being noticed and may lack sustained focus Concerns thinking and acting in interesting and even eccentric ways. They seem creative but possibly lacking in judgement Concerns being conscientious, perfectionistic, and hard to please. They tend to disempower staff Concerns being eager to please and reluctant to act independently. They tend to be pleasant and agreeable, but reluctant to support subordinates
3.1. Procedure Participants were tested by a British based psychological consultancy over a 10-year period. Each participant was given personal feedback on their score. They were nearly all employed as middle or senior managers in British companies. They took this test as part of an assessment exercise, run by an external psychological consultancy. Inevitably, this could have affected their results because of issues such as impression management and dissimulation. However, the HDS has a ‘‘social desirability’’ scale which can be used to control for this problem. In this study we controlled for social desirability in the hierarchical regression analysis.
4.1. HPI trait results Tables 1 and 2 shows the correlations between the seven HPI scales and time taken as well as the full correlation matrix. Three were significant but low. They indicated that the more Adjusted and Prudent the person the longer they took. On the other hand those with a high Learning Approach took less time. A hierarchical regression was then run with time taken as a criterion variable and sex, age and social-desirability being entered in the first step and the seven traits in the second. Older people, and those with lower social desirability scores took longer. Five of the seven personality traits showed significant betas. Those higher on Adjustment, Prudence and Inquisitive took longer, while those lower on Sociability and Learning Approach took longer. In all only 6% of the variance was accounted for in the final regression. 4.2. HPI occupational scales Tables 3 and 4 shows the correlational and regressional results following a similar analysis to that of the HPI occupational traits. Two of the six scales showed significant correlations. Those who scored higher on Stress tolerance and Reliability took longer. The hierarchical regression showed older people with higher Stress tolerance and Reliability scores took longer to complete the HPI. In all only 3% of the variance was accounted for. 4.3. HDS trait results Tables 5 and 6 shows the correlational and regressional results following a similar analysis to that of the HPI traits. With only one exception (cautious) all correlations were similar and significant, and, with two exceptions, (Diligent and Dutiful) all were negative. The hierarchical regression showed that older females took longer. It also identified six dark side traits related to time taken: those
4. Results The mean time in seconds to complete the HPI was 1020.42 (SD = 414.06) and the HDS was 915.69 (SD = 370.97). The distribution of the scores were inspected and based on this those over 2 SDs from the mean were excluded. It is possible that people interrupted their administration making the time taken seem very long. This reduced the N by less than 5%. The results were repeated using a cut-off point of 1.5 SDs which reduced the N but did not significantly change the results. Correlations between the time taken for both tests was r = .58 (p < .001) indicating a similarity in style. Older people took longer (r = .10, p < .001) and males were faster than females (r = .04, p < .001) on both tests. Thus, all correlations were partial correlations controlling for age and gender. Further, in the regressions, age, gender and social desirability were placed in the first step before the other variables were examined.
Table 3 Regression results for the HPI occupational scales. r
Beta
Social desirability Age Sex Service orientation Stress tolerance Reliability Clerical potential Sales potential Managerial potential
.07 .09*** .12*** .06 .06 .05
t
03 09 02
1.67 6.33*** 1.46
03 07 09 02 04 03
1.51 2.79** 4.45*** 0.61 2.32 0.92
F(9,45722) = 14.24, p < .001, Adj. R2 = 0.03. p < .05. *** p < .001. **
Table 2 Correlation results for the HPI. Correlations
Time taken (TT) Adjustment (Ad) Ambition (Am) Sociability (S) Agreeability (Ag) Prudence (P) Inquisitiveness (I) Learning Approach (LA)
Mean
SD
1020.42 27.80 24.97 14.65 18.90 19.71 14.73 9.32
414.07 6.26 4.26 4.47 2.48 4.15 4.43 2.97
TT .10 .02 .03 .02 .12 .00 .16
Ad
Am
.50 .08 .40 .38 .14 .19
.37 .22 .10 .25 .25
S
Ag
.27 .22 .35 .13
.28 .09 .08
P
I
.11 .02
.29
719
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
Table 4 Correlation results for the HPI occupational scales. Correlations
Time taken (TT) Service orientation (SO) Stress tolerance (ST) Reliability (R) Clerical Potential (CP) Sales potential (SP) Managerial potential (MP)
Mean
SD
1020.42 10.4 20.70 11.38 19.53 44.73 31.12
414.07 2.39 3.92 3.24 3.20 8.35 4.06
TT .07 .09 .12 .06 .06 .05
ST
.43 .41 .38 .18 .27
.36 .76 .25 .65
R
.37 .23 .38
CP
SP
.37 .81
.31
5. Discussion
Table 5 Regression results for the HDS. r Social desirability Age Sex Excitable Sceptical Cautious Reserved Leisurely Bold Mischievous Colourful Imaginative Diligent Dutiful
SO
Beta
t
02 05 07 05 02 05 07 08 02 03 04 05 06 06
.07** .05** .02 .06** .08** .05** .06** .05** .07** .06** .06**
0.82 3.10** 4.56*** 2.79** 1.03 2.52** 4.05*** 4.53*** 0.98 1.53 1.79 2.81** 3.80*** 0.00
F(14,4510) = 10.09, p < .001, Adj. R2 = .03. p < .01. *** p < .001. **
who were high on Diligent but low on Excitable, Cautious, Reserved, Leisurely, and Imaginative took longer. Again, 3% of the variance was accounted for. Because participants completed both measures a total time for both tests was computed. A hierarchical regression was then computed with step 1 including sex, age and social desirability; step 2, the seven HPI factors and step 3 the eleven HDS factors. The regression was significant (F(21,4431) = 23.35, p < .001; Adj2 = .10. Seven betas were significant at p < .001. Older people who were higher on Adjusted, Inquisitive, and Diligent but low on Learning Approach, Reserved and Leisurely took longer. By far the largest beta ( .26, t = 15.79) was for Learning Approach.
There are three important issues relating to the results of this study. The first concerns which factors (bright and dark side traits, and occupational scales) relate to test time taken. The correlational and regression results showed that around half of the bright side, nearly all of the dark side and a third of the occupational scales were very modestly correlated with test time taken. The results were straightforward to interpret. Thus those with low Learning Approach scores did the tests very slowly which indicates low curiosity and learning anything from the exercise. This could be due to reading ability: a facet of this dimension is concerned with reading enjoyment and experience. Adjusted people took longer possibly because they were not rendered tense, irritable and negative by doing the test. Possibly low Adjusted people are quicker because they are actually more candid (so answer straight away without deliberation) and more inclined to give a negative impression. Very low Adjustment scorers can sometimes trip the Validity scale on the HPI because these people are too honest and so creating a negative impression of themselves that most other people would not admit to. Thus, it is possible that higher Adjusted people are managing their impression more and therefore taking longer to answer the questions. Similarly Prudent people took longer no doubt because they wanted to do the test accurately and thoroughly not missing out items or responding impulsively and incorrectly. The regression showed also that inquisitiveness was a positive predictor but this accounted for very little of the variance. Only two of the Occupational Scales related to time taken. Those with better Stress Tolerance and those who were more Reliable took longer, especially the latter. Interestingly Sales Potential narrowly missed being significant indicating that those with most potential were very fast at this task which confirms the stereotype of the easily bored, form-filling phobia of successful sales staff.
Table 6 Correlation Results for the HDS. Correlations
Time taken (TT) Excitable (E) Sceptical (Sc) Cautious (Ca) Reserved (Re) Leisurely (L) Bold (B) Mischievous (M) Colourful (Co) Imaginative (Im) Diligent (Di) Dutiful (Du)
Mean
SD
915.69 2.93 4.43 3.42 4.05 5.07 7.15 6.73 7.75 5.47 9.18 7.43
370.80 2.65 2.38 2.68 2.09 2.27 2.72 2.50 2.97 2.38 2.56 2.42
TT .07 .05 .02 .06 .08 .05 .06 .05 .07 .06 .06
E
Sc
.39 .50 .33 .27 .05 .00 .10 .10 .04 .10
.19 .24 .33 .28 .26 .05 .22 .14 .02
Ca
.31 .31 .25 .27 .39 .10 .06 .33
Re
.23 .05 .04 .25 .02 .01 .11
L
B
.16 .11 .02 .13 .12 .11
M
.44 .48 .35 .12 .14
.47 .42 .11 .17
Co
.39 .18 .15
Im
.07 .09
Di
.20
720
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
The results of the dark side analysis were illuminating. Apart from the Dutiful and Diligence, all correlations were negative indicating that the more dark side a person had (especially Leisurely) the less time they took to complete the test, possibly indicating a more careless, less concerned, attitude to test taking. This reinforces the fundamental ideas behind the dark side measures. The second issue relates to the size of the correlations and the amount of variance accounted for. Overall, none of the correlations were r > 0.20, and at best the measures only accounted for 6% of the variance (see Table 1). Thus although time taken is clearly related to bright and dark side traits the correlations are probably too low for time taken to be used as a reliable indicator of any traits even Learning Approach. On the other hand if a person scored over or under two standard deviations it may prove interesting to investigate further. Finally the correlation between the time taken to do the two tests was high (r = 0.58). This provides some indication of a test taking style which is clearly related to age in all three regressions. This may reflect a general slowing down with age as well as less familiarity with doing tests online compared to young people. The issue then is which factors, like intelligence or motivation best predict on-line questionnaire time taken. Inspection and reaction time are well known, sensitive and highly discriminatory measures in differential psychology. Yet what precisely on-line questionnaire completion time measures remains to be explored. It would be desirable in further studies to replicate these results with other types of questionnaires such as those measuring values to see if the individual difference variables had a similar effect. Equally, given the possible role of reading ability on the speed of response it would be desirable to have a measure of general cognitive ability to see the role of intelligence in taking preference tests. A central question for the researcher and practitioner is of what practical, decision-oriented values are such low correlations found in these studies. There is no way that time taken could be reliably used as an index of, or proxy for, personality. The highest correlation in this study (namely r = .16, see Table 1) means that
the predictive accuracy of time-to-complete remains a relatively unreliable measure of any trait measured in this study. However, these findings should not result in researchers ceasing to explore the meaning of on-line completion time of preference tests. Acknowledgment The authors are most grateful to the editor for his very thoughtful response to earlier drafts particularly the cautioning of overinterpreting the findings. References Bassili, J., & Fletcher, J. (1991). Response-time measurement in survey research. Public Opinion Quarterly, 55, 331–346. Furnham, A., & Crump, J. (2005). Personality traits, types and disorders. European Journal of Personality, 19, 167–184. Furnham, A., Forde, L., & Cotter, T. (1998a). Personality scores and test taking style. Personality and Individual Differences, 24, 19–23. Furnham, A., Forde, L., & Cotter, T. (1998b). Personality and intelligence. Personality and Individual Differences, 24, 187–192. Furnham, A., Forde, L., & Ferarri, K. (1999). Personality and work motivation. Personality and Individual Differences, 26, 1035–1043. Furnham, A., Jackson, C., Forde, L., & Cotter, T. (2001). Correlates of the Eysenck personality profiler. Personality and Individual Differences, 30, 587–594. Furnham, A., Trickey, G., & Hyde, G. (2012). Bright aspects to dark side traits. Personality and Individual Differences, 52, 908–913. Hogan, R., & Hogan, J. (1997a). Hogan personality inventory manual. Tulsa, OK: Hogan Assessment Systems. Hogan, R., & Hogan, J. (1997b). Hogan developmental survey manual. Tulsa, OK: Hogan Assessment Systems. Jensen, A. (2006). Clocking the mind. Amsterdam: Elsevier. Malhotra, N. (2008). Completion time and response effects in web surveys. Public Opinion Quarterly, 72, 914–934. Merten, T., & Ruch, W. (1996). A comparison of computerised and conventional administration of the German version of the Eysenck Personality Questionnaire and the Carnell Rating Scale for Depression. Personality and Individual Differences, 20, 281–291. Merten, T., & Siebert, K. (1997). A comparison of computerised and conventional administration of the EPQ-R and the CRS. Personality and Individual Differences, 22, 283–286. Molto, J., Segarra, P., & Avila, C. (1993). Impulsivity and total response speed to a personality questionnaire. Personality and Individual Differences, 15, 97–98.
Contents lists available at SciVerse ScienceDirect
Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid
On-line questionnaire completion time and personality test scores Adrian Furnham a,⇑, Gillian Hyde b, Geoff Trickey b a b
Department of Clinical, Educational, and Health Psychology, University College London, London WC1H 0AP, United Kingdom PCL, Tunbridge-Wells TN4 8AS, United Kingdom
a r t i c l e
i n f o
Article history: Received 16 July 2012 Received in revised form 23 November 2012 Accepted 28 November 2012 Available online 3 January 2013 Keywords: Response time Questionnaires Bright and dark side traits
a b s t r a c t Participants completed two well established questionnaires on line (HPI: Hogan Personality Inventory; and the HDS: Hogan Developmental Survey). Time taken to complete each study was correlated with scale scores from both questionnaires including the occupational scales derived from the HPI. Those who scored higher on Adjustment (Stability), and Prudence (Conscientiousness) but lower on Learning Approach took longer to complete the test. Those who scored higher on Stress Tolerance and Reliability took significantly longer than those with low scores on these measures. With only the exception of Diligent and Dutiful all correlations between Dark Side variables and time taken were negative, particularly Leisurely, Excitable and Imaginative. Regression showed that up to 6% of the time taken variance could be accounted for. Implications for measurement were considered. Ó 2012 Elsevier Ltd. All rights reserved.
1. Introduction Until relatively recently most people completed questionnaires of all types by pen, pencil and paper. However, there has been a shift to online assessment which offers certain economic and computational advantages (Merten & Ruch, 1996; Merten & Siebert, 1997). It can also offer the opportunity to measure how long people take to complete a questionnaire. Completion time has been of interest to pollsters. They are primarily interested in the demographics of people and how the nature of the questions affects response time. Bassili and Fletcher (1991) found that response time to a telephone administered questionnaire was a function of the type of question asked. People responded fastest to simple, then complex factual questions. They took longer to answer social judgement questions and even longer for value conflict questions. Malhotra (2008) examined education differences in completion time to web surveys. He found low education respondents who filled out the questionnaire most quickly were most prone to primacy effects when faced with uni-polar rating scales. Psychological studies on correlates of response speed to questionnaires however preceded electronic administrations. Thus Molto, Segarra, and Avila (1993) divided students into two groups (fasts and slows) by the time they took to complete the Eysenck Personality Scale (EPQ). As predicted the fast group (both males and females analysed separately) scored higher on impulsivity and extraversion. That is, it was argued that the way they filled in the questionnaire reflected their underlying personality traits. ⇑ Corresponding author. E-mail address: [email protected] (A. Furnham). 0191-8869/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.paid.2012.11.030
There is a considerable and important literature on mental speed and processing reviewed by Jensen (2006). As all intelligence tests are timed, speed of response is fundamental as a criterion in that area. However personality researchers have shown much less interest in time taken to complete preference tests. In a series of studies Furnham and colleagues studied correlates of the time taken to complete the 420 item Eysenck Personality Profiler (EPP). Furnham, Forde, and Cotter (1998a) looked at the time taken to complete the Eysenck Personality Profile in over 900 New Zealand based employees. They found associations between time taken and both scores at the domain and facet level. The amount of total time taken to complete the EPP was significantly correlated with Extraversion and Psychoticism: Introverts and those scoring low on Psychoticism took longer. Two primary factors from Extraversion were correlated with time taken – Inhibited and Submissive people took longer. Predictably, the Obsessive primary factor from the Neuroticism superfactor correlated with time taken – the more Obsessive one is, the longer one takes. Five of the seven primary factors from the Psychoticism superfactor were correlated with time taken, one negatively. They showed that the more Careful, Controlled, Responsible, Unadventurous and Practical subjects were, the significantly longer they took to complete the questionnaire. In a second study Furnham, Forde, and Cotter (1998b) found the time taken to complete the EPP test correlated negatively with two brief intelligence tests. Baddeley Reasoning Test r = .35, Wonderlic r = .18; N = 233). Furnham, Forde, and Ferarri (1999) found a significant negative correlation (r = .12) between time taken on the test and a preference for Motivator as opposed to Hygiene factors in a work motivation questionnaire. Finally Furnham, Jackson, Forde, and Cotter (2001) examined the relationship between time taken to complete the EPP and scores on two other personality
717
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
tests. They found time taken correlated positively with the Feelings (r = .15) and Perceiving (r = .13) dimension of the MBTI and with three dimensions of the Learning Styles Questionnaire. It is possible that time taken could be used as a proxy measure for various traits if, and only if, the results were clear and highly significant. Thus if well adjusted or prudent people took significantly more time than others this could be used as additional ‘‘unobtrusive measure’’ of personality, little affected by social desirability. It is therefore the size and direction of the correlation results between time taken and traits that is the focus of this paper. This study has various advantages over previous studies. First, it had a large population (N around 5000) of working adults. Second, we have scores on two tests to check test-taking style. Third, participants were unaware that they were being timed while they took the test. 2. Method
g. The Learning Approach scale reflects the degree to which a person enjoys academic activities and values education as an end in itself. High scorers tend to enjoy education and training. Low scorers are less interested in formal learning and more interested in hands-on learning on the job. The manual reports alphas for all the scales over .80 and test– retest reliability varying between .69 and .87. The measure has six established criterion related scales, called the occupational scales: Service orientation (being attentive, pleasant, and courteous to clients and customers), Stress tolerance (being able to handle stress; low scores are associated with absenteeism and health problems), Reliability (integrity: high scores and organisational delinquency: low scores, Clerical potential (the ability to follow directions, pay attention to details, communicate clearly), Sales potential (energy, social skill, and the ability to solve problems for clients), and Managerial potential (leadership ability, planning and decision making skills) (Furnham, Trickey, & Hyde, 2012).
2.1. Participants In this study there were 4943 participants of which 2827 were male and 2116 female. They ranged in age from 19 to 70 with a mean of 40.59 years (SD = 8.76). Nearly 80% (79.8%) described themselves as white British. All had completed secondary school and over half were graduates. All were in full time work.
3. Materials 1. Hogan Personality Inventory (HPI) (Hogan & Hogan, 1997a) – This 206 item measure has two types of scales: seven personality traits and six criterion scores. The seven personality traits assessed are: adjustment, ambition, sociability, interpersonal sensitivity, prudence, inquisitive, Learning Approach. a. The Adjustment scale reflects the degree to which a person is steady in the face of pressure, or conversely, sensitive and self-critical. High scorers seem confident, resilient, and optimistic. Low scorers seem self-critical, tense, irritable, and negative. b. The Ambition scale evaluates the degree to which a person seems leader like, seeks status, and values achievement. High scorers seem competitive and eager to advance. Low scorers seem unassertive and less interested in advancement. c. The Sociability scale assesses the degree to which a person needs and/or enjoys social interaction. High scorers seem outgoing, colourful and impulsive, and they dislike working by themselves. Low scorers seem reserved and quiet; they avoid calling attention to themselves and do not mind working alone. d. The Interpersonal Sensitivity scale reflects social sensitivity, tact, and perceptiveness. High scorers seem friendly, warm, and popular. Low scorers seem independent, frank, and direct. e. The Prudence scale concerns self-control and conscientiousness. High scorers seem organised, dependable, and thorough; they follow rules and are easy to supervise. Low scorers seem impulsive and flexible. They tend to resist rules and close supervision; however, they may be creative and spontaneous. f. The Inquisitive scale reflects the degree to which a person seems creative, adventurous, and analytical. High scorers seem imaginative, quick-witted and visionary; they may be easily bored and not pay attention to details. Low scorers tend to be practical, focused, and able to concentrate for long periods.
2. Hogan Development Survey (Hogan & Hogan, 1997b) – This survey has 154 items, scored for 11 scales, each grouping 14 items. Respondents are requested to answer ’True’ or ’False’ to the items. The scales are excitable, sceptical, cautious, reserved, leisurely, bold, mischievous, colourful, imaginative, diligent, dutiful. The HDS has been cross-validated with the MMPI personality disorder scales as well as ‘‘normal traits’’ (Furnham & Crump, 2005). The eleven traits are fully described in Table 1. The manual reports internal reliabilities for the scales varying from .50 to .78 with an average of .67. The 3 month test–retest reliability ranged from .58 to .87. Excitable Concerns being overly enthusiastic about people or projects, and then becoming disappointed with them. They seem to lack persistence Sceptical Concerns being socially insightful, but cynical and overly sensitive to criticism. They seem to lack trust Cautious Concerns being overly worried about being criticised. They seem resistant to change and reluctant to take chances Reserved Concerns lacking interest in or awareness of the feelings of others. They seem to be a poor communicator Leisurely Concerns being independent, ignoring others’ requests, and becoming irritable if they persist. They seem stubborn, uncooperative, and a procrastinator (continued on next page)
Table 1 Regression results for the HPI. r
Beta
Social desirability Age Sex Adjustment Ambition Sociability Agreeability Prudence Inquisitiveness Learning Approach
.10** .02 .03 .02 .12** .00 .16***
F(10,4538) = 31.85, p < .001, Adj. R2 = .06. ** p < .01. *** p < .001.
t
05 07 00
2.97** 4.94*** 0.18
09 01 06 02 09 08 21
4.29*** 0.76 3.21*** 1.08 5.08*** 4.73*** 13.16***
718
Bold
Mischievous
Colourful
Imaginative
Diligent
Dutiful
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
Concerns having inflated views of one’s competence and worth. They seem unable to admit to mistakes or learn from experience Concerns being charming, risk-taking, and excitement-seeking. They seem to have trouble maintaining commitments and learning from experience Concerns being dramatic, engaging, and attention-seeking. They seem preoccupied with being noticed and may lack sustained focus Concerns thinking and acting in interesting and even eccentric ways. They seem creative but possibly lacking in judgement Concerns being conscientious, perfectionistic, and hard to please. They tend to disempower staff Concerns being eager to please and reluctant to act independently. They tend to be pleasant and agreeable, but reluctant to support subordinates
3.1. Procedure Participants were tested by a British based psychological consultancy over a 10-year period. Each participant was given personal feedback on their score. They were nearly all employed as middle or senior managers in British companies. They took this test as part of an assessment exercise, run by an external psychological consultancy. Inevitably, this could have affected their results because of issues such as impression management and dissimulation. However, the HDS has a ‘‘social desirability’’ scale which can be used to control for this problem. In this study we controlled for social desirability in the hierarchical regression analysis.
4.1. HPI trait results Tables 1 and 2 shows the correlations between the seven HPI scales and time taken as well as the full correlation matrix. Three were significant but low. They indicated that the more Adjusted and Prudent the person the longer they took. On the other hand those with a high Learning Approach took less time. A hierarchical regression was then run with time taken as a criterion variable and sex, age and social-desirability being entered in the first step and the seven traits in the second. Older people, and those with lower social desirability scores took longer. Five of the seven personality traits showed significant betas. Those higher on Adjustment, Prudence and Inquisitive took longer, while those lower on Sociability and Learning Approach took longer. In all only 6% of the variance was accounted for in the final regression. 4.2. HPI occupational scales Tables 3 and 4 shows the correlational and regressional results following a similar analysis to that of the HPI occupational traits. Two of the six scales showed significant correlations. Those who scored higher on Stress tolerance and Reliability took longer. The hierarchical regression showed older people with higher Stress tolerance and Reliability scores took longer to complete the HPI. In all only 3% of the variance was accounted for. 4.3. HDS trait results Tables 5 and 6 shows the correlational and regressional results following a similar analysis to that of the HPI traits. With only one exception (cautious) all correlations were similar and significant, and, with two exceptions, (Diligent and Dutiful) all were negative. The hierarchical regression showed that older females took longer. It also identified six dark side traits related to time taken: those
4. Results The mean time in seconds to complete the HPI was 1020.42 (SD = 414.06) and the HDS was 915.69 (SD = 370.97). The distribution of the scores were inspected and based on this those over 2 SDs from the mean were excluded. It is possible that people interrupted their administration making the time taken seem very long. This reduced the N by less than 5%. The results were repeated using a cut-off point of 1.5 SDs which reduced the N but did not significantly change the results. Correlations between the time taken for both tests was r = .58 (p < .001) indicating a similarity in style. Older people took longer (r = .10, p < .001) and males were faster than females (r = .04, p < .001) on both tests. Thus, all correlations were partial correlations controlling for age and gender. Further, in the regressions, age, gender and social desirability were placed in the first step before the other variables were examined.
Table 3 Regression results for the HPI occupational scales. r
Beta
Social desirability Age Sex Service orientation Stress tolerance Reliability Clerical potential Sales potential Managerial potential
.07 .09*** .12*** .06 .06 .05
t
03 09 02
1.67 6.33*** 1.46
03 07 09 02 04 03
1.51 2.79** 4.45*** 0.61 2.32 0.92
F(9,45722) = 14.24, p < .001, Adj. R2 = 0.03. p < .05. *** p < .001. **
Table 2 Correlation results for the HPI. Correlations
Time taken (TT) Adjustment (Ad) Ambition (Am) Sociability (S) Agreeability (Ag) Prudence (P) Inquisitiveness (I) Learning Approach (LA)
Mean
SD
1020.42 27.80 24.97 14.65 18.90 19.71 14.73 9.32
414.07 6.26 4.26 4.47 2.48 4.15 4.43 2.97
TT .10 .02 .03 .02 .12 .00 .16
Ad
Am
.50 .08 .40 .38 .14 .19
.37 .22 .10 .25 .25
S
Ag
.27 .22 .35 .13
.28 .09 .08
P
I
.11 .02
.29
719
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
Table 4 Correlation results for the HPI occupational scales. Correlations
Time taken (TT) Service orientation (SO) Stress tolerance (ST) Reliability (R) Clerical Potential (CP) Sales potential (SP) Managerial potential (MP)
Mean
SD
1020.42 10.4 20.70 11.38 19.53 44.73 31.12
414.07 2.39 3.92 3.24 3.20 8.35 4.06
TT .07 .09 .12 .06 .06 .05
ST
.43 .41 .38 .18 .27
.36 .76 .25 .65
R
.37 .23 .38
CP
SP
.37 .81
.31
5. Discussion
Table 5 Regression results for the HDS. r Social desirability Age Sex Excitable Sceptical Cautious Reserved Leisurely Bold Mischievous Colourful Imaginative Diligent Dutiful
SO
Beta
t
02 05 07 05 02 05 07 08 02 03 04 05 06 06
.07** .05** .02 .06** .08** .05** .06** .05** .07** .06** .06**
0.82 3.10** 4.56*** 2.79** 1.03 2.52** 4.05*** 4.53*** 0.98 1.53 1.79 2.81** 3.80*** 0.00
F(14,4510) = 10.09, p < .001, Adj. R2 = .03. p < .01. *** p < .001. **
who were high on Diligent but low on Excitable, Cautious, Reserved, Leisurely, and Imaginative took longer. Again, 3% of the variance was accounted for. Because participants completed both measures a total time for both tests was computed. A hierarchical regression was then computed with step 1 including sex, age and social desirability; step 2, the seven HPI factors and step 3 the eleven HDS factors. The regression was significant (F(21,4431) = 23.35, p < .001; Adj2 = .10. Seven betas were significant at p < .001. Older people who were higher on Adjusted, Inquisitive, and Diligent but low on Learning Approach, Reserved and Leisurely took longer. By far the largest beta ( .26, t = 15.79) was for Learning Approach.
There are three important issues relating to the results of this study. The first concerns which factors (bright and dark side traits, and occupational scales) relate to test time taken. The correlational and regression results showed that around half of the bright side, nearly all of the dark side and a third of the occupational scales were very modestly correlated with test time taken. The results were straightforward to interpret. Thus those with low Learning Approach scores did the tests very slowly which indicates low curiosity and learning anything from the exercise. This could be due to reading ability: a facet of this dimension is concerned with reading enjoyment and experience. Adjusted people took longer possibly because they were not rendered tense, irritable and negative by doing the test. Possibly low Adjusted people are quicker because they are actually more candid (so answer straight away without deliberation) and more inclined to give a negative impression. Very low Adjustment scorers can sometimes trip the Validity scale on the HPI because these people are too honest and so creating a negative impression of themselves that most other people would not admit to. Thus, it is possible that higher Adjusted people are managing their impression more and therefore taking longer to answer the questions. Similarly Prudent people took longer no doubt because they wanted to do the test accurately and thoroughly not missing out items or responding impulsively and incorrectly. The regression showed also that inquisitiveness was a positive predictor but this accounted for very little of the variance. Only two of the Occupational Scales related to time taken. Those with better Stress Tolerance and those who were more Reliable took longer, especially the latter. Interestingly Sales Potential narrowly missed being significant indicating that those with most potential were very fast at this task which confirms the stereotype of the easily bored, form-filling phobia of successful sales staff.
Table 6 Correlation Results for the HDS. Correlations
Time taken (TT) Excitable (E) Sceptical (Sc) Cautious (Ca) Reserved (Re) Leisurely (L) Bold (B) Mischievous (M) Colourful (Co) Imaginative (Im) Diligent (Di) Dutiful (Du)
Mean
SD
915.69 2.93 4.43 3.42 4.05 5.07 7.15 6.73 7.75 5.47 9.18 7.43
370.80 2.65 2.38 2.68 2.09 2.27 2.72 2.50 2.97 2.38 2.56 2.42
TT .07 .05 .02 .06 .08 .05 .06 .05 .07 .06 .06
E
Sc
.39 .50 .33 .27 .05 .00 .10 .10 .04 .10
.19 .24 .33 .28 .26 .05 .22 .14 .02
Ca
.31 .31 .25 .27 .39 .10 .06 .33
Re
.23 .05 .04 .25 .02 .01 .11
L
B
.16 .11 .02 .13 .12 .11
M
.44 .48 .35 .12 .14
.47 .42 .11 .17
Co
.39 .18 .15
Im
.07 .09
Di
.20
720
A. Furnham et al. / Personality and Individual Differences 54 (2013) 716–720
The results of the dark side analysis were illuminating. Apart from the Dutiful and Diligence, all correlations were negative indicating that the more dark side a person had (especially Leisurely) the less time they took to complete the test, possibly indicating a more careless, less concerned, attitude to test taking. This reinforces the fundamental ideas behind the dark side measures. The second issue relates to the size of the correlations and the amount of variance accounted for. Overall, none of the correlations were r > 0.20, and at best the measures only accounted for 6% of the variance (see Table 1). Thus although time taken is clearly related to bright and dark side traits the correlations are probably too low for time taken to be used as a reliable indicator of any traits even Learning Approach. On the other hand if a person scored over or under two standard deviations it may prove interesting to investigate further. Finally the correlation between the time taken to do the two tests was high (r = 0.58). This provides some indication of a test taking style which is clearly related to age in all three regressions. This may reflect a general slowing down with age as well as less familiarity with doing tests online compared to young people. The issue then is which factors, like intelligence or motivation best predict on-line questionnaire time taken. Inspection and reaction time are well known, sensitive and highly discriminatory measures in differential psychology. Yet what precisely on-line questionnaire completion time measures remains to be explored. It would be desirable in further studies to replicate these results with other types of questionnaires such as those measuring values to see if the individual difference variables had a similar effect. Equally, given the possible role of reading ability on the speed of response it would be desirable to have a measure of general cognitive ability to see the role of intelligence in taking preference tests. A central question for the researcher and practitioner is of what practical, decision-oriented values are such low correlations found in these studies. There is no way that time taken could be reliably used as an index of, or proxy for, personality. The highest correlation in this study (namely r = .16, see Table 1) means that
the predictive accuracy of time-to-complete remains a relatively unreliable measure of any trait measured in this study. However, these findings should not result in researchers ceasing to explore the meaning of on-line completion time of preference tests. Acknowledgment The authors are most grateful to the editor for his very thoughtful response to earlier drafts particularly the cautioning of overinterpreting the findings. References Bassili, J., & Fletcher, J. (1991). Response-time measurement in survey research. Public Opinion Quarterly, 55, 331–346. Furnham, A., & Crump, J. (2005). Personality traits, types and disorders. European Journal of Personality, 19, 167–184. Furnham, A., Forde, L., & Cotter, T. (1998a). Personality scores and test taking style. Personality and Individual Differences, 24, 19–23. Furnham, A., Forde, L., & Cotter, T. (1998b). Personality and intelligence. Personality and Individual Differences, 24, 187–192. Furnham, A., Forde, L., & Ferarri, K. (1999). Personality and work motivation. Personality and Individual Differences, 26, 1035–1043. Furnham, A., Jackson, C., Forde, L., & Cotter, T. (2001). Correlates of the Eysenck personality profiler. Personality and Individual Differences, 30, 587–594. Furnham, A., Trickey, G., & Hyde, G. (2012). Bright aspects to dark side traits. Personality and Individual Differences, 52, 908–913. Hogan, R., & Hogan, J. (1997a). Hogan personality inventory manual. Tulsa, OK: Hogan Assessment Systems. Hogan, R., & Hogan, J. (1997b). Hogan developmental survey manual. Tulsa, OK: Hogan Assessment Systems. Jensen, A. (2006). Clocking the mind. Amsterdam: Elsevier. Malhotra, N. (2008). Completion time and response effects in web surveys. Public Opinion Quarterly, 72, 914–934. Merten, T., & Ruch, W. (1996). A comparison of computerised and conventional administration of the German version of the Eysenck Personality Questionnaire and the Carnell Rating Scale for Depression. Personality and Individual Differences, 20, 281–291. Merten, T., & Siebert, K. (1997). A comparison of computerised and conventional administration of the EPQ-R and the CRS. Personality and Individual Differences, 22, 283–286. Molto, J., Segarra, P., & Avila, C. (1993). Impulsivity and total response speed to a personality questionnaire. Personality and Individual Differences, 15, 97–98.