The validity of self-reported driver competence: Relations between measures of perceived driver competence and actual driving skill

Transportation Research Part F 14 (2011) 155–163

Contents lists available at ScienceDirect

Transportation Research Part F journal homepage: www.elsevier.com/locate/trf

The validity of self-reported driver competence: Relations between measures of perceived driver competence and actual driving skill Anna Sundström ⇑ Department of Applied Educational Science, Umeå University, SE-901 87 UMEÅ, Sweden

a r t i c l e

i n f o

Article history: Received 20 April 2010 Received in revised form 23 September 2010 Accepted 18 November 2010

Keywords: Self-assessment Self-report Driving skill Validity

a b s t r a c t Perceived driver competence is commonly assessed by asking drivers to compare their own competence to that of the average driver. This method has been criticised however, and it has been suggested that perceived driver competence should be assessed with respect to specific aspects of competence. This study examines the validity of two such self-report instruments. 136 driving test candidates completed self-report instruments for Perceived Practical Driver Competence (PPDC), and Self-Assessment of Driving Skills (SADS), as well as items about confidence in passing the driving test and took the Swedish driving test. The strong positive correlation between PPDC and SADS scores indicated that the instruments measure the construct of perceived driver competence, which provided evidence of convergent validity. Moreover, the moderate relationship between confidence in passing the test and perceived driver competence provided support for discriminant validity as these measures can be assumed to tap outcome expectations and self-efficacy beliefs, respectively. In addition, the results were in line with previous findings, indicating that when perceived driver competence is related to driving performance rather than the skill of the average driver, a relatively large proportion of driving test candidates can accurately assess their own driver competence and there are no differences in overconfidence between males and females or drivers of different age. Ó 2010 Elsevier Ltd. All rights reserved.

1. Self-assessment of driver competence It is a well-known phenomenon that young drivers are more involved in accidents than older and more experienced drivers (Ferguson, 2003; OECD, 2006). The literature has provided several explanations for young drivers’ accident involvement. One of these explanations is that drivers are overconfident in their own skills (Gregersen & Bjurulf, 1996; OECD, 2006). The capability of making an accurate assessment of one’s own driving skill is of great importance in the driver’s regulation of his or her driving behaviour (Hatakka, Keskinen, Gregersen, Glad, & Hernetkoski, 2002). In order to drive safely, the demands of the task should match the driver’s competence. Adapting the behaviour to the demands of the task requires an accurate assessment of one’s own driving skills and the complexity of the situation. If the self-assessment is inaccurate, the driver might engage in driving tasks that are too demanding and unsafe (De Craen, Twisk, Hagenzieker, Elffers, & Brookhuis, 2007a). In several studies, subjective driving skill has been assessed in order to explore the hypothesised overconfidence of young drivers. The most common way of assessing perceived driving skills have been by asking drivers to compare their own skill to that of the average driver or their peers (see e.g. Delhomme, 1991; Goszcynska & Roslan, 1989; McCormick, Walkey, & Green, 1986; Svenson, 1981). These studies indicate that the majority of drivers believe they are more skilled than the average driver. For example, Svenson (1981) showed that 69% of Swedish drivers and 93% of American drivers rate themselves as more ⇑ Tel.: +46 90 786 95 21; fax: +46 90 786 66 86. E-mail address: [email protected] 1369-8478/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.trf.2010.11.011

156

A. Sundström / Transportation Research Part F 14 (2011) 155–163

skilful than the average driver. Further, Goszcynska and Roslan (1989) reported that 77% of Polish drivers rate themselves as more skilful than the average driver. Studies also report findings that suggest that men are more confident in their own skills than women (McKenna, Stanier, & Lewis, 1991), and that young drivers are more confident than older drivers when comparing their own skills to that of the average driver (Matthews & Moran, 1986; Soames Job, 1990). The relevance and reliability of this method of assessing perceived driving skill has been subject for discussion, however (Sundström, 2008a). Studies in the psychological field have indicated that people make biased assessments when comparing themselves to the ‘‘average person’’ (see e.g. Dunning, Heath, & Suls, 2004; Kruger, 1999). Moreover, research has suggested that people tend to overestimate their ability when the definition of the ability is ambiguous (Dunning, Meyerowitz, & Holzberg, 1989). The term ‘‘average driver’’ can be considered a vague term, which makes it difficult to obtain valid measures of peoples’ perceived driver competence (Groeger, 2001). Due to the problems associated with the average driver as a frame of reference, it has been proposed that perceived driver competence should be measured with respect to clearly defined aspects of driving skill and that perceived driver competence should be compared with observed driving performance in order to determine whether drivers are overconfident or not with respect to their own skills (Deery, 1999; Sundström, 2008a). This approach has been used in a few studies that attempt to examine the accuracy of drivers’ perceived driver competence (see e.g. De Craen et al., 2007a; Eby, Molnar, Shope, Vivoda, & Fordyce, 2003; Victoir, Eertmans, van den Bergh, & van den Broucke, 2005). More recently, two studies examined the accuracy of Finnish, Swedish and Dutch driving-license candidates’ perceived driver competence by comparing it with their performance in the driving test (Mynttinen, Sundström, Koivukoski, et al., 2009; Mynttinen, Sundström, Vissers, et al., 2009). Compared to studies that assess perceived driver competence in relation to that of the average driver, the results from these two studies indicate that a relatively large proportion of candidates can make a realistic assessment of their competence. About 50% of the Finnish candidates and between 30% and 40% of the Swedish candidates accurately assessed their driver competence with respect to vehicle manoeuvring, economic driving and traffic safety (Mynttinen, Sundström, Koivukoski, et al., 2009). Among the Dutch candidates between 40% and 50% made a realistic assessment of their competence depending on the competence area assessed (Mynttinen, Sundström, Vissers, et al., 2009). Moreover, different from previous studies where perceived driver competence has been assessed in relation to the average driver, the results from these studies show neither gender differences nor age differences with respect to accuracy of driver competence (Mynttinen, Sundström, Koivukoski, et al., 2009; Mynttinen, Sundström, Vissers, et al., 2009). These results indicate that novice drivers can more accurately assess their driver competence when assessing specific competences, rather than comparing their general driving skill to that of the average driver. As was pointed out by Mynttinen, Sundström, Koivukoski, et al. (2009) however, the findings need to be cross-validated in other samples in order obtain support for these hypotheses. The aim of the present study was to examine novice drivers’ accuracy of perceived driver competence in order to evaluate the validity of previous findings from these self-report instruments. Moreover, the aim was also to examine the construct validity of the self-report instruments for perceived driver competence used in the study by Mynttinen, Sundström, Koivukoski, et al. (2009) by administering them to a sample of Swedish driving-license candidates. Evidence of convergent validity (Kane, 2006) was obtained by examining the relationships between the two self-report instruments. Evidence for discriminant validity was obtained by examining the relationship between the self-report instrument tapping candidates’ confidence in performing tasks in the driving test and a measure of confidence in passing the driving test. These two measures can be regarded to tap two different theoretical constructs; self-efficacy beliefs and outcome expectations. Self-efficacy is an aspect of self-knowledge concerned with individuals’ perceptions of their own capabilities. Self-efficacy has been defined as ‘‘people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances’’ (Bandura, 1986, p. 391). Outcome expectations, on the other hand, are beliefs concerning the probable outcomes of actions (Schunk, 1991). Thus, an outcome expectation is a person’s estimate that a certain behaviour will produce a resulting outcome, whereas a self-efficacy belief is the person’s conviction that he or she can execute the behaviour needed to produce the desired outcome (Bandura, 1997). 2. Method 2.1. Procedure and participants Data was collected from candidates that took the Swedish driving test between October 2008 and January 2009. The candidates received a questionnaire by mail that they were asked to complete and bring with them to the driving test. They also received information about the study1 by mail. More detailed information was provided at the driving test centre, where informed consent was obtained as well (Alger, Henriksson, & Sundström, 2009). In total, the sample comprised 576 candidates. Of these, 357 agreed to participate in the study, completed the questionnaire and took the driving test. However, only 136 of these candidates completed the questionnaire before taking the driving test. Because some of the items about perceived driver competence are concerned with how confident candidates are that they can perform different tasks in the driving test, only those who completed the questionnaire before they took the test were selected for analysis in the present study. If all candidates would have been included, their ratings of how confident

1

The present study was conducted within a larger research project, aiming to study the inter-rater reliability of the driving test.

A. Sundström / Transportation Research Part F 14 (2011) 155–163

157

they were to perform tasks in the driving test would probably be affected by the outcome of the driving test for those candidates who filled out the questionnaire after the test. Of the 136 participants, 57% were female, 51% were privately educated learner drivers2 and 50% passed the driving test. Their age ranged from 18 to 47 years (M = 21.9). In the total sample (n = 576) 47% were female, 58% were privately educated learner drivers, and 42% passed the driving test. The age in the total sample varied between 18 and 64 years (M = 25.5). Thus, the participants in the study were somewhat younger on average and there was a slight over-representation of women, drivingschool students and the proportion of passed candidates were somewhat larger than in the total sample. A missing value analysis indicated that the levels of partial missing values were low, less than 5% for all variables. 2.2. Instruments 2.2.1. Questionnaire to driving test candidates The questionnaire included the self-report instruments for perceived driver competence used in Finland and Sweden (Mynttinen, Sundström, Koivukoski, et al., 2009). In this study the Swedish instrument is referred to as the scale for Perceived Practical Driver Competence (PPDC) and the Finnish instrument is referred to as the Self-Assessment of Driving Skills (SADS). In addition, the questionnaire included items about factors that candidates believe affect their result on the driving test, one item about confidence in passing the driving test, as well as an item about factors affecting the confidence in passing. In the sections below the items included in the questionnaire are described in more detail. 2.2.1.1. Perceived Practical Driver Competence. The scale for Perceived Practical Driver Competence used in this study is a revised version of a self-report instrument for perceived theory and practical driver competence entitled the Self-efficacy Scale for Driver Competence (SSDC) (Sundström, 2008b). As indicated by its name, the theoretical basis of the SSDC is the psychological construct of self-efficacy. The SSDC measures confidence in performing tasks in the Swedish driving-license test, and taps two dimensions of perceived driver competence: theory and practical. These dimensions comprise five aspects of driver competence; vehicle knowledge and manoeuvring, economic driving, and traffic regulations, traffic safety and personal circumstances and goals in life. These aspects are represented in the Swedish curriculum for driver education and in the contents of the driving-license test, as well as in the Goals for Driver Education (GDE) model, which is a conceptual model of driver training and education (Hatakka et al., 2002). For a detailed description of the development of the SSDC, see Sundström (2008b). The rationale for developing the SSDC was to assess the goals about self-assessment of driver competence stated in the curriculum for driver education in Sweden. The reliability and validity of the SSDC has been examined by pre-testing the instrument, but it has not been implemented in the driver testing procedure. The findings from the psychometric evaluations have provided support for internal aspects of construct validity as the psychometric properties were sound, and as the structure of the SSDC scores corresponded to the two dimensions of perceived theory and practical driver competence in the theoretical model underlying the instrument (Sundström, 2008b; Wänglund, 2010). Although the psychometric evaluations indicated that the SSDC seems to function fairly well, some improvements of the scale were suggested by Wänglund (2010). The SSDC was revised according to these suggestions; four items that demonstrated poor fit to the item response theory model that was used to calibrate the items were removed, the wording of one misfitting item was revised, and the number of response categories was decreased from 10 to 5. Moreover, all categories were provided with a label, making each category more comprehensible. The psychometric evaluation of the SSDC showed that Cronbach’s alpha was very high, which raised the question of redundancy between some items in the instrument. Therefore, in the revision of the SSDC, items that had formulations very similar to those of other items were removed. In addition, the wording of some items in the SSDC was somewhat revised to make the language clear and straightforward. Some of the items were also shortened. The original version of the SSDC comprised two subscales; Perceived Theory Driver Competence (PTDC) and Perceived Practical Driver Competence (PPDC). The revised version of the SSDC that was used in the present study included only the subscale of Perceived Practical Driver Competence, which included 11 items (item content is presented in Table 1). Therefore, the revised version of the SSDC will be referred to as the PPDC. The reason for only including PPDC items was that the participants had already taken the theory test but received the PPDC before the driving test. When completing the PPDC, candidates were asked to indicate how confident they are in their ability to successfully complete tasks in the driving test, e.g. manoeuvre the car. Responses were made on a five-point scale, where the categories are labelled: very sure, sure, neither sure nor unsure, unsure, and very unsure. 2.2.1.2. Self-Assessment of Driving Skills. In Finland, driving-license candidates complete a self-report instrument where they assess their present skill as a driver with respect to seven different areas, before they take the driving test (Mynttinen, Sundström, Koivukoski, et al., 2009). The assessments are made on a five-point scale, ranging from poor to excellent. During the Finnish driving test the driver examiner makes an assessment of the candidate’s performance in the corresponding areas. After the driving test, the candidate’s self-assessment is compared with the driver examiner’s assessment, which provides

2 The categorisation of private learner drivers and driving-school students were based on how they applied to the driving test; whether they applied via a driving school or not.

158

A. Sundström / Transportation Research Part F 14 (2011) 155–163

Table 1 Item statistics for the Perceived Practical Driver Competence (PPDC).

1 2 3 4 5 6 7 8 9 10 11

Items

M

Mdn

SD

r

Min–max

Perform a security check Use different braking methods Show good detection routines in various traffic environments Interact with other road users Economic driving Drive with adequate safety margins Adapt the speed to the prevailing conditions Apply traffic regulations Foresee different courses of event in traffic Identify risks in various traffic situations Manoeuvre the car

3.89 4.39 4.07 4.18 4.15 4.31 4.49 4.44 3.84 4.15 4.37

4.0 4.0 4.0 4.0 4.0 4.0 5.0 4.0 4.0 4.0 4.0

0.77 0.60 0.67 0.70 0.78 0.66 0.60 0.58 0.67 0.62 0.64

.37 .48 .60 .59 .46 .59 .56 .59 .64 .64 .53

2–5 3–5 2–5 1–5 2–5 3–5 3–5 3–5 2–5 3–5 2–5

Note: In the PPDC, the candidates were asked how confident they are in their ability to successfully complete the subsequent tasks in the driving test.

Table 2 Item statistics for the Self-Assessment of Driving Skills (SADS). Items

M

Mdn

SD

r

Min–max

1. 2. 3. 4. 5. 6. 7.

4.13 4.07 4.19 4.03 3.96 4.06 4.03

4.0 4.0 4.0 4.0 4.0 4.0 4.0

0.67 0.70 0.71 0.81 0.72 0.80 0.88

.55 .67 .56 .62 .62 .67 .46

2–5 1–5 2–5 1–5 2–5 1–5 1–5

Manoeuvring the car Controlling the vehicle independently in various traffic situations Identifying and showing consideration to pedestrians and cyclists Adapting driving to the traffic flow, planning in advance Recognizing and avoiding risks Interacting with other road users Economic driving

Note: In the SADS, the candidates were asked to assess their present driving skill in the following areas.

the starting point for a structured feedback discussion. In the present study, the items were translated into Swedish (by the author) using an English translation and description of the items in the Finnish instrument (see Table 2). The translated items were included in the questionnaire in order to examine the convergent validity of the PPDC scores. In this paper the instrument is referred to as the Self-Assessment of Driving Skills (SADS). 2.2.1.3. Confidence in passing the test. In order to examine the discriminant validity, the relationship between perceived driver competence (as assessed by the PPDC and SADS) and confidence in passing the test were analysed. From the perspective of self-efficacy theory, confidence in performing tasks can be regarded as self-efficacy beliefs, whereas confidence in passing the test can be regarded as an outcome expectation. The questionnaire included one item about confidence in passing the driving test, one item about factors that affect their confidence in passing the test, as well as one item about factors that candidates believe can affect their result on the driving test (e.g. test anxiety, familiarity with the car). 2.2.2. Driver examiner assessment of driving performance The procedure of the Swedish driving test is that the driver examiner makes a holistic judgment of the candidate’s driving performance with respect to four competences that relate to the driver’s attentiveness to risks; vehicle knowledge and manoeuvring, economic driving, traffic regulations and traffic safety. During the test different traffic situations are observed, and the performance in each situation is related to the four aspects of competence. If the candidate fails in any respect, the examiner notes in which specific traffic situation the error did occur. One error is enough to fail the candidate. In the present study, the driver examiners assessed the candidates’ over-all driving performance using a six-point scale instead of the common pass/fail judgment, in order to obtain a more nuanced description of the performance. Ratings from 1 to 3 represented different degrees of failure, whereas ratings between 4 and 6 represented different degrees of a passed test. Each scale category was provided with a clear description.3 2.3. Analysis The items in the questionnaire were subjected to statistical analysis. First, descriptive statistics (mean, median and standard deviation) were computed for the items in the PPDC and the SADS. Moreover, Cronbach’s alpha as well as corrected item-total correlations (r) was computed for the PPDC and the SADS. Second, the relationships between the PPDC, SADS,

3 On the six-point scale, the categories were described as: (1) Fail. Very poor driving, severe lack of competence in several areas. (2) Fail. Poor driving, lack of competence in several areas. (3) Fail. Lack of competence in one isolated area that is of significance for traffic safety. (4) Pass. Lack of competence not particularly significant with respect to traffic safety. (5) Pass. Good driving performance. (6) Pass. Very good driving performance (Alger et al., 2009).

A. Sundström / Transportation Research Part F 14 (2011) 155–163

159

and confidence in passing the test were examined using Spearman’s rank-order correlation coefficient, rs, due to ordinal level data. In order to examine how accurately the candidates can assess their own driver competence, the ratings of perceived driver competence (PPDC and SADS scores) were correlated with driver examiners assessments. In order to obtain a more nuanced description of the self-assessment accuracy, the PPDC and SADS scores were also compared with the performance on the driving test (pass/fail). The candidate’s average ratings on the PPDC and SADS items respectively were used as measures of perceived driver competence. Based on their average PPDC ratings, candidates were categorised as unsure, neither sure nor unsure, or sure of performing tasks in the test. The categorisation was based on the labels of the rating scale. Categories 1 and 2 on the five-point rating scale were labelled ‘‘very unsure’’ and ‘‘unsure’’ and therefore ratings between 1 and 2.9 were categorised as unsure. Category 3 was labelled ‘‘neither sure nor unsure’’ and therefore those who had an average score between 3.0 and 3.9 were categorised as neither sure nor unsure. Categories 4 and 5 were labelled ‘‘sure’’ and ‘‘very sure’’ and therefore those who had an average score between 4.0 and 5.0 were categorised as sure. Those who were sure of performing tasks in the test and passed the test, as well as those who were unsure or neither sure nor unsure and failed the test, were considered to have a realistic view of their competence. Those who were sure of performing tasks in the test and failed the test were considered to overestimate their competence. Those who were unsure as well as neither sure nor unsure of performing tasks in the test and passed the test were considered to underestimate their competence. For the SADS scores a similar procedure was adopted. Based on their average SADS responses, candidates’ self-assessed driving skill were categorised as poor, mediocre or good.4 The categorisation was based on the labels on the extremes of the rating scale. The same intervals were used as for the categorisation of the SSDC responses. Those who perceived themselves as good and passed the test, as well as those who perceived themselves as poor or mediocre and failed the test, were considered to have a realistic perception. Those who perceived themselves as poor or mediocre and passed the test were considered to underestimate their competence. Those who perceived themselves as good and failed the test were considered to overestimate their competence. 3. Results 3.1. Descriptive statistics The candidates assessed their perceived driver competence in the PPDC as well as in the SADS. In Table 1 descriptive statistics for the PPDC are presented. Between 70% and 95% of the candidates were fairly confident or very confident that they could perform the tasks in the driving test. The mean and median indicate that the candidates on average were fairly confident in successfully performing the tasks included in the driving test (see Table 1). Item 2 ‘‘use different braking methods’’ obtained the highest average rating; the median indicates that candidates were very confident in performing this task. Item 9, ‘‘foresee different courses of events in traffic’’, which obtained the lowest average rating, still had a mean of 3.84. Thus, candidates were fairly confident in performing all tasks stated in the PPDC. Moreover, the results indicated that the PPDC ratings were homogenous as the internal consistency was high (a = .85), and the item-total correlations were of moderate strength (see Table 1). It is worth noting, however, that item 1 had a lower item-total correlation than the other items. This might be explained by the fact that this item focuses on security check and not actual driving, as the other items do. Descriptive statistics for the SADS are presented in Table 2. The candidates made positive assessments of their driving skills. Between 75% and 90% of the candidates assessed their driver competence as a 4 or 5 on the five-point scale, where 5 corresponded to ‘‘excellent’’. On average, candidates assessed their skills around 4 on the five-point scale. The high internal consistency (a = .84), and the relatively high item-total correlations (see Table 2) indicated that the items in the SADS are homogenous. 3.2. Convergent and discriminant validity The relationships between items in the PPDC and items in the SADS that assessed similar driving skills were analysed in order to examine the convergent validity. Based on similarities in item content, seven items in the PPDC were compared to six items in the SADS (see Table 3). Due to different phrasings of the items in the PPDC and SADS, the items were not identical even though they assessed the same competence area. There were moderate to strong positive correlations between the items assessing similar competences (see Table 3). In addition, there was a strong positive relationship between the total scores on the PPDC and the SADS, rs = .80 (p < .001). The relation between candidates’ confidence in performing tasks in the test assessed in the PPDC and their confidence in passing the test was analysed in order to examine the discriminant validity. The high ratings on the PPDC indicated that the candidates were fairly confident that they could complete the tasks included in the driving test. When asked about their confidence in passing the driving test; 4.5% were very sure, 41.4% were fairly sure, 49.6% were neither sure nor unsure, 3.0% were fairly unsure, and 1.5% were very unsure of passing the test. There was a moderate correlation between their confidence in passing the test and their total PPDC score, rs = .34 (p < .01).

4 Those who had an average SADS score of 1.0–2.9 were categorised as poor, those who had an average score of 3.0–3.9 were categorised as mediocre, and those who had an average score of 4.0–5.0 were categorised as good.

160

A. Sundström / Transportation Research Part F 14 (2011) 155–163

Table 3 Correlations between items with similar content in the PPDC and the SADS. PPDC 3 4 5 7 9 10 11

Show good detection routines in various traffic environments Interact with other road users Economic driving Adapt the speed to the prevailing conditions Foresee different courses of event in traffic Identify risks in various traffic situations Manoeuvre the car

3 6 7 4 5 5 1

SADS

rs

Identifying and showing consideration to pedestrians and cyclists Interacting with other road users Economic driving Adapting driving to the traffic flow, planning in advance Recognizing and avoiding risks. Recognizing and avoiding risks Manoeuvring the car

.43** .72** .73** .36** .55** .52** .64**

Note: In the PPDC, candidates were asked how confident they are in their ability to successfully complete eleven different tasks in the driving test, whereas in he SADS, the candidates were asked to assess their present driving skill in seven different areas. ** p<.01.

The candidates were asked to what extent a number of factors affect their confidence in passing the test. 55% of the candidates answered that test anxiety was highly related to the fact that they were unsure of passing the test. About 30% of the candidates responded that familiarity with test standards, that something unexpected might happen during the test and the familiarity with the car were factors highly related to how sure they were of passing the test. The candidates were also asked to report on other factors that they believe affect their confidence in passing the test. This resulted in a considerable number of factors such as: which driver examiner makes the assessment; the temper and mood of the driver examiner; whether one has booked the test as a private learner driver and not via a driving school; that one might miss something important in the test; that one might make a small error during the test; that one is not familiar with the traffic environment; language difficulties; test anxiety and stress. In addition, the candidates were asked to what extent they believed that a number of factors would affect their result on the practical test. 86% of the candidates responded that they believed that their own competence affected the result on the driving test to a great extent or a fairly great extent. About 50% of the candidates responded that test anxiety, familiarity with the car, which traffic situations are tested and traffic intensity are factors that affected the result on the driving test to a great extent or to a fairly great extent. This indicates that even though the candidates’ own competence is seen as the most important factor that affects the result on the driving test, they believe that there are other factors operating as well. 3.3. Candidates’ accuracy of self-assessment The accuracy of perceived driver competence was examined by correlating PPDC and SADS scores with examiner assessments. For both self-report instruments, the correlations were low and not significantly different from zero (PPDC rs = .044; SADS rs = .083). This indicates that candidates and examiners do not agree on the assessments of driver competence. In order to obtain a more nuanced description of candidates’ self-assessment accuracy, the perceived competence (based on average PPDC and SADS responses) for each individual were compared with the driver examiner’s assessment of the performance on the driving test in terms of pass or fail. The comparison of PPDC scores and test performance showed that 50% made a realistic assessment of their competence, 13% underestimated and 37% overestimated their competence. The comparison of SADS scores and test performance indicated that 49% made a realistic assessment, 19% underestimated and 31% overestimated their driver competence. Thus, the comparison of perceived driver competence and test performance yields similar results for the PPDC and SADS. The two measures of self-assessment accuracy were also examined for differences in gender and age. There were no significant differences between men and women with regard to accuracy of self-assessment for neither of the measures. Furthermore, there were no significant age differences between those who overestimated, underestimated or made a realistic assessment of their driver competence for any of the self-assessment accuracy measures. 4. Discussion The aim of this study was to examine the construct validity of two self-report instruments for perceived driver competence; the SADS and PPDC. Moreover, the aim was also examine the validity of findings from previous studies using these self-report instruments. 4.1. Evidence of construct validity The results from the study provided support for construct validity of the PPDC and SADS. First, there was a strong positive relationship between the total scores on the PPDC and the SADS. This was in line with the expectations, and provided support for convergent validity as it indicates that the instruments measure different aspects of the same construct, that is, perceived driver competence. At the item level correlations were moderate to strong. This could be expected as even though both instruments had items about, e.g. manoeuvring, the items were formulated differently; in the PPDC the candidates assessed how confident they are that they can successfully manoeuvre the car in the driving test, whereas in the SADS, the candidates

A. Sundström / Transportation Research Part F 14 (2011) 155–163

161

assessed their present car manoeuvring skill. Second, the internal consistency indicated that items in the PPDC as well as the SADS are homogenous and measure a common construct, which is in line with the theoretical assumption that the instruments tap Perceived Practical Driver Competence. Third, the discriminant validity was examined by analysing the relationship between confidence in performing tasks in the driving test (PPDC scores) and confidence in passing the test. Drawing on self-efficacy theory, the PPDC can be viewed as a measure of self-efficacy beliefs of driver competence, whereas confidence in passing the test can be viewed as an outcome expectation (Bandura, 1997). Based on this, we could expect the scores on the PPDC to be only weakly or moderately related to confidence in passing the test. This expectation was confirmed as there was only a moderate correlation between PPDC scores and confidence in passing the test. In addition, the questionnaire responses indicated that many factors other than confidence in one’s skills affect how confident candidates are of passing the test. For example, many candidates stated that test anxiety, the degree of familiarity with the test standards, and that something unexpected might happen during the test affect their confidence in passing the test to a great extent. Thus, it seems as their confidence in performing the tasks in the driving test is only one of several factors that affect their confidence in passing the test. 4.2. Self-assessment accuracy Most previous studies of self-assessment of driver competence have asked drivers to compare their own skill to that of the average driver (Sundström, 2008a). These studies have indicated that the majority of drivers are overconfident as they believe they are more skilled than the average driver. However, studies that compare perceived driver competence to measures of actual driving show that a relatively large proportion can make a realistic assessment of their competence (Mynttinen, Sundström, Koivukoski, et al., 2009; Mynttinen, Sundström, Vissers, et al., 2009). This indicates that the strategy used to examine accuracy of perceived driver competence affects the results obtained. The second aim of this study was to examine the self-assessment accuracy of novice drivers in order to provide further support for these findings. The results indicated that about 50% of the candidates had an accurate perception of their own driver competence, which is in line with the results found in the previous studies using version of the same self-report instruments (Mynttinen, Sundström, Koivukoski, et al., 2009; Mynttinen, Sundström, Vissers, et al., 2009). In addition, similar results have been found in studies using other self-report questionnaires as well. In a Dutch study of drivers self-assessment accuracy, 68% were found to be well calibrated with respect to their own skills, 15% underestimated and 17% overestimated their driver competence (De Craen, Twisk, Hagenzieker, Elffers, & Brookhuis, 2007b). In the present study, about 35% overestimated their driver competence. Previous studies that examine overconfidence by asking drivers to compare themselves to the average driver, have found that the majority of drivers are overconfident (Goszcynska & Roslan, 1989; Svenson, 1981). In this perspective, 35% overestimation in the present study is fairly low. However, from the viewpoint that accurate self-assessments can be regarded as important for safe driving, it is desirable to take measures to improve selfassessment accuracy and thereby decrease the proportion that overestimate their skills. Studies that assess perceived driver competence in relation to the average driver have indicated that males are more confident than females (McKenna et al., 1991), and that younger drivers are more confident than older drivers (Matthews & Moran, 1986; Soames Job, 1990). The results from the present study indicate that there are no differences in self-assessment accuracy with respect to age or gender, when perceived driver competence is related to observed driving performance instead of the skills of the average driver. This provides support for the findings reported in the studies by Mynttinen, Sundström, Koivukoski, et al. (2009) and Mynttinen, Sundström, Vissers, et al. (2009). These results indicate that the strategy used when assessing perceived driver competence is of great importance for the results obtained. The strategy used in the present study, where perceived driver competence is assessed with respect to specific competence areas, is beneficial when examining the accuracy of driver competence because the assessments of perceived competence can be related to measures of observed competence. In this way it is possible to evaluate whether drivers are overconfident with respect to their actual competence instead of in relation to the skills of the average driver (Deery, 1999). The present study has some limitations that need to be mentioned. First, the missing data analysis indicated those who answered the questionnaire were not entirely representative for the population of candidates that take the driving test. The participants in the study were somewhat younger, and the proportion of women and driving-school students were somewhat larger in the sample than in the population. However, as there were no differences in self-assessment accuracy with regard to gender or age, the differences in proportion women and driving-school students probably does not affect the results of this study. Moreover, when comparing the results from this study to the previous studies (Mynttinen, Sundström, Koivukoski, et al., 2009; Mynttinen, Sundström, Vissers, et al., 2009), there are some differences in the measures used that need to be considered. First, in the present study a revised version of the PPDC was used. In this version a five-point scale was used instead of the previous 10-point scale, and some items were revised. Thus, the version of the PPDC used in this study was not identical to the previous version. Second, in order to use the Finnish self-report instrument for perceived competence on the Swedish sample, the items were translated into Swedish. As the translation was made for research purposes, a simple translation procedure was used, in which an English translation of the items was used to translate the items into Swedish. However, this implies that we cannot presume that the Swedish translation exactly corresponds to the items in the Finnish instrument. Another difference between this study and the study by Mynttinen, Sundström, Koivukoski, et al. (2009) is that in the present study the driver examiners made a holistic assessment of the performance in the driving test, whereas in the previous study

162

A. Sundström / Transportation Research Part F 14 (2011) 155–163

different competence areas were assessed by examiners. Consequently, the measure of accuracy of self-assessment addresses different competence areas (vehicle manoeuvring, traffic safety, economic driving) in the previous studies, whereas it focuses on the whole domain of Perceived Practical Driver Competence in the present study. Despite these limitations this study makes an important contribution to the field, as the results provide further evidence for the fact that when perceived driver competence is assessed with respect to specific aspects of competence, and is compared to driving performance, the majority does not overestimate their own driver competence. In order to further explore what impact the strategy of measurement of self-assessment accuracy has on the proportion of drivers that are regarded to make a realistic assessment, it would be interesting to let candidates rate their competence with respect to specific aspects of their skill as well as in comparison to the average driver.

4.3. Summarised conclusions In conclusion, the results from this study provided support for different aspects of the construct validity of the PPDC and SADS. First, the results indicated that the two self-reports instruments measure homogenous constructs, and there was support for the fact that this construct is perceived driver competence. The perceptions of driver competence was also showed to measure a construct different from confidence in passing the test, which provides further support for the construct validity of the self-report instruments. The results from this study also provided support for the fact that perceived driver competence, measured by asking drivers to assess specific aspects of their perceived driver competence, and comparing these to observed driving performance, generates another picture of how capable drivers are to accurately assess their own driver competence, than when overconfidence is assessed by comparing one’s own driving skill to that of the average driver. When perceived driver competence is assessed with respect to specific competences, the self-assessment can be compared to the actual driving performance in order to evaluate its accuracy. Therefore it can be argued that this strategy provides more reliable and useful information of self-assessment accuracy than by asking how skilled drivers believe they are in relation to the average driver. Besides using the self-report instruments for perceived driver competence for research purposes exploring novice drivers’ self-assessment accuracy, they could be used as pedagogical tools in driver education and driver testing. In the Finnish driver testing system, candidates assess their driver competence before the driving test and obtain feedback on the accuracy of their self-assessment after the test. Using the instrument under study in this paper, a similar approach could be implemented in Sweden. Thereby, all driving candidates would be given the opportunity to reflect on their competence as a driver and obtain feedback from the examiner, not only on their test performance, but also on their self-assessment.

References Alger, S., Henriksson, W., & Sundström, A. (2009). Likvärdigheten i körprovet. En studie av samstämmighet i bedömningen (consistency of assessment of the driving test) (BVM 39:2009). Umeå University, Department of Educational Measurement. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman. De Craen, S., Twisk, D. A. M., Hagenzieker, M. P., Elffers, H., & Brookhuis, K. A. (2007a). Do young novice drivers overestimate their driving skills? Paper presented at the Young researchers seminar in SWOW, Brno, Czech Republic. De Craen, S., Twisk, D. A. M., Hagenzieker, M. P., Elffers, H., & Brookhuis, K. A. (2007b). Overestimation of skills affects drivers’ adaptation to task demands. Paper presented at the fourth international driving symposium on human factors in driver assessment, training and vehicle design, Washington, USA. Deery, H. (1999). Hazard and risk perception among young novice drivers. Journal of Safety Research, 30(4), 225–236. Delhomme, P. (1991). Comparing one’s driving with others’: Assessment of abilities and frequency of offences. Evidence for a superior conformity of selfbias? Accident Analysis and Prevention, 23(6), 493–508. Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment. Implications for health, education and workplace. Psychological Science in the Public Interest, 5(3), 69–121. Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989). Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. Journal of Personality and Social Psychology, 57(6), 1082–1090. Eby, D. W., Molnar, L. J., Shope, J. T., Vivoda, J. M., & Fordyce, T. A. (2003). Improving older driver knowledge and self-awareness through self-assessment: The driving decisions workbook. Journal of Safety Research, 34, 371–381. Ferguson, S. A. (2003). Other high-risk factors for young drivers – How graduated licensing does, doesn’t, or couldn’t address them. Journal of Safety Research, 34, 71–77. Goszcynska, M., & Roslan, A. (1989). Self-evaluation of drivers’ skill: A cross-cultural comparison. Accident Analysis and Prevention, 21(3), 217–224. Gregersen, N. P., & Bjurulf, P. (1996). Young novice drivers: Towards a model of their accident involvement. Accident Analysis and Prevention, 28, 229–241. Groeger, J. A. (2001). Understanding driving. Applying cognitive psychology to a complex everyday task. Philadelphia: Taylor and Francis. Hatakka, M., Keskinen, E., Gregersen, N. P., Glad, A., & Hernetkoski, K. (2002). From control of the vehicle to personal self-control; Broadening the perspectives to driver education. Transportation Research Part F: Traffic Psychology and Behaviour, 5, 201–215. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: American Council on Education/Praeger Publishers. Kruger, J. (1999). Lake Wobegon be gone! The ‘‘below-average effect’’ and the egocentric nature of comparative ability judgements. Journal of Personality and Social Psychology, 77(2), 221–232. Matthews, M. L., & Moran, A. R. (1986). Age differences in male drivers’ perception of accident risk: The role of perceived driving ability. Accident Analysis and Prevention, 18, 299–314. McCormick, I. A., Walkey, F. H., & Green, D. E. (1986). Comparative perceptions of driver ability – A confirmation and expansion. Accident Analysis and Prevention, 18(3), 205–208. McKenna, F. P., Stanier, R. A., & Lewis, C. (1991). Factors underlying illusory self-assessment of driving skill in males and females. Accident Analysis and Prevention, 23(1), 45–52.

A. Sundström / Transportation Research Part F 14 (2011) 155–163

163

Mynttinen, S., Sundström, A., Koivukoski, M., Hakuli, K., Keskinen, E., & Henriksson, W. (2009). Are novice drivers overconfident? A comparison of selfassessed and examiner-assessed driver competences in a Finnish and a Swedish sample. Transportation Research Part F: Traffic Psychology and Behaviour, 12(2), 120–130. Mynttinen, S., Sundström, A., Vissers, J., Koivukoski, M., Hakuli, K., & Keskinen, E. (2009). Self-assessed driver competence among novice drivers – A comparison of driving test candidate assessments and examiner assessments in a Dutch and Finnish sample. Journal of Safety Research, 40(4), 301–309. OECD (2006). Young drivers: The road to safety. OECD/CEMT Publications. Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26(3), 207–231. Soames Job, R. F. (1990). The application of learning theory to driving confidence: The effect of age and the impact of random breath testing. Accident Analysis and Prevention, 22(2), 97–107. Sundström, A. (2008a). Construct validation and psychometric evaluation of the Self-efficacy Scale for Driver Competence. European Journal of Psychological Assessment, 24(3), 198–206. Sundström, A. (2008b). Self-assessment of driving skill – A review from a measurement perspective. Transportation Research Part F: Traffic Psychology and Behaviour, 11(1), 1–9. Svenson, O. (1981). Are we all less risky and more skillful than our fellow driver? Acta Psychologica, 47, 143–148. Victoir, A., Eertmans, A., van den Bergh, O., & van den Broucke, S. (2005). Learning to drive safely: Social-cognitive responses are predictive of performance rated by novice drivers and their instructors. Transportation Research Part F: Traffic Psychology and Behaviour, 8(1), 59–74. Wänglund, A. (2010). Using the rating scale model to examine the psychometric properties of the Self-efficacy Scale for Driver Competence. European Journal of Psychological Assessment, accepted for publication.