- Email: [email protected]
Predicting accident involvement with the motorcycle operator skill test
Accid Anal. & prey., Vol 13, No 4, pp. 307-318, 1981 Printed in Grea~ Britain
0~01-4575/81/040307-12502.00/0 O 1981 Pergamon Press Lid.
PREDICTING ACCIDENT INVOLVEMENT WITH THE MOTORCYCLE OPERATOR SKILL TEST BRIAN A. JONAH, NANCY E. DAWSONand BARRY W. E. BRAGGt Road and Motor Vehicle Traific Safety Branch, Transport Canada, Ottawa, Canada KIA ON5 (Received 7 August 1979; in revised[orm 22 August 1980)
Abstract--The present study was conducted to assess the predictive validity of the Motorcycle Operator Skill Test (MOST) developed by McPherson and McKnight (1976). It was anticipated that among licensed motorcyclists those scoring higher on this off-road skill test would be less likely to become involvedin an accident than motorcyclists scoring lower. A sample of motorcycle operator licence applicants was administered the MOST in addition to the regular licensing test. These motorcyclistswere contacted about a year later and queried about their riding experiences including motorcycleaccidents. Driver records were also searched for six months after the motorcyclistwas licensed. Although bivariate analyses indicated that high scorers on the MOST were more likely to have had an accident than low scorers, multivariateanalyses revealed no effect of test performance on subsequent accident involvement. However, accident likelihood was found to decrease as a function of age and increase as a function of vehicle distance travelled (i.e. exposure to risk). INTRODUCTION The boom in the popularity of motorcycles in the last 10-15 years has created concern among traffic safety experts, the focus of which has been a 125% increase in motorcycle accidents reported in the United States between 1965 and 1975 [Ford and Anderson, 1978] and a 250% increase observed in Canada [White 1980] during the same period. In order to deal with the motorcycle accident problem, researchers have attempted to determine the causes of motorcycle accidents. Several studies have implicated inexperience as a major causal factor [California Highway Patrol, 1968; Harano and Peck, 1968; Reiss and Haley, 1968; Waller, 1972; Organization for Economic Co-operation and Development, 1978]. Motorcycle safety experts [OECD, 1978] have suggested improved testing for licence applicants based on accident data and a comprehensive task analysis. The present study examined the predictive validity of one such test, the Motorcycle Operator Skill Test (MOST). Most jurisdictions in North America now require motorcyclists to obtain an operator's licence by passing tests of knowledge and riding skill. However, relatively little research has assessed the effectiveness of these motorcycle licensing programmes. Using time series analysis, Hall (1976) observed a decrease in the number of motorcycle accident fatalities per number of registered motorcycles during the year following the implementation of motorcycle operator licensing programmes in a number of American states. However, it was found that this reduction in fatalities continued after the first year of licensing programmes only in those states with both off-road and in-traffic skill tests and not in those states with only off-road tests. A study conducted in Western Australia failed to demonstrate that a graded motorcycle licensing programme had any effect on motorcycle accidents [Smith, 1976]. McPherson and McKnight [1976] have maintained that for a test to be a valid measure of critical on-street operating skills, its development should begin with a task analysis of the skills necessary for safe on-street operation of a motorcycle. Based on a task analysis performed by McKnight and Heywood [1974], McPherson and McKnight constructed the Motorcycle Operator Skill Test which consists of nine riding exercises of increasing difficulty. Initial test validation indicated a high correspondence between MOST scorers and the overall ratings of the rider's skill made by expert motorcycle riders. Recently, Jonah and Dawson [1979] have shown that performance on the MOST was better among younger, male, experienced (i.e. distance travelled) riders who had not received any formal training in motorcycling. The M O S T was used in an evaluation of an improved motorcyclist licensing project in California [Anderson, 1980]. Applicants for motorcycle operator licences were randomly ?Present address: Abt Associates of Canada, 23 Yonge Street, Toronto, Ontario, Canada M4P 1G2. 307
308
B.A. Jorq~H et al,
assigned to one of three conditions: (i) the current licensing system;(ii) the new licensing consisting of a new knowledge test and the MOST; or (iii) the new licensing system with the addition of remedial training on the MOST for applicants who failed the test. Analysis of official driver records indicated that applicants from either of the two groups which granted licences on the basis of the new test had significantly fewer accidents in the year following licensing than applicants licensed through the current system. The greatest difference in accident history was found between applicants in the new system with training and applicants in the current system such that there were 21% fewer accidents for the new system group. It should be noted that the riders' exposure in terms of vehicle miles travelled during the criterion period was not controlled in the California study so that it is possible that the lower accident involvement of riders in the new licensing programmes was due to their lower exposure. The fact that the pass rate was lower in the new systems and the fact that the applicants in the new systems took longer to obtain their licences suggest that applicants in the new systems had lower exposure than applicants in the current system. Cognizant of this problem with exposure, Anderson attempted to control it by using instruction permit issuance as a proxy measure of temporal exposure and covarying it in analysis of covariance performed on the accident involvement of licensed riders. While the improved testing group with remedial training was found to have fewer accidents than the current system group, the improved testing group without training was not significantly different from the control group. Anderson [1980] concluded that the major impact of the new licensing system per se (i.e. no remedial training) was to reduce the riding exposure of motorcyclists. It was also concluded that the training component of the program had an upgrading effect on the skill of motorcycle riders. However, since vehicle distance travelled was not controlled, it is still possible that the group with training had fewer accidents because they had lower exposure in terms of distance.'~ Although the California study may well have achieved the objective of reducing motorcycle accidents by making the licensing test more difficult, the basic question concerning the predictive validity of the MOST was not addressed in the California study. As pointed out by Anderson, "None of the results of this study provided support for the hypothesis that the content of the improved knowledge and skill tests, per se, were an important basis of program effectiveness" [1980, p. 398]. Furthermore, no attempt to assess the independent contribution of the MOST to the improvement of the licensing programme was made in the California study. Therefore, the purpose of the present study was to assess the predictive validity of the Motorcycle Operator Skill Test. If the test is valid, then motorcyclists with high scores on it should have fewer accidents than motorcyclists with low scores on it during a follow-up period. METHOD
The testing was conducted between June and August, 1977 at the Ontario Ministry of Transportation and Communications' vehicle operator licensing centre at Downsview, Ontario by two provincial licensing examiners who were trained to administer the MOST in a uniform and objective manner. About 12 months after licensing, follow-up interviews were conducted with most of the motorcyclists who had been tested. Driver record searches were also conducted to determine whether the motorcyclists had been involved in any motorcycle accidents. Subjects: The subjects consisted of 584 male and 53 female applicants for a motorcycle operator's license, with ages ranging from 16 to 57 years. The sample of 637 applicants includes only those individuals who passed the provincial licensing test which represnts 80% of licence applicants. Of this sample, 400 motorcyclists were interviewed during the follow-up phase and 538 driver records were obtained. Apparatus and Materials: A test course was set up at the licensing centre by painting exercise lines on the pavement in accordance with specifications outlined in the administrator's manual for the MOST [see Jonah and Dawson, 1979]. Scoring: The MOST is comprised of the following nine exercises each of which is scored separately by assigning penalty points to the operator for errors committed on the exercise: (1) starting on a hill, (2) making a sharp turn, (3) accelerating in a curve, (4) slowing in a curve, (5) ;Apparently an attempt is being made to control for exposure in terms of vehicle distance travelled in the California study.
Predictingaccidentinvolvementwith the MotorcycleOperatorSkill Test
309
making a controlled stop, (6) judging maximum safe turning speed, (7) emergency stopping in a straight line, (8) making a quick evasive turn and (9) emergency stopping in a turn. Basically, errors of three types are scored: path (i.e. straying outside of the lines); distance (i.e. stopping too late) and time/speed (i.e. travelling too slowly). Details for assigning penalty points can be found in the examiner's manual for the MOST. If the operator accumulated 13 or more penalty points, testing was terminated and no further exercises were attempted. The testing was also terminated if the operator dropped his motorcycle or ran into equipment. Termination of the testing meant that the operator failed the test. Therefore, one measure of performance was whether or not the operator failed the test. Once the data was collected by the examiner, a total score on the test was computed using the following formula: 9
Total Score = ~ [(TPi + 1) - PLi]Ai i=1
where TPi = total possible points that could be lost on exercise i, PLi = actual points lost on exercise i, Ai = attempted exercise (1 -- yes, 0 = no). The resultant scores ranged from 1 to 80 with a larger score representing a greater degree of skill. Note than an operator who terminated on exercise 3, for example, and therefore did not attempt exercise 4, received a score of 0 on exercise 4 and on subsequent exercises whereas an operator who attempted exercise 4 but lost all the points possible received a score of 1. This scoring system provided a continuous measure of skill in addition to the dichotomous pass or fail classification. Procedure: The administration of the MOST was incorporated into the regular licensing procedure. A basic performance test was administered to the applicants in order to screen them for the receipt of a learner's permit. Those candidates who wanted to take the regular operator's licensing test that same day were directed to the MOST examiners. While waiting their turn to take the MOST, applicants completed a "Motorcycle Riding Survey" designed to obtain pre-licensing information. The MOST was then administered to each applicant in turn. Regardless of their performance on the MOST, the applicants then took the regular test for a motorcycle operator's licence. Those applicants who wanted to take only the test for their learner's permit were administered the questionnaire and the MOST when they returned to take the licensing test at a later date. The follow-up interviews were conducted between June and October, 1978 such that the motorcyclists were contacted on or soon after the first anniversary of being licensed to ride a motorcycle. Although it was intended that as many as possible of the 637 licensed motorcyclists given the MOST would be interviewed, only 400 (63%) motorcyclists were interviewed. The major reason for sample attrition was the failure to make contact with the motorcyclists either by telephone or by going to their last known address (25%). The major reasons for non-contact were insufficient address information, failure to contact after six call-backs, or the subject had moved. Since a motorcyclist was defined as a rider who was currently licensed and had ridden 100 or more miles (160 km) in the last year, some of the motorcyclists contacted (9.0%) were found to be ineligible according to these criteria and were therefore excluded from the study. Only 2.8% of the sample refused to be interviewed. Following the screening questions, respondents were questioned about motorcycle ownership, riding experience (e.g. months riding), riding exposure (vehicle distance travelled in last year), training, accident involvement (e.g. seriousness, location, circumstances), traffic violations, general riding behaviour (e.g. number of trips carrying a passenger) and demographic characteristics.t Accident Criteria: The first criterion was whether or not the subject had been involved in any accidents during the last year while riding a motorcycle. The second criterion was whether the motorcyclist had been involved in a reportable accident where there was an injury or
tA copyof the questionnaireused for the follow-upinterviewscan be obtainedfrom the first author.
310
B.A. JONAHet al.
property damage greater than $200.00t These reportable accidents should have been reported to the police but they were not necessarily reported. In addition, driver records were searched for the first six months after the applicants received their licence to determine whether they had any recorded accidents which is the period when novice motorcyclists are most likely to be involved in accidents.$ These three criteria are referred to as total, reportable and recorded accidents respectively. RESULTS The age and sex classifications for the interviewed motorcyclists varied by less than 5% from the classification of the total sample of 637 motorcyclists who had taken the MOST. However, the pass rate on the MOST among those riders interviewed (30.2%) was significantly different, X2 (1)= 6.33, p <0.02, from the pass rate of those riders not interviewed (21.1%). Therefore, riders with higher scores on the test tended to be over-represented in the interviewed sample. Nevertheless, the relationship between MOST scores and accident likelihood could still be assessed using the interviewed motorcyclists. If the MOST is valid, then riders with high test scores should be less likely to have an accident than riders with relatively low test scores.
Bivariate analyses Total accidents: The respondents were asked whether or not they had had a motorcycle accident since obtaining their licence one year ago. This variable included minor accidents in which the rider was not injured nor was more than $200.00 damage done as a result of the accident. Overall, 26% of the respondents reported having a motorcycle accident during the previous year, Table 1 presents the percentage of motorcyclists who had an accident for various classifications of predictor variables including whether or not they had passed the MOST. Surprisingly, motorcyclists who had passed the test were 42% more likely to have had an accident than motorcyclists who had failed the test although the difference was only marginally significant (p < 0.06). Further examination of the differences between the accident/no accident groups presented in Table 1 showed that riders with the following characteristics were over-represented in the accident group: younger, lower levels of education, single, divorced or separated, riders of larger motorcycles, riders with greater exposure (i.e. distance ridden), riders with traffic violations, and riders reporting riding after alcohol use. Reportable accidents: As can be seen in Table 2, the proportion or riders who had a reportable accident (12%) was less than half the proportion of riders who had an accident of any kind. With respect to the MOST, no difference in likelihood of having a reportable accident was observed between motorcyclists grouped according to performance on the test. The effect of the other predictors on reportable accidents was the same as for the total accident data, although the effects were weaker due to the lower proportion of riders who had had reportable accidents. The exceptions were: (i) training, which indicated that the trained riders were significantly less likely to have had an accident; and (ii) marital status, which was not significantly related to having a reportable accident. Recorded accidents: Table 3 indicates that of the interviewed motorcyclists 3.3% had t~ motorcycle accident on their driving record. Those passing the MOST were significantly more likely to have a motorcycle accident on their driver record than those failing the MOST (p < 0.05). The only other significant finding in Table 3 was that motorcyclists reporting a traffic violation were more likely to have a motorcycle accident on their record. Of the original 637 tested motorcyclists, the driving records of 538 of them were identified. It was found that 7.2% tit should be noted that the property damage limit for reporting an accident to the police in Ontario was raised from $200.00to $400.00as of 1January, 1978.Since this changewouldapply to accidentsof highand low scorerson the MOST,it is not considered to be a threat to the validityof the present study. :~The driver record search was a two step process. Provincial records were searched first to determine whether the motorcyclist had had any accident. Since the provincial records did not indicate whether the subject was riding a motorcycle at the time of the accident, the accident file was accessed and if one of the vehicles involvedwas a motorcycle it was assumed that it was the subject who was the motorcyclist.This two step process resulted in some }oss in sample size (15%) and some inaccuracyin the incidenceof recorded accidents.
Predicting accident involvement with the Motorcycle Operator Skill Test
311
Table 1. Percentage of respondents having motorcycle accident in the last 12 months Classification
% Having Accident
Total N
M.O.S.T.
Pass Fail
32.2 22.6
Sex
Male Female
Own Motorcycle
Age
Predictor
Education
Total F a m i l y Income
Taken Training Course Marital
Engine
Status
Size
Distance ridden i n last 12 months
X2
p
121 279
3.64
.06
26.6 12.5
368 32
2.40
.13
Yes NO
26.4 19.6
349 51
0.74
n.s.
16-18 19-21 22-26 27-58
36.6 39.8 16.5 8.7
112 93
3~.3~
.001
S o m e lllgh School Completed High School/Some or Completed Comm. Coll. S o m e or C o m p l e t e d University
28.9
201
11.40
.01
30.6
III
11.6
86
<$I0,000 $I0,000-$14,999 $15,000-$19,999 $20,000-$24,999 $25,000. Yes NO
33.3 23.5 19.5 24.0 29.7 17.2 28.0
30 51 77 50 155 93 307
Single Married Divorced/ Separated
31.6 8.2 24.0
275 98 25
21.42
16.2 22.5 33".1 28.7
99 89 118 94
9.03
11.6 18.3 33.7 40.2
95 109 86 102
27.08
.001
~300cc 301-399cc 400-550cc 551, ~2000kms 2001-4800kms 4801-9000kms 9001~
91 I03
3.87
n.s.
3.84
.06
.001
.03
Riding after drinklng alcohol
Yes No
35.5 17.1
183 217
16.87
.001
}lad t r a f f i c v i o l a t i o n in last 12 m o n t h s
Yes No
42.4 ]6.0
144 256
32.30
.001
AAPVO[.13,No.
3t2
B.A. JONAHet aL
Table 2. Percen~ge of respondents having reportable motorcycle accident in last 12 months Classification
% Having Accident
Total N
M.O.S.T.
Pass Fail
14.9 I0.0
121 279
1.50
n.s.
Sex
Male Female
12.2 3.1
368 32
1.59
n.s.
Own M o t o r c y c l e
Yes No
12.3 5.9
349 51
1.24
n.s.
Age
16-18 19-21 22-26 27-58
15.2 17.2 9.9 3.9
112 93 91 103
10.54
.02
Education
Some lligh School C o m p l e t e d High School/Some Comm. Coll. Some or C o m p l e t e d University
10.68
.01
Predictor
Total F a m i l y Income
Taken Training Course Marital
Status
Engine Size
Distance Ridden in last 12 m o n t h s
12.4
201
17.1
Iii
2.3
86
KS10,000 $i0,000-$14,999 $15,000-$19,999 $20,000-$24,999 $25,000+ Yes No
20.0 7.8 9.1 10.0 12.9 4.3 13.7
30 51 77 50 155 93 307
Single Married Divorced/ Separated Widowed
13.8 5.1 8.0
275 98 25
0.0
1
3.0 14.6 14.4 13.8
&300cc 301-399cc 400-550cc 551+ ~2000kms 2001-4800kms 4801-9000kms 9001,
Xz
p
3.63
n.s.
5.29
.03
5.90
.12
99 89 118 94
9.30
.03
5.3 9.2 15.1 16.7
95 109 86 102
8.00
.05
.01
Riding After Drinking Alcohol
Yes No
16.9 6.9
183 217
8.85
Had T r a f f i c V i o l a t i o n in Last 12 m o n t h s
Yes No
19.4 7.0
144 256
12.76
.001
Predicting accident involvement with the Motorcycle Operator Skill Test
313
Table 3. Percentage of respondents having a motorcycle accident on their driving record a % Having Predictor
Classification
Tot~l
Accident
2
N~
×
p
M.O.S.T.
Pass Fail
6.6 1.8
121 278
4.76
.03
Sex
Male Female
3.5 0.0
367 32
0.32
n.s.
Yes No
3.7 0.0
348 51
.96
n.s.
Age
16-18 19-21 22-26 27-58
6.3 4.3 i.] 1.0
112 93 90 103
6.51
.09
Education
Some High School Completed High School/Some or C o m p l e t e d Com. Coll. S o m e or c o m p l e t e d University
4.5
201
3.87
n.s.
3.6
Ii0
0.0
86
<$10,000 $i0,000-~14,999 $15,000-$19,999 $20,000-$24,999 $25,000~ Yes No
6.7 0.0 1.3 4.0 4.5 0.0 4.2
30 51 76 50 155 93 306
Single Married Divorced/ Separated
4.4 1.0 0.0
Own
Motorcycle
Total Family Income
Taken Training Course Marital
Engine
Status
Size
~300ec 301-399cc 400-550cc 551~
Distance ridden in last 12 months
~2000kms 2001-4800kms 4801-9000kms 9001+
4.52
n.s.
2.85
.09
274 98 26
3.52
n.s.
1.0 4.5 4.2 3.2
99 88 118 94
2.41
n.s.
0.0 2.8 4.7 5.9
95 109 85 102
5.96
n.s.
Riding after drinking alcohol
Yes NO
4.9 1.9
183 216
2.06
n.s.
Had traffic v i o l a t i o n in last 12 m o n t h s
Yes No
6.3 1.6
144 255
4.99
.03
a D r i v i n g r e c o r d s of r e c e i v i n g licence. bRecord
for one
400
interviewed
interviewed
motorcyclists
motorcyclist
could
not
searched
be
for
found.
six m o n t h s
after
314
B.A. JONAH et al.
of the passing motorcyclists had a motorcycle accident on their record compared to 2.3% for the failing motorcyclists which was a significant difference, X2 (1) = 6.02, p < 0.022 Correlations: The use of pass/fail as a predictor with accident/no accident as the criterion resulted in the loss of information contained in the continuous scores. Consequently, the total scores on the MOST were correlated with the total number of accidents. Since scores were available for each exercise, the relationship between the level of various skills and accident involvement was also examined and these correlations are presented in Table 4. Although the total score was positively related to the total number of accidents, it appears that exercises 1, 6 and 9 accounted for more of the variance in accidents than the other exercises. These exercises measure starting on a hill, turning speed judgement and stopping quickly on a curve respectively. The better the rider had performed on these exercises, the greater the number of accidents in which he had been involved. Correlations with reportable accidents were generally weaker. Although the correlation between total score and reportable accidents was not significant, exercises 1 and 6 were positively correlated with the number of reportable accidents. While none of the correlations between total and exercise scores and recorded accidents were significant, they were generally in the same direction as the correlations for total and reportable accidents. These correlations reveal that relatively little variability in accident involvement can be accounted for by performance on MOST.
Multivariate analyses The bivariate analyses presented in Tables 1-3 indicate that there were several variables (age, exposure, engine size) in addition to MOST performance which were related to the accident criteria. Moreover, test performance was correlated with age, r (398) = - 0.30, p < 0.01 and sex, r ( 3 9 8 ) = - 0 . 2 8 , p <0.01. In order to control for the relationships among the Table 4. Co.elations between total and exercise scores and number of accidents Reportable Accidents a
Total Accidents a
Total
Score
Exercise 1
turn)
Exercise 3 (Accelerating
in curve)
Exercise
4 ( S l o w i n g in curve) 5
(Making
.~4-*
.08
.07
.i0
.13"*
.06
on hill)
Exercise 2 (Making s h a r p
Exercise
Recorded b Accidents
controlled
Exercise 6 (Turning speed
.00
Ol
.00
05
Ol
.03
07
06
.05
04
05
.02
12.
12-
.06
stop)
judgement) 09
- 03
.07
Exercise 8 ( E v a s i v e t~rn)
09
09
.05
Exercise
20**
06
.08
Exericse 7 (Emergency
9
(Emergency
abased
on
stop-straight)
400
stop-turning)
interviewed
bBased on 400 i n t e r v i e w e d were n o t i n t e r v i e w e d . *p
<. 05
**p
<.01
motorcyclists.
motorcyclists
and
138 m o t o r c y c l i s t s
who
%The characteristicsof the motorcycle accidents in which the motorcyclists had been involved were analyzed. The only significanteffect of skilllevel was the observation that riders who passed the test were less likely to have had their most recent accident at or near an intersection.
Predicting accident involvement with the Motorcycle Operator Skill Test
315
predictors, discriminant analyses (Cohen and Cohen, 1975) were performed on the accident involvement criteria. Exposure (i.e. distance ridden last year), age, sex, engine size and MOST continuous scores were entered into the discriminant function in a stepwise fashion. These analyses are summarized in Table 5. The function which discriminated between riders with and without any motorcycle accidents, rc = 0.33, was highly significant, X2 (5) = 45.36, p < 0.001 and correctly classified 63% of the motorcyclists. However, only age and exposure significantly contributed to the discrimination between the accident and accident free groups. The likelihood of having any accident decreased with age but increased with exposure. The function discriminating between motorcyclists with and without reportable accidents, rc =0.21, was also significant, ) 2 (5)= 17.04, p < 0.01 and correctly classified 62% of the motorcyclists. As with total accidents, only age and exposure made significant contributions to the discriminant function such that accidents decreased with age and increased with exposure. While the function which discriminated between motorcyclists with a motorcycle accident on their record and those whose record was free of motorcycle accidents, r~ = 0.15, only approached significance, X2 (5)= 9.13, p <0.11, it also indicated that exposure and age were the major predictors of recorded accidents. In summary, once age and exposure were controlled MOST performance did not predict accident involvement.t DISCUSSION
It was expected that motorcyclists with high scores on the Motorcycle Operator Skill Test would be less likely to have an accident while riding a motorcycle than motorcyclists with low scores on MOST. The results of the study indicate, however, that with the effects of age and
Table 5. Summary of discriminant analyses performed on motorcycle accident criteria a
Criterion
Predictor
F to E n t e r
Total accidents
Age
26.52***
.936
.766
Exposure
17.96"**
Reportable accidents
Recorded accidents
p p £
< < <
Lamda
.895
-.612
E n g i n e size
0.20
.890
-.071
Sex
1.89
.889
-.224
M.O.S.T.
0.29
.889
.082
Age
8,26***
.979
.650
Exposure
6.68***
.963
-.557
Engine
0.63
.959
-.206
Sex
1.61
.957
-.263
M.O.S.T.
0.09
,957
-.082
Exposure
5.39**
.986
-.703
size
Age
2.98*
.979
.549
Sex
0.74
.977
-.274
Engine size
0.07
.977
.080
M.O.S.T.
0.05
.977
-.084
aBased on 400 i n t e r v i e w e d *** ** *
Wilks
Standardized Discriminant Coefficients
motorcyclists.
.01 .05 .i0
tDiscriminant analyses were also performed on the accident criteria to determine whether test performance significantly interacted with any of the other predictors. No significant interactions were observed.
316
B.A. JONAHet al.
exposure statistically controlled, performance on the MOST was unrelated to accident likelihood. It would appear that the more skilled riders were more likely to have a motorcycle accident because they travelled farther on a motorcycle and they were younger. The results of the present study are not all that dissimilar from the results of the California evaluation of the MOST [Anderson, 1980]. Once temporal exposure was controlled, it was found that licensed riders in the group which was licensed on the basis of the MOST without remedial training had the same accident involvement as the riders licensed on the basis of the current California test. Only when riders received remedial training during the licensing process, was there any effect on accidents. This suggests that the value of the more difficult test per se may be to reduce the exposure of motorcyclists while they prepare to pass the licensing test. Neither the present study nor the California study can demonstrate that it is the unskilled riders--who are presumably more likely to have an accident--that are reducing their exposure. Indeed, the present study suggests that at least among licensed riders, exposure increases with level of skill as measured by the MOST. Further research should be conducted to determine whether the MOST is screening out the appropriate applicants for a motorcycle licence (i.e. those most likely to have an accident). The increase in motorcycle accident likelihood as a function of vehicle distance travelled is consistent with other research [Jonah et al., 1979]. The distance travelled in the last twelve months may be considered representative of exposure to risk and the experience of the rider. However, these variables should have opposing effects on accident likelihood such that increases in exposure should be related to increased accident likelihood and increases in experience should be related to decreased accident likelihood. For the case where the effects of exposure and experience balance each other, the variable distance travelled should have no relationship with accident likelihood. In the present study, accident likelihood increased as a function of distance travelled suggesting that distance travelled represents mainly exposure to risk at least during the first year of licensed riding. Age was found to be strongly related to accident involvement. This observation is consistent with other research on motorcycle accidents [e.g. Harano and Peck, 1968; Anderson, 1978; Jonah et al., 1980] and vehicle accidents in general [e.g. Pelz and Schuman, 1971]. The fact that age predicted accident likelihood indicates that while age itself is not a cause of motorcycle accidents, it probably reflects differences in experience and risk-taking behavior which may cause accidents. Younger riders may lack experience in such important areas as danger perception, perhaps because they have not been exposed to as many potentially dangerous situations as older riders and hence do not know what to look for. Younger riders also may be more willing to take risks while riding, because they fail to perceive the potentially hazardous consequences of such risk-taking behaviours. The present study tends to reject the belief that younger riders have a greater number of accidents because they have poorer motorcycle operational skills since it was found that younger riders actually performed better on the MOST than older riders. Other observations arising from the data include the finding that males were twice as likely to have had an accident as females although this difference was not significant. The lack of significant sex difference was most likely due to the relatively small proportion of female riders (8%) in the sample. Nevertheless, the effect of sex as a variable in this study is consistent with the findings of other research [Anderson, 1978; White, 1980] where the male riders were more likely to have had accidents. It was observed that riders who had been formally trained in a course were less likely to have had a reportable accident or a traffic violation. After controlling for exposure, age, engine size and test performance in a discriminant analysis, training was not related to accident likelihood. However, formally trained riders were less likely to have committed a traffic violation than informally trained riders, F (1,387)= 9.57, p <0.01. This is consistent with a recent study by Jonah, Dawson and Bragg (1980). This lower violation likelihood may have been the result of the formal training or it could be due to safer riders being more likely to take formal training. Motorcyclists who reported riding a motorcycle after drinking alcohol were more likely to have had an accident in the past year. It may be that drinking-and-riding constitutes another
Predicting accident involvement with the Motorcycle Operator Skill Test
317
aspect of a general riding style typified by greater risk-taking and the failure to appreciate the consequences of such behaviour. A significant positive relationship was observed between traffic violations, total accidents and reportable accidents. Although this observation is consistent with previous research [e.g. Peck et al., 1971; Jonah, Dawson and Bragg, 1980), it is clear that the common variance between accident likelihood and traffic violation likelihood is low (7%). Of methodological importance was the observation that similar effects of test performance and other predictors were observed on the self-reported accident data and on the driver record accident data even though there was a 12 month self-report period and only a 6 month driver record search. If the recorded accidents are prorated to the full 12 month period, we find that while 6.5% of the interviewed motorcyclists had an officially recorded accident, 11.5% reported that they had had an accident which should have been reported to the police and recorded on their driving record. This discrepancy clearly indicates that the use of official driving records will underestimate the frequency of motorcycle accidents, a finding consistent with previous research [Zylman, 1972; McGuire, 1973; Smith, 1976b]. Therefore, it is recommended that self-reported motorcycle accident data be used rather than officially recorded accidents whenever possible. Motorcyclists who failed the regular provincial licensing test were not included in the study. These riders who failed the current licensing test generally obtained low MOST scores. Moreover, the inability to obtain accident data, self-reported or recorded, for 15% of the original sample of 637 resulted in a further loss of riders with low scores from the sample. While the effect of the exclusion of these riders on the results of the study is not known, perhaps a curvilinear relationship between skill and accident involvement would have been observed such that riders with low or high scores on the MOST would have had greater accident likelihood than riders of moderate skill level. In summary, no significant relationship was found between skill as measured by the Motorcycle Operator Skill Test and accident likelihood once exposure and age were statistically controlled. However, this does not necessarily mean that skill is unrelated to accident likelihood. Although this off-road test appears to have little predictive validity, perhaps an on-road test like the Motorcyclist In-Traffic Test which assesses vehicle interaction skills (e.g. search behaviour) in addition to operational riding skills (e.g. speed and direction control) will prove to be successful in predicting accident likelihood. Therefore, research should be directed toward an evaluation of on-road skill tests such as the Motorcyclist In-Traffic Test, with respect to their ability to predict accident involvement. In addition, further research should be directed toward assessing which factors (e.g. danger perception, risk-taking) account for the greater involvement of younger motorcyclists in accidents. REFERENCES Anderson J., Improved motorcyclist licensing and testing project. Analytic Study. Research and Development Section. California Department of Motor Vehicles, July 1978. Anderson J., The effect of new motorcycle licensing programs and skills training on the driver records of original applicants. Proc. Int. Motorcycle Safety Conf., Washington, D.C. May, 1980. California Highway Patrol. A Motorcycle Accident Study. (DOT-HS-002560) Jan. 1968. Cohen J. and Cohen P., Applied Multiple Regression[Correlation Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, New Jersey, 1975. Ford J. L. and Anderson J., Improved Motorcyclist Licensing and Test Project. State of California, Department of Motor Vehicles, July 1978. Hall R. G., Evaluation of the effectiveness of motorcycle licensing programmes in other states. In Classified Licensing: Development of Procedures and Materials. Vol. lI: Licensing of Motorcycle Operators (Edited by P. F. Waller, S. S. Padgett, L. K. Li, H. A. Lowery and R. G. Hall). University of North Carolina, Highway Safety Research Center, 1976. Harano R. M. and R. C. Peck, The California Motorcycle Study: Driver and Accident Characteristics. Research Rep. 28, California Department of Motor Vehicles, July 1%8. Jonah B. A. and Dawson N. E., Validation of the motorcycle operator skill test. Accid. Anal. & Pre~. 11,163-171, 1979. Jonah B. A., Dawson N. E. and Bragg B. W. E., Evaluation of the Canada Safety Council's Motorcycle Training Program. Proc. Int. Motorcycle Safety Conf., Washington, D.C., May 1980. McGuire F. L., The nature of bias in otficial accident and violation records. J. AppL Psychology 57, 300-305, 1973. McKnight A. J. and Heywood H. B., Motorcycle task analysis. Motorcycle Safety Foundation, Linthicum, Maryland, Nov. 1974. McPherson K. and McKnight A. J., A task analytic approach to development of a Motorcycle Operator Licence Skill Test. Human Factors 15(4), 351-360, 1976.
318
B.A. JOYAHet al.
Organization for Economic Co-operation and Development. Safety of Two-Wheelers. Report by Road Research Group, March 1978. Peck R. C., McBride R. S. and Coppin R. S., The distribution and prediction of driver accident frequencies. Accid. Anal. & Prey. 2, 243-299, 1971. Pelz D. C. and Schuman S. H. Are young drivers really more dangerous after controlling for exposure and experience? J. Safety Res. 3, 38--79, 1971. Reiss M. L. and Haley J. A., Motorcycle Safety. Final report, Transportation Research Department, Airborne Instruments Laboratory, Deer Park, Long Island, New York, May 1%8. Smith D. I., Evaluation of the graded motorcycle licence scheme in Western Australia. In Motorcycle and Safety Syrup. Australian Road Research Board and Commonwealth Department of Transport, 1976(a). Smith D. I., Otficial driver records and self-reports as sources of accident and conviction data for research purposes. Accid. Anal. & Prey. 8,207-211, 1976 (b). Waller P. F., An Analysis of Motorcycle Accidents with Recommendations for Licensing and Operation. University of North Carolina, Chapel Hill, 1972. White J. G., Motorcycle Accident Study. TP 2673, Vehicle Systems, Road and Motor Vehicle Traffic Safety Branch, Transport Canada, July, 1980. Zylman R., Drivers' records: Are they a valid measure of driving behaviour? Accid. Anal. & Prey. 4, 333-349, 1972.
0~01-4575/81/040307-12502.00/0 O 1981 Pergamon Press Lid.
PREDICTING ACCIDENT INVOLVEMENT WITH THE MOTORCYCLE OPERATOR SKILL TEST BRIAN A. JONAH, NANCY E. DAWSONand BARRY W. E. BRAGGt Road and Motor Vehicle Traific Safety Branch, Transport Canada, Ottawa, Canada KIA ON5 (Received 7 August 1979; in revised[orm 22 August 1980)
Abstract--The present study was conducted to assess the predictive validity of the Motorcycle Operator Skill Test (MOST) developed by McPherson and McKnight (1976). It was anticipated that among licensed motorcyclists those scoring higher on this off-road skill test would be less likely to become involvedin an accident than motorcyclists scoring lower. A sample of motorcycle operator licence applicants was administered the MOST in addition to the regular licensing test. These motorcyclistswere contacted about a year later and queried about their riding experiences including motorcycleaccidents. Driver records were also searched for six months after the motorcyclistwas licensed. Although bivariate analyses indicated that high scorers on the MOST were more likely to have had an accident than low scorers, multivariateanalyses revealed no effect of test performance on subsequent accident involvement. However, accident likelihood was found to decrease as a function of age and increase as a function of vehicle distance travelled (i.e. exposure to risk). INTRODUCTION The boom in the popularity of motorcycles in the last 10-15 years has created concern among traffic safety experts, the focus of which has been a 125% increase in motorcycle accidents reported in the United States between 1965 and 1975 [Ford and Anderson, 1978] and a 250% increase observed in Canada [White 1980] during the same period. In order to deal with the motorcycle accident problem, researchers have attempted to determine the causes of motorcycle accidents. Several studies have implicated inexperience as a major causal factor [California Highway Patrol, 1968; Harano and Peck, 1968; Reiss and Haley, 1968; Waller, 1972; Organization for Economic Co-operation and Development, 1978]. Motorcycle safety experts [OECD, 1978] have suggested improved testing for licence applicants based on accident data and a comprehensive task analysis. The present study examined the predictive validity of one such test, the Motorcycle Operator Skill Test (MOST). Most jurisdictions in North America now require motorcyclists to obtain an operator's licence by passing tests of knowledge and riding skill. However, relatively little research has assessed the effectiveness of these motorcycle licensing programmes. Using time series analysis, Hall (1976) observed a decrease in the number of motorcycle accident fatalities per number of registered motorcycles during the year following the implementation of motorcycle operator licensing programmes in a number of American states. However, it was found that this reduction in fatalities continued after the first year of licensing programmes only in those states with both off-road and in-traffic skill tests and not in those states with only off-road tests. A study conducted in Western Australia failed to demonstrate that a graded motorcycle licensing programme had any effect on motorcycle accidents [Smith, 1976]. McPherson and McKnight [1976] have maintained that for a test to be a valid measure of critical on-street operating skills, its development should begin with a task analysis of the skills necessary for safe on-street operation of a motorcycle. Based on a task analysis performed by McKnight and Heywood [1974], McPherson and McKnight constructed the Motorcycle Operator Skill Test which consists of nine riding exercises of increasing difficulty. Initial test validation indicated a high correspondence between MOST scorers and the overall ratings of the rider's skill made by expert motorcycle riders. Recently, Jonah and Dawson [1979] have shown that performance on the MOST was better among younger, male, experienced (i.e. distance travelled) riders who had not received any formal training in motorcycling. The M O S T was used in an evaluation of an improved motorcyclist licensing project in California [Anderson, 1980]. Applicants for motorcycle operator licences were randomly ?Present address: Abt Associates of Canada, 23 Yonge Street, Toronto, Ontario, Canada M4P 1G2. 307
308
B.A. Jorq~H et al,
assigned to one of three conditions: (i) the current licensing system;(ii) the new licensing consisting of a new knowledge test and the MOST; or (iii) the new licensing system with the addition of remedial training on the MOST for applicants who failed the test. Analysis of official driver records indicated that applicants from either of the two groups which granted licences on the basis of the new test had significantly fewer accidents in the year following licensing than applicants licensed through the current system. The greatest difference in accident history was found between applicants in the new system with training and applicants in the current system such that there were 21% fewer accidents for the new system group. It should be noted that the riders' exposure in terms of vehicle miles travelled during the criterion period was not controlled in the California study so that it is possible that the lower accident involvement of riders in the new licensing programmes was due to their lower exposure. The fact that the pass rate was lower in the new systems and the fact that the applicants in the new systems took longer to obtain their licences suggest that applicants in the new systems had lower exposure than applicants in the current system. Cognizant of this problem with exposure, Anderson attempted to control it by using instruction permit issuance as a proxy measure of temporal exposure and covarying it in analysis of covariance performed on the accident involvement of licensed riders. While the improved testing group with remedial training was found to have fewer accidents than the current system group, the improved testing group without training was not significantly different from the control group. Anderson [1980] concluded that the major impact of the new licensing system per se (i.e. no remedial training) was to reduce the riding exposure of motorcyclists. It was also concluded that the training component of the program had an upgrading effect on the skill of motorcycle riders. However, since vehicle distance travelled was not controlled, it is still possible that the group with training had fewer accidents because they had lower exposure in terms of distance.'~ Although the California study may well have achieved the objective of reducing motorcycle accidents by making the licensing test more difficult, the basic question concerning the predictive validity of the MOST was not addressed in the California study. As pointed out by Anderson, "None of the results of this study provided support for the hypothesis that the content of the improved knowledge and skill tests, per se, were an important basis of program effectiveness" [1980, p. 398]. Furthermore, no attempt to assess the independent contribution of the MOST to the improvement of the licensing programme was made in the California study. Therefore, the purpose of the present study was to assess the predictive validity of the Motorcycle Operator Skill Test. If the test is valid, then motorcyclists with high scores on it should have fewer accidents than motorcyclists with low scores on it during a follow-up period. METHOD
The testing was conducted between June and August, 1977 at the Ontario Ministry of Transportation and Communications' vehicle operator licensing centre at Downsview, Ontario by two provincial licensing examiners who were trained to administer the MOST in a uniform and objective manner. About 12 months after licensing, follow-up interviews were conducted with most of the motorcyclists who had been tested. Driver record searches were also conducted to determine whether the motorcyclists had been involved in any motorcycle accidents. Subjects: The subjects consisted of 584 male and 53 female applicants for a motorcycle operator's license, with ages ranging from 16 to 57 years. The sample of 637 applicants includes only those individuals who passed the provincial licensing test which represnts 80% of licence applicants. Of this sample, 400 motorcyclists were interviewed during the follow-up phase and 538 driver records were obtained. Apparatus and Materials: A test course was set up at the licensing centre by painting exercise lines on the pavement in accordance with specifications outlined in the administrator's manual for the MOST [see Jonah and Dawson, 1979]. Scoring: The MOST is comprised of the following nine exercises each of which is scored separately by assigning penalty points to the operator for errors committed on the exercise: (1) starting on a hill, (2) making a sharp turn, (3) accelerating in a curve, (4) slowing in a curve, (5) ;Apparently an attempt is being made to control for exposure in terms of vehicle distance travelled in the California study.
Predictingaccidentinvolvementwith the MotorcycleOperatorSkill Test
309
making a controlled stop, (6) judging maximum safe turning speed, (7) emergency stopping in a straight line, (8) making a quick evasive turn and (9) emergency stopping in a turn. Basically, errors of three types are scored: path (i.e. straying outside of the lines); distance (i.e. stopping too late) and time/speed (i.e. travelling too slowly). Details for assigning penalty points can be found in the examiner's manual for the MOST. If the operator accumulated 13 or more penalty points, testing was terminated and no further exercises were attempted. The testing was also terminated if the operator dropped his motorcycle or ran into equipment. Termination of the testing meant that the operator failed the test. Therefore, one measure of performance was whether or not the operator failed the test. Once the data was collected by the examiner, a total score on the test was computed using the following formula: 9
Total Score = ~ [(TPi + 1) - PLi]Ai i=1
where TPi = total possible points that could be lost on exercise i, PLi = actual points lost on exercise i, Ai = attempted exercise (1 -- yes, 0 = no). The resultant scores ranged from 1 to 80 with a larger score representing a greater degree of skill. Note than an operator who terminated on exercise 3, for example, and therefore did not attempt exercise 4, received a score of 0 on exercise 4 and on subsequent exercises whereas an operator who attempted exercise 4 but lost all the points possible received a score of 1. This scoring system provided a continuous measure of skill in addition to the dichotomous pass or fail classification. Procedure: The administration of the MOST was incorporated into the regular licensing procedure. A basic performance test was administered to the applicants in order to screen them for the receipt of a learner's permit. Those candidates who wanted to take the regular operator's licensing test that same day were directed to the MOST examiners. While waiting their turn to take the MOST, applicants completed a "Motorcycle Riding Survey" designed to obtain pre-licensing information. The MOST was then administered to each applicant in turn. Regardless of their performance on the MOST, the applicants then took the regular test for a motorcycle operator's licence. Those applicants who wanted to take only the test for their learner's permit were administered the questionnaire and the MOST when they returned to take the licensing test at a later date. The follow-up interviews were conducted between June and October, 1978 such that the motorcyclists were contacted on or soon after the first anniversary of being licensed to ride a motorcycle. Although it was intended that as many as possible of the 637 licensed motorcyclists given the MOST would be interviewed, only 400 (63%) motorcyclists were interviewed. The major reason for sample attrition was the failure to make contact with the motorcyclists either by telephone or by going to their last known address (25%). The major reasons for non-contact were insufficient address information, failure to contact after six call-backs, or the subject had moved. Since a motorcyclist was defined as a rider who was currently licensed and had ridden 100 or more miles (160 km) in the last year, some of the motorcyclists contacted (9.0%) were found to be ineligible according to these criteria and were therefore excluded from the study. Only 2.8% of the sample refused to be interviewed. Following the screening questions, respondents were questioned about motorcycle ownership, riding experience (e.g. months riding), riding exposure (vehicle distance travelled in last year), training, accident involvement (e.g. seriousness, location, circumstances), traffic violations, general riding behaviour (e.g. number of trips carrying a passenger) and demographic characteristics.t Accident Criteria: The first criterion was whether or not the subject had been involved in any accidents during the last year while riding a motorcycle. The second criterion was whether the motorcyclist had been involved in a reportable accident where there was an injury or
tA copyof the questionnaireused for the follow-upinterviewscan be obtainedfrom the first author.
310
B.A. JONAHet al.
property damage greater than $200.00t These reportable accidents should have been reported to the police but they were not necessarily reported. In addition, driver records were searched for the first six months after the applicants received their licence to determine whether they had any recorded accidents which is the period when novice motorcyclists are most likely to be involved in accidents.$ These three criteria are referred to as total, reportable and recorded accidents respectively. RESULTS The age and sex classifications for the interviewed motorcyclists varied by less than 5% from the classification of the total sample of 637 motorcyclists who had taken the MOST. However, the pass rate on the MOST among those riders interviewed (30.2%) was significantly different, X2 (1)= 6.33, p <0.02, from the pass rate of those riders not interviewed (21.1%). Therefore, riders with higher scores on the test tended to be over-represented in the interviewed sample. Nevertheless, the relationship between MOST scores and accident likelihood could still be assessed using the interviewed motorcyclists. If the MOST is valid, then riders with high test scores should be less likely to have an accident than riders with relatively low test scores.
Bivariate analyses Total accidents: The respondents were asked whether or not they had had a motorcycle accident since obtaining their licence one year ago. This variable included minor accidents in which the rider was not injured nor was more than $200.00 damage done as a result of the accident. Overall, 26% of the respondents reported having a motorcycle accident during the previous year, Table 1 presents the percentage of motorcyclists who had an accident for various classifications of predictor variables including whether or not they had passed the MOST. Surprisingly, motorcyclists who had passed the test were 42% more likely to have had an accident than motorcyclists who had failed the test although the difference was only marginally significant (p < 0.06). Further examination of the differences between the accident/no accident groups presented in Table 1 showed that riders with the following characteristics were over-represented in the accident group: younger, lower levels of education, single, divorced or separated, riders of larger motorcycles, riders with greater exposure (i.e. distance ridden), riders with traffic violations, and riders reporting riding after alcohol use. Reportable accidents: As can be seen in Table 2, the proportion or riders who had a reportable accident (12%) was less than half the proportion of riders who had an accident of any kind. With respect to the MOST, no difference in likelihood of having a reportable accident was observed between motorcyclists grouped according to performance on the test. The effect of the other predictors on reportable accidents was the same as for the total accident data, although the effects were weaker due to the lower proportion of riders who had had reportable accidents. The exceptions were: (i) training, which indicated that the trained riders were significantly less likely to have had an accident; and (ii) marital status, which was not significantly related to having a reportable accident. Recorded accidents: Table 3 indicates that of the interviewed motorcyclists 3.3% had t~ motorcycle accident on their driving record. Those passing the MOST were significantly more likely to have a motorcycle accident on their driver record than those failing the MOST (p < 0.05). The only other significant finding in Table 3 was that motorcyclists reporting a traffic violation were more likely to have a motorcycle accident on their record. Of the original 637 tested motorcyclists, the driving records of 538 of them were identified. It was found that 7.2% tit should be noted that the property damage limit for reporting an accident to the police in Ontario was raised from $200.00to $400.00as of 1January, 1978.Since this changewouldapply to accidentsof highand low scorerson the MOST,it is not considered to be a threat to the validityof the present study. :~The driver record search was a two step process. Provincial records were searched first to determine whether the motorcyclist had had any accident. Since the provincial records did not indicate whether the subject was riding a motorcycle at the time of the accident, the accident file was accessed and if one of the vehicles involvedwas a motorcycle it was assumed that it was the subject who was the motorcyclist.This two step process resulted in some }oss in sample size (15%) and some inaccuracyin the incidenceof recorded accidents.
Predicting accident involvement with the Motorcycle Operator Skill Test
311
Table 1. Percentage of respondents having motorcycle accident in the last 12 months Classification
% Having Accident
Total N
M.O.S.T.
Pass Fail
32.2 22.6
Sex
Male Female
Own Motorcycle
Age
Predictor
Education
Total F a m i l y Income
Taken Training Course Marital
Engine
Status
Size
Distance ridden i n last 12 months
X2
p
121 279
3.64
.06
26.6 12.5
368 32
2.40
.13
Yes NO
26.4 19.6
349 51
0.74
n.s.
16-18 19-21 22-26 27-58
36.6 39.8 16.5 8.7
112 93
3~.3~
.001
S o m e lllgh School Completed High School/Some or Completed Comm. Coll. S o m e or C o m p l e t e d University
28.9
201
11.40
.01
30.6
III
11.6
86
<$I0,000 $I0,000-$14,999 $15,000-$19,999 $20,000-$24,999 $25,000. Yes NO
33.3 23.5 19.5 24.0 29.7 17.2 28.0
30 51 77 50 155 93 307
Single Married Divorced/ Separated
31.6 8.2 24.0
275 98 25
21.42
16.2 22.5 33".1 28.7
99 89 118 94
9.03
11.6 18.3 33.7 40.2
95 109 86 102
27.08
.001
~300cc 301-399cc 400-550cc 551, ~2000kms 2001-4800kms 4801-9000kms 9001~
91 I03
3.87
n.s.
3.84
.06
.001
.03
Riding after drinklng alcohol
Yes No
35.5 17.1
183 217
16.87
.001
}lad t r a f f i c v i o l a t i o n in last 12 m o n t h s
Yes No
42.4 ]6.0
144 256
32.30
.001
AAPVO[.13,No.
3t2
B.A. JONAHet aL
Table 2. Percen~ge of respondents having reportable motorcycle accident in last 12 months Classification
% Having Accident
Total N
M.O.S.T.
Pass Fail
14.9 I0.0
121 279
1.50
n.s.
Sex
Male Female
12.2 3.1
368 32
1.59
n.s.
Own M o t o r c y c l e
Yes No
12.3 5.9
349 51
1.24
n.s.
Age
16-18 19-21 22-26 27-58
15.2 17.2 9.9 3.9
112 93 91 103
10.54
.02
Education
Some lligh School C o m p l e t e d High School/Some Comm. Coll. Some or C o m p l e t e d University
10.68
.01
Predictor
Total F a m i l y Income
Taken Training Course Marital
Status
Engine Size
Distance Ridden in last 12 m o n t h s
12.4
201
17.1
Iii
2.3
86
KS10,000 $i0,000-$14,999 $15,000-$19,999 $20,000-$24,999 $25,000+ Yes No
20.0 7.8 9.1 10.0 12.9 4.3 13.7
30 51 77 50 155 93 307
Single Married Divorced/ Separated Widowed
13.8 5.1 8.0
275 98 25
0.0
1
3.0 14.6 14.4 13.8
&300cc 301-399cc 400-550cc 551+ ~2000kms 2001-4800kms 4801-9000kms 9001,
Xz
p
3.63
n.s.
5.29
.03
5.90
.12
99 89 118 94
9.30
.03
5.3 9.2 15.1 16.7
95 109 86 102
8.00
.05
.01
Riding After Drinking Alcohol
Yes No
16.9 6.9
183 217
8.85
Had T r a f f i c V i o l a t i o n in Last 12 m o n t h s
Yes No
19.4 7.0
144 256
12.76
.001
Predicting accident involvement with the Motorcycle Operator Skill Test
313
Table 3. Percentage of respondents having a motorcycle accident on their driving record a % Having Predictor
Classification
Tot~l
Accident
2
N~
×
p
M.O.S.T.
Pass Fail
6.6 1.8
121 278
4.76
.03
Sex
Male Female
3.5 0.0
367 32
0.32
n.s.
Yes No
3.7 0.0
348 51
.96
n.s.
Age
16-18 19-21 22-26 27-58
6.3 4.3 i.] 1.0
112 93 90 103
6.51
.09
Education
Some High School Completed High School/Some or C o m p l e t e d Com. Coll. S o m e or c o m p l e t e d University
4.5
201
3.87
n.s.
3.6
Ii0
0.0
86
<$10,000 $i0,000-~14,999 $15,000-$19,999 $20,000-$24,999 $25,000~ Yes No
6.7 0.0 1.3 4.0 4.5 0.0 4.2
30 51 76 50 155 93 306
Single Married Divorced/ Separated
4.4 1.0 0.0
Own
Motorcycle
Total Family Income
Taken Training Course Marital
Engine
Status
Size
~300ec 301-399cc 400-550cc 551~
Distance ridden in last 12 months
~2000kms 2001-4800kms 4801-9000kms 9001+
4.52
n.s.
2.85
.09
274 98 26
3.52
n.s.
1.0 4.5 4.2 3.2
99 88 118 94
2.41
n.s.
0.0 2.8 4.7 5.9
95 109 85 102
5.96
n.s.
Riding after drinking alcohol
Yes NO
4.9 1.9
183 216
2.06
n.s.
Had traffic v i o l a t i o n in last 12 m o n t h s
Yes No
6.3 1.6
144 255
4.99
.03
a D r i v i n g r e c o r d s of r e c e i v i n g licence. bRecord
for one
400
interviewed
interviewed
motorcyclists
motorcyclist
could
not
searched
be
for
found.
six m o n t h s
after
314
B.A. JONAH et al.
of the passing motorcyclists had a motorcycle accident on their record compared to 2.3% for the failing motorcyclists which was a significant difference, X2 (1) = 6.02, p < 0.022 Correlations: The use of pass/fail as a predictor with accident/no accident as the criterion resulted in the loss of information contained in the continuous scores. Consequently, the total scores on the MOST were correlated with the total number of accidents. Since scores were available for each exercise, the relationship between the level of various skills and accident involvement was also examined and these correlations are presented in Table 4. Although the total score was positively related to the total number of accidents, it appears that exercises 1, 6 and 9 accounted for more of the variance in accidents than the other exercises. These exercises measure starting on a hill, turning speed judgement and stopping quickly on a curve respectively. The better the rider had performed on these exercises, the greater the number of accidents in which he had been involved. Correlations with reportable accidents were generally weaker. Although the correlation between total score and reportable accidents was not significant, exercises 1 and 6 were positively correlated with the number of reportable accidents. While none of the correlations between total and exercise scores and recorded accidents were significant, they were generally in the same direction as the correlations for total and reportable accidents. These correlations reveal that relatively little variability in accident involvement can be accounted for by performance on MOST.
Multivariate analyses The bivariate analyses presented in Tables 1-3 indicate that there were several variables (age, exposure, engine size) in addition to MOST performance which were related to the accident criteria. Moreover, test performance was correlated with age, r (398) = - 0.30, p < 0.01 and sex, r ( 3 9 8 ) = - 0 . 2 8 , p <0.01. In order to control for the relationships among the Table 4. Co.elations between total and exercise scores and number of accidents Reportable Accidents a
Total Accidents a
Total
Score
Exercise 1
turn)
Exercise 3 (Accelerating
in curve)
Exercise
4 ( S l o w i n g in curve) 5
(Making
.~4-*
.08
.07
.i0
.13"*
.06
on hill)
Exercise 2 (Making s h a r p
Exercise
Recorded b Accidents
controlled
Exercise 6 (Turning speed
.00
Ol
.00
05
Ol
.03
07
06
.05
04
05
.02
12.
12-
.06
stop)
judgement) 09
- 03
.07
Exercise 8 ( E v a s i v e t~rn)
09
09
.05
Exercise
20**
06
.08
Exericse 7 (Emergency
9
(Emergency
abased
on
stop-straight)
400
stop-turning)
interviewed
bBased on 400 i n t e r v i e w e d were n o t i n t e r v i e w e d . *p
<. 05
**p
<.01
motorcyclists.
motorcyclists
and
138 m o t o r c y c l i s t s
who
%The characteristicsof the motorcycle accidents in which the motorcyclists had been involved were analyzed. The only significanteffect of skilllevel was the observation that riders who passed the test were less likely to have had their most recent accident at or near an intersection.
Predicting accident involvement with the Motorcycle Operator Skill Test
315
predictors, discriminant analyses (Cohen and Cohen, 1975) were performed on the accident involvement criteria. Exposure (i.e. distance ridden last year), age, sex, engine size and MOST continuous scores were entered into the discriminant function in a stepwise fashion. These analyses are summarized in Table 5. The function which discriminated between riders with and without any motorcycle accidents, rc = 0.33, was highly significant, X2 (5) = 45.36, p < 0.001 and correctly classified 63% of the motorcyclists. However, only age and exposure significantly contributed to the discrimination between the accident and accident free groups. The likelihood of having any accident decreased with age but increased with exposure. The function discriminating between motorcyclists with and without reportable accidents, rc =0.21, was also significant, ) 2 (5)= 17.04, p < 0.01 and correctly classified 62% of the motorcyclists. As with total accidents, only age and exposure made significant contributions to the discriminant function such that accidents decreased with age and increased with exposure. While the function which discriminated between motorcyclists with a motorcycle accident on their record and those whose record was free of motorcycle accidents, r~ = 0.15, only approached significance, X2 (5)= 9.13, p <0.11, it also indicated that exposure and age were the major predictors of recorded accidents. In summary, once age and exposure were controlled MOST performance did not predict accident involvement.t DISCUSSION
It was expected that motorcyclists with high scores on the Motorcycle Operator Skill Test would be less likely to have an accident while riding a motorcycle than motorcyclists with low scores on MOST. The results of the study indicate, however, that with the effects of age and
Table 5. Summary of discriminant analyses performed on motorcycle accident criteria a
Criterion
Predictor
F to E n t e r
Total accidents
Age
26.52***
.936
.766
Exposure
17.96"**
Reportable accidents
Recorded accidents
p p £
< < <
Lamda
.895
-.612
E n g i n e size
0.20
.890
-.071
Sex
1.89
.889
-.224
M.O.S.T.
0.29
.889
.082
Age
8,26***
.979
.650
Exposure
6.68***
.963
-.557
Engine
0.63
.959
-.206
Sex
1.61
.957
-.263
M.O.S.T.
0.09
,957
-.082
Exposure
5.39**
.986
-.703
size
Age
2.98*
.979
.549
Sex
0.74
.977
-.274
Engine size
0.07
.977
.080
M.O.S.T.
0.05
.977
-.084
aBased on 400 i n t e r v i e w e d *** ** *
Wilks
Standardized Discriminant Coefficients
motorcyclists.
.01 .05 .i0
tDiscriminant analyses were also performed on the accident criteria to determine whether test performance significantly interacted with any of the other predictors. No significant interactions were observed.
316
B.A. JONAHet al.
exposure statistically controlled, performance on the MOST was unrelated to accident likelihood. It would appear that the more skilled riders were more likely to have a motorcycle accident because they travelled farther on a motorcycle and they were younger. The results of the present study are not all that dissimilar from the results of the California evaluation of the MOST [Anderson, 1980]. Once temporal exposure was controlled, it was found that licensed riders in the group which was licensed on the basis of the MOST without remedial training had the same accident involvement as the riders licensed on the basis of the current California test. Only when riders received remedial training during the licensing process, was there any effect on accidents. This suggests that the value of the more difficult test per se may be to reduce the exposure of motorcyclists while they prepare to pass the licensing test. Neither the present study nor the California study can demonstrate that it is the unskilled riders--who are presumably more likely to have an accident--that are reducing their exposure. Indeed, the present study suggests that at least among licensed riders, exposure increases with level of skill as measured by the MOST. Further research should be conducted to determine whether the MOST is screening out the appropriate applicants for a motorcycle licence (i.e. those most likely to have an accident). The increase in motorcycle accident likelihood as a function of vehicle distance travelled is consistent with other research [Jonah et al., 1979]. The distance travelled in the last twelve months may be considered representative of exposure to risk and the experience of the rider. However, these variables should have opposing effects on accident likelihood such that increases in exposure should be related to increased accident likelihood and increases in experience should be related to decreased accident likelihood. For the case where the effects of exposure and experience balance each other, the variable distance travelled should have no relationship with accident likelihood. In the present study, accident likelihood increased as a function of distance travelled suggesting that distance travelled represents mainly exposure to risk at least during the first year of licensed riding. Age was found to be strongly related to accident involvement. This observation is consistent with other research on motorcycle accidents [e.g. Harano and Peck, 1968; Anderson, 1978; Jonah et al., 1980] and vehicle accidents in general [e.g. Pelz and Schuman, 1971]. The fact that age predicted accident likelihood indicates that while age itself is not a cause of motorcycle accidents, it probably reflects differences in experience and risk-taking behavior which may cause accidents. Younger riders may lack experience in such important areas as danger perception, perhaps because they have not been exposed to as many potentially dangerous situations as older riders and hence do not know what to look for. Younger riders also may be more willing to take risks while riding, because they fail to perceive the potentially hazardous consequences of such risk-taking behaviours. The present study tends to reject the belief that younger riders have a greater number of accidents because they have poorer motorcycle operational skills since it was found that younger riders actually performed better on the MOST than older riders. Other observations arising from the data include the finding that males were twice as likely to have had an accident as females although this difference was not significant. The lack of significant sex difference was most likely due to the relatively small proportion of female riders (8%) in the sample. Nevertheless, the effect of sex as a variable in this study is consistent with the findings of other research [Anderson, 1978; White, 1980] where the male riders were more likely to have had accidents. It was observed that riders who had been formally trained in a course were less likely to have had a reportable accident or a traffic violation. After controlling for exposure, age, engine size and test performance in a discriminant analysis, training was not related to accident likelihood. However, formally trained riders were less likely to have committed a traffic violation than informally trained riders, F (1,387)= 9.57, p <0.01. This is consistent with a recent study by Jonah, Dawson and Bragg (1980). This lower violation likelihood may have been the result of the formal training or it could be due to safer riders being more likely to take formal training. Motorcyclists who reported riding a motorcycle after drinking alcohol were more likely to have had an accident in the past year. It may be that drinking-and-riding constitutes another
Predicting accident involvement with the Motorcycle Operator Skill Test
317
aspect of a general riding style typified by greater risk-taking and the failure to appreciate the consequences of such behaviour. A significant positive relationship was observed between traffic violations, total accidents and reportable accidents. Although this observation is consistent with previous research [e.g. Peck et al., 1971; Jonah, Dawson and Bragg, 1980), it is clear that the common variance between accident likelihood and traffic violation likelihood is low (7%). Of methodological importance was the observation that similar effects of test performance and other predictors were observed on the self-reported accident data and on the driver record accident data even though there was a 12 month self-report period and only a 6 month driver record search. If the recorded accidents are prorated to the full 12 month period, we find that while 6.5% of the interviewed motorcyclists had an officially recorded accident, 11.5% reported that they had had an accident which should have been reported to the police and recorded on their driving record. This discrepancy clearly indicates that the use of official driving records will underestimate the frequency of motorcycle accidents, a finding consistent with previous research [Zylman, 1972; McGuire, 1973; Smith, 1976b]. Therefore, it is recommended that self-reported motorcycle accident data be used rather than officially recorded accidents whenever possible. Motorcyclists who failed the regular provincial licensing test were not included in the study. These riders who failed the current licensing test generally obtained low MOST scores. Moreover, the inability to obtain accident data, self-reported or recorded, for 15% of the original sample of 637 resulted in a further loss of riders with low scores from the sample. While the effect of the exclusion of these riders on the results of the study is not known, perhaps a curvilinear relationship between skill and accident involvement would have been observed such that riders with low or high scores on the MOST would have had greater accident likelihood than riders of moderate skill level. In summary, no significant relationship was found between skill as measured by the Motorcycle Operator Skill Test and accident likelihood once exposure and age were statistically controlled. However, this does not necessarily mean that skill is unrelated to accident likelihood. Although this off-road test appears to have little predictive validity, perhaps an on-road test like the Motorcyclist In-Traffic Test which assesses vehicle interaction skills (e.g. search behaviour) in addition to operational riding skills (e.g. speed and direction control) will prove to be successful in predicting accident likelihood. Therefore, research should be directed toward an evaluation of on-road skill tests such as the Motorcyclist In-Traffic Test, with respect to their ability to predict accident involvement. In addition, further research should be directed toward assessing which factors (e.g. danger perception, risk-taking) account for the greater involvement of younger motorcyclists in accidents. REFERENCES Anderson J., Improved motorcyclist licensing and testing project. Analytic Study. Research and Development Section. California Department of Motor Vehicles, July 1978. Anderson J., The effect of new motorcycle licensing programs and skills training on the driver records of original applicants. Proc. Int. Motorcycle Safety Conf., Washington, D.C. May, 1980. California Highway Patrol. A Motorcycle Accident Study. (DOT-HS-002560) Jan. 1968. Cohen J. and Cohen P., Applied Multiple Regression[Correlation Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, New Jersey, 1975. Ford J. L. and Anderson J., Improved Motorcyclist Licensing and Test Project. State of California, Department of Motor Vehicles, July 1978. Hall R. G., Evaluation of the effectiveness of motorcycle licensing programmes in other states. In Classified Licensing: Development of Procedures and Materials. Vol. lI: Licensing of Motorcycle Operators (Edited by P. F. Waller, S. S. Padgett, L. K. Li, H. A. Lowery and R. G. Hall). University of North Carolina, Highway Safety Research Center, 1976. Harano R. M. and R. C. Peck, The California Motorcycle Study: Driver and Accident Characteristics. Research Rep. 28, California Department of Motor Vehicles, July 1%8. Jonah B. A. and Dawson N. E., Validation of the motorcycle operator skill test. Accid. Anal. & Pre~. 11,163-171, 1979. Jonah B. A., Dawson N. E. and Bragg B. W. E., Evaluation of the Canada Safety Council's Motorcycle Training Program. Proc. Int. Motorcycle Safety Conf., Washington, D.C., May 1980. McGuire F. L., The nature of bias in otficial accident and violation records. J. AppL Psychology 57, 300-305, 1973. McKnight A. J. and Heywood H. B., Motorcycle task analysis. Motorcycle Safety Foundation, Linthicum, Maryland, Nov. 1974. McPherson K. and McKnight A. J., A task analytic approach to development of a Motorcycle Operator Licence Skill Test. Human Factors 15(4), 351-360, 1976.
318
B.A. JOYAHet al.
Organization for Economic Co-operation and Development. Safety of Two-Wheelers. Report by Road Research Group, March 1978. Peck R. C., McBride R. S. and Coppin R. S., The distribution and prediction of driver accident frequencies. Accid. Anal. & Prey. 2, 243-299, 1971. Pelz D. C. and Schuman S. H. Are young drivers really more dangerous after controlling for exposure and experience? J. Safety Res. 3, 38--79, 1971. Reiss M. L. and Haley J. A., Motorcycle Safety. Final report, Transportation Research Department, Airborne Instruments Laboratory, Deer Park, Long Island, New York, May 1%8. Smith D. I., Evaluation of the graded motorcycle licence scheme in Western Australia. In Motorcycle and Safety Syrup. Australian Road Research Board and Commonwealth Department of Transport, 1976(a). Smith D. I., Otficial driver records and self-reports as sources of accident and conviction data for research purposes. Accid. Anal. & Prey. 8,207-211, 1976 (b). Waller P. F., An Analysis of Motorcycle Accidents with Recommendations for Licensing and Operation. University of North Carolina, Chapel Hill, 1972. White J. G., Motorcycle Accident Study. TP 2673, Vehicle Systems, Road and Motor Vehicle Traffic Safety Branch, Transport Canada, July, 1980. Zylman R., Drivers' records: Are they a valid measure of driving behaviour? Accid. Anal. & Prey. 4, 333-349, 1972.