- Email: [email protected]
Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year
JSR-01435; No of Pages 6 Journal of Safety Research xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
Journal of Safety Research journal homepage: www.elsevier.com/locate/jsr
4Q5 5 6
a
8 7
a r t i c l e
Q8 9 10 11 12 13 18 17 16 15 14
Article history: Received 31 January 2017 Received in revised form 17 May 2017 Accepted 9 October 2017 Available online xxxx
Virginia Tech Transportation Institute, United States Motorcycle Safety Foundation, United States
i n f o
D
b
19 20
T
23
1. Problem
27 28
Motorcycle crashes and fatalities are on the rise, and multiple transportation agencies have called for research into this issue. The motorcyclist fatality rate per 100,000 registered motorcycles in 2014 was six times that of the fatality rate for drivers of passenger cars (per 100,000 registered vehicles), and the fatality rate per motorcycle miles traveled was 27 times that of automobile miles traveled (National Highway Traffic Safety Administration, 2016). Many public and private organizations are interested in identifying causes of these crashes and related fatalities and injuries. One factor could be how capable riders are of estimating their own riding experience. The importance of “SelfAwareness” (including awareness of one's own skill level) is emphasized in the Motorcycle Safety Foundation (MSF) Basic Rider Course (2014). Further, in targeting training or considering survey responses, it is necessary to understand how indicative self-reported mileage is of actual future mileage. In addition, understanding the relationship between motorcyclists' estimated mileage and actual mileage is valuable because there continues to be some difficulty in obtaining accurate estimates of exposure (motorcyclist mileage) for measures such as crash and injury statistics as well as in efforts involving funding allocations, infrastructure planning, and financial forecasting (Lyon, Persaud, & Himes, 2017; Middleton et al., 2013). Exposure estimates have been obtained via various methods, but fall short in terms of likely accuracy. Motorcycle registration does not provide a complete tally of riders due to unlicensed riders or licensing
37 38 39 40 41 42 43 44 45 46 47 48 49 50
E
R
R
N C O
35 36
U
33 34
C
Q7 26
31 32
© 2017 National Safety Council and Elsevier Ltd. All rights reserved.
E
24 22 21 25
29 30
F
Vicki Williams, a,⁎ Shane McLaughlin, a Robert McCall, a Tim Buche b
O
Q43Q3
R O
2
Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year
P
1Q2
⁎ Corresponding author. E-mail addresses: [email protected] (V. Williams), [email protected] (S. McLaughlin), [email protected] (R. McCall), [email protected] (T. Buche).
issued to individuals other than the actual rider. As indicated by the U.S. Department of Transportation (USDOT, 2015), the Motorcyclist Fatality Rate includes the “statistical issue” that the Federal Highway Administration (FHWA) likely underestimates the number of motorcycles on the road each year (supported by the finding from organizations such as the Motorcycle Industry Council that not all riders register their motorcycles). Annual motorcycle inspections rely on accurate odometer readings and faithful inspection scheduling. Observational and roadway detector equipment recordings of motorcyclist traffic flow present multiple difficulties related to collection protocol, location and timing choice, and sensor accuracy. Middleton et al. (2013) discuss some of these shortcomings, and offer various guidelines for calculating the accuracy of current methods used to report motorcycle traffic data. The authors provide detailed research methods and recommendations in terms of equipment and collection methods, and note that there are ongoing efforts to improve motorcycle traffic data. Their paper also presents the possibility of supplementing travel data through motorcyclist surveys, such as the National Household Traffic Survey (NHTS), origin and destination (O & D) surveys, and driver exposure surveys. If this additional source of motorcyclist mileage information (in the form of selfreported mileage) appears to be fairly accurate, it could be used to check or supplement mileage estimations collected through other methods. As the first large-scale naturalistic instrumented motorcycle study to collect real-time mileage and self-reported mileage estimates, the MSF 100 study provides a unique data set from which to draw inferences about the characteristics of motorcyclist self-reported mileage and its application. Investigation of the data found in the study provides knowledge useful in answering multiple questions. Is collection of motorcyclists' self-reported mileage a useful method of checking or supplementing motorcycle travel data (which is in need of improved
https://doi.org/10.1016/j.jsr.2017.10.004 0022-4375/© 2017 National Safety Council and Elsevier Ltd. All rights reserved.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123
C
107 108
E
105 106
R
103 104
R
101 102
O
99 100
C
97 98
N
95 96
U
90 91
F
The MSF 100 Motorcyclists Naturalistic Study was sponsored by the Motorcycle Safety Foundation and conducted by the Virginia Tech Transportation Institute (VTTI) to collect real-world riding data from riders in their natural day-to-day experiences while riding their own motorcycles. Individual rider participation in the study ranged from two months to two years. These 100 riders resided in California, Florida, Virginia, and Arizona, and both video data and motorcycle kinematic data were collected for every trip (defined as the time between keyon and key-off, during which the rider travels from one destination to another). These data were collected via unobtrusive instrumentation of a VTTI-developed data acquisition system (DAS) on each motorcycle, which continuously recorded five video views of the rider and the surrounding environment as well as motorcycle data such as GPS (Global Positioning System), acceleration, gyro, and brake activation. The full set of collected data incorporated over 366,000 miles of riding. Participants also completed various questionnaires prior to equipment installation, including riding exposure surveys, indicating riding habits to date such as annual mileage. This paper explores motorcyclists' self-reported annual riding mileage and the actual amount of riding done within the study. For this evaluation, 91 riders (those who had been riding for at least one year before study enrollment and reported an annual mileage for that year) were considered. Self-reported annual mileage was recorded immediately preceding study participation, directly from the survey question “Approximately how many miles have you ridden a motorcycle on public roads in the past 12 months?” This sample of 71 males and 20 females were of various age groups and represented all three types of motorcycles, as indicated in Fig. 1. Two approaches were used for determining the actual miles ridden. For 78 cases, starting and ending odometer readings were used. These
88 89
O
94
87
R O
2. Method
85 86
P
93
83 84
readings were collected by technicians during equipment installation, and again during de-installation (at the close of participation). During the incorporation of the odometer readings for the data analysis phase, 13 cases were observed with recorded odometer reading that were missing or suspect (did not reasonably pair with the corresponding reading). In those 13 cases, integration of GPS-based speed and time data were used to calculate the distance traveled. Missing GPS values within a trip, including the period of time between DAS start and GPS signal acquisition, were replaced with the mean speed of that same trip. In the event that an entire trip was missing GPS speed, then the mean speed across all trips taken by that participant was used in conjunction with the trip duration to estimate mileage. However, the GPS dropout rate was not such that major adjustments were necessary, and thus total mileage estimates were not significantly affected. In general, the riders were not surrounded by high rise buildings or in weather that would interfere with GPS signal, and cellular signals were not used (the GPS was onboard with an external antenna). All riding data (including odometer mileage and GPS-based calculations) were compared within riders to ensure the most accurate actual mileage was being used in the final analysis. The mileage for each rider was then translated to an adjusted annual riding mileage based on their study participation duration. At the beginning of the study, riders completed a survey which included a question about the number of months they tend to ride each year. Because the accuracy of selfreported data is not guaranteed, nor is it necessarily predictive of how many months overall they would have ridden during the study year, annualized mileage calculations were not based on any attempts at adjustment for riding season. However, each case of a reported abbreviated riding season was investigated to consider the potential effect of the riding season on annualized mileage calculations. The majority of riders who reported an abbreviated riding season participated for around a year and/or recorded very low mileage, so the riding season was either taken into account or was likely altered by a relatively small amount. Those riders falling outside of this category tended to ride nearly yearround, so any over- or under-estimation would also be relatively small. During initial quality control efforts related to this MSF 100 study, one aspect of ensuring data integrity involved a video analysis sampling technique to prevent the inclusion of non-participant rider data files in the final data set. Although participants were informed that riding by anyone other than themselves was to be reported so applicable files could be deleted from the data set, this further analysis provided an extra safeguard against including non-participant trips. To verify that the rider was likely the consented study participant for all files included in MSF analyses (since viewing all of the more than 30,000 trip files was time- and cost-prohibitive), a video review sampling technique was conducted. This VTTI-developed tool, the Rapid Driver Identification (RDI) task, was used in the SHRP2 Naturalistic Driving Study, and is described fully for that application in McClafferty, Perez, and Hankey (2015). For the MSF review of each participant's selected files, two snapshots known to be the rider (one with and one without a helmet) were downloaded into the RDI system as a reference for comparison. Then snapshots from the file to be tested were gathered by dividing the trip into three segments of equal length (beginning, middle, and end) and using a face-detection algorithm to select a maximum of 12 snapshots from the trip (4 from each segment). At this point, through the video viewing tool, an analyst compared the known rider reference snapshots to the sample of actual file snapshots to determine whether the rider in the file was the participant. Appropriate quality control checks were also performed. The goal of the rider identification task was to protect against a suspected non-rider contamination of 20% or more of each rider's trips (assuming that such contamination would significantly alter behavioral conclusions about that rider, such as substantially affecting collected mileage). Based on hypotheses testing and Bayes factors testing, if 6 or more out of a random sample of 20 of a rider's video files were contaminated (not the consented rider), then 20% or more
T
92
accuracy)? Can we rely on rider self-reporting, especially reports of previous annual mileage, to estimate current or future mileage? If we survey riders about their previous mileage, can we make any predictions about the upcoming year? Would it be better to phrase this inquiry in terms of mileage during the most recent year, or would we most likely obtain a better estimate if we ask about the rider's overall annual average mileage (perhaps riders, for example, tend to perceive that they ride more now than in previous years)? Although motorcyclists are not expected to ride the same number of miles from year to year, any pattern in mileage estimates for previous years versus actual mileage on the road in the upcoming year can be informative in starting to uncover the actual relationship between estimated and actual rider mileage.
D
81 82
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
E
2
Fig. 1. Description of 91-Rider Sample.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
3
212
3.1. Self-reported mileage (previous year) vs. collected mileage
213
Self-reported mileage during the year immediately preceding study participation for the 91 riders ranged from 100 to 40,000 annual miles (the average was 7868 miles). Annual riding mileage collected during the study for these individuals ranged from 65 to 21,696 miles (mileage was annualized, since some riders participated for less than a year, and others for more). The average of the collected mileage was 4847 miles. Fig. 2 shows individual data points for each of the riders. The best-fit linear relationship between self-reported mileage and adjusted annual riding mileage (blue line) is weak (R2 = 0.3468). The red line indicates a direct linear relationship (x = y), where collected mileage would equal reported mileage. Of the 91 riders, 66 (73%) rode less the following year than reported for the previous year and 25 (27%) rode more. The mean difference between the previous year and the study year was 3022 miles less recorded in the following (study) year. This recorded mileage was, on average, 89% of the reported mileage from the previous year. The range was from 0.03 (collected mileage was 3/100ths of the reported previous annual mileage) to 5.88 (collected mileage was nearly 6 times the self-reported mileage). A chi-square test based on the null hypothesis that the same number of riders report more mileage (compared to collected mileage) as those who report less indicates that there is a significant difference in this sample of 91 riders (significantly more riders reported more mileage than collected, χ2 = 18.473, p b .05). Table 1 includes the estimation tendencies of the 91 riders across Age Group and Gender (whether collected mileage was more or less than the previous year mileage report). The proportion of the self-reported annual mileage that was represented by the collected annual mileage is included as frequencies within “bins” in the table. For the purpose of presentation, collected mileages between 0.9 and 1.1 times the estimate are grouped as being a “close” match. Based on these groupings, the tendency toward riding less than the estimated mileage held regardless of age group, motorcycle type, or gender (in all categories, the majority reported higher mileage than collected values). In other words, for every categorization level (row) for these riders, the total number of riders in the “Rode less than reported” cells is greater than or equal to the number of riders in the “Rode more than reported” cells. However, the tendency was most pronounced for the 31–40 age group (both genders) and the 51– 60 year old female group, with 100% of these groups reporting an annual mileage higher than the mileage collected during the study. Table 2 includes similar data by motorcycle type, and shows that the pattern of riding less than the previously reported mileage held across all categories. Note that, although the majority of riders recorded less mileage than the reported mileage, there were six riders who recorded more than twice the number of miles than they reported riding the previous year (see Tables 1 and 2, column heading “More than 2 ×”). Based on
214 215
R O
O
F
3. Results
201 202 203 204 205 206 207 208 209 210 211
D
E
T
C
199 200
E
197 198
R
195 196
R
193 194
contamination was likely, and a full sample of that rider's video files would be reviewed. This full video review was not necessary, since all rider samples resulted in b 6 non-rider files per sample (84 riders' samples provided no non-rider trips). The few files that were discovered to be a non-consented rider were removed from the database. This structured rider identification task was the minimum check performed for each participant—other types of data analyses were performed on the MSF 100 data set, and every time a video was viewed for analysis, rider identification was the first step. Any time that a non-rider file was discovered, that file was flagged and removed from all subsequent data analyses. Following all data integrity assessments and mileage calculations, the level of association between self-reported mileage and actual recorded riding was evaluated. Differences between these two values were explored in terms of tendencies for actual mileage to be either more or less than the reported mileage for certain demographic groups. In addition, variances between the self-reported mileage when reported as the rider's average annual mileage versus the mileage specific to the year immediately prior to study enrollment were investigated. These analyses were used to discuss the implications of variations in riding behavior or recollection of pre-study riding versus observed riding behavior during study participation.
N C O
191 192
t1:1 t1:2
U
190
P
Fig. 2. Reported annual mileage in previous year compared to actual annual mileage in the following year (91 riders, adjusted for participation time).
Table 1 Mileage estimation by age and gender.
t1:3
Rode less than reported
About the same
Rode more than reported
t1:4
Gender
Age group
0 to 0.2×
0.3× to 0.5×
0.6× to 0.8×
0.9× to 1.1×
1.2× to 1.4×
1.5× to 1.7×
1.8× to 2.0×
N2×
t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15
Female
21–30 31–40 41–50 51–60 21–30 31–40 41–50 51–60 61–70 71–80
1 1 2 0 2 1 4 2 1 0 14
2 0 0 1 2 6 2 6 4 1 24
1 2 1 1 1 2 0 4 3 0 15
0 1 3 1 3 0 3 2 3 1 17
1 0 1 0 2 0 2 2 3 1 12
0 0 0 0 0 0 0 1 1 0 2
0 0 0 0 0 0 0 1 0 0 1
0 0 1 0 2 0 1 2 0 0 6
Male
Q1
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258
4
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
t2:1 t2:2
Table 2 Mileage estimation by motorcycle type.
t2:3
Proportion of reported miles represented by actual collected miles
t2:4
278 279 280 281 282 283 284
N2×
t2:6 t2:7 t2:8 t2:9
Cruiser Sport Touring
6 3 5 14
10 6 8 24
6 4 5 15
5 2 10 17
5 2 5 12
0 0 2 2
0 0 1 1
5 1 0 6
285
Since the most recent travel patterns of participants were expected to be more closely related to the patterns observed during study participation, comparison of self-reported versus collected mileage used the rider's estimate of mileage for the year immediately preceding study participation. However, another question asked of the participants during pre-study survey completion was “On average, how many miles per year do you ride a street motorcycle?”. Perhaps this reported average value was closer to what the rider typically rides, and thus would be more closely correlated with the collected mileage. The relationship of this average annual mileage to the collected annual mileage was investigated to answer these questions. This sample included 90 riders because, of the 91 riders who had been riding for at least a year before study participation, one rider neglected to respond to the question regarding average annual mileage and therefore was removed from this phase of analysis. The first comparison of interest was whether the self-reported previous year mileage was closely related to the reported average annual mileage (whether riders feel that they rode similar miles the previous year as they tend to ride annually). If the mileage tended to be the same, then it would not matter whether previous or average annual mileage was the barometer against which to measure collected mileage. The best-fit linear relationship between self-reported mileage of the previous year and the annual average was fairly strong (R2 = 0.7253). Fig. 3 illustrates this relationship. Many riders (40 of them) tended to report an average annual mileage that was the same as the previous year mileage, whereas the remaining riders were fairly evenly split between riding more than average in the previous year (28 riders) and less than average during the previous year (22 riders). The fairly high overall correlation indicates that most riders are either consistent in their yearly mileage, or they tend to feel that their riding patterns are fairly uniform from year to year. This correlation would support the idea that riders think they ride or want to ride a given number of miles; perhaps they “categorize” themselves as riders of a particular level, regardless of year-to-year variations. For other riders (50 whose previous year and average mileages were different), since the number reporting lower mileage was comparable to those reporting higher mileage than average, these riders may more closely recognize their own variation in riding patterns. However, a t-test performed on the differences between estimated previous year and annual mileage resulted in no significant difference between these two measures (p = 0.2265). Although the relationship between the two types of self-reported mileage indicated similarity, since the estimates were different for 50 of the riders, the correlation of self-reported average annual mileage was tested against the recorded mileage (in the same way that the previous year mileage was tested to produce Fig. 2). Fig. 4 includes the results of this analysis, demonstrating a correlation slightly stronger than that of the previous year versus collected mileage (Fig. 2), but still only moderate to weak (R2 = 0.4017). Again, the pattern tended to be that the estimated annual mileage was more than the actual mileage recorded during the study.
286
P
R O
O
F
3.2. Self-reported mileage (annual average) vs. collected mileage
D
demographic self-reported information, these six riders were all novice (based on self-reported mileage indicating b1600 miles traveled during the previous year, and also taking into account a very rough estimate of the total mileage during the rider's lifetime based on survey responses) and/or newly returning (after taking a break from riding of a year or longer). Within the sample of 91 riders, 20 riders were novice and/or returning. There were 25 riders (27%) who rode in the study for less than one year. To investigate the possibility that the results may be biased by extrapolation of year-long mileage for this sample, data were analyzed with only the 66 riders (73%) who participated longer than a year. Very similar patterns of riding less the following year than reported (whether results were grouped by age, motorcycle type, or gender) were evident in this 73% sample. Another avenue of investigation was to focus only on riders who reported owning or leasing one motorcycle, to reduce the likelihood that substantial additional mileage was being logged on non-instrumented motorcycles. Although the motorcycle used in the study was the primary motorcycle, perhaps additional mileage on a secondary motorcycle provided the “missing” mileage (the discrepancy between reported and recorded mileage). However, the same analyses used to evaluate reported versus recorded mileage for the full set of 91 riders was applied to a subset of 65 riders who owned or leased only the instrumented motorcycle, with similar results. Whether results were stratified by rider age, gender, or motorcycle type, reported mileage was consistently greater than mileage recorded during the study.
E
276 277
1.8× to 2.0×
T
274 275
1.5× to 1.7×
C
272 273
1.2× to 1.4×
E
270 271
0.9× to 1.1×
R
268 269
0.6× to 0.8×
R
266 267
0.3× to 0.5×
O
264 265
0 to 0.2×
C
262 263
Rode less than reported
N
260 261
Rode more than reported
Motorcycle Type
U
259
About the same
t2:5
Fig. 3. Reported annual mileage in previous year compared to reported average annual mileage (90 riders, adjusted for participation time).
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
5
5. Summary
338 339
In general, results indicate that riders may tend to overestimate their annual mileage. Although this pattern is a bit stronger when the estimation is for the most current year of riding, it still occurs regardless of whether the estimation is an average or specific to the previous year. However, when novice or returning riders report annual mileage, some may either significantly underestimate or tend to significantly increase actual mileage as they begin (or continue) to ride. Because of the probability that these riders increase mileage as they learn or continue to ride, what appears to be underestimation is likely to be a true difference related to a change in riding habits. For the general riding population, however, riding less than reported for the previous year(s) seems to be common, and may not be the result of an actual decrease in mileage. Consideration of demographics support these patterns (i.e., these tendencies hold regardless of rider gender, age group, or motorcycle type). There are multiple hypotheses about what may lead to a difference between reported riding distance and actual mileage the subsequent year. For one rider the difference between predicted and actual might be because of a change in work demands. For another, differences might be due to an inability to estimate annual mileage accurately. The impact of weather on riding frequency could certainly change from year to year. It could be that riders over-estimated mileage in the interest of increasing their chances of becoming a study participant, however their response to this question was taken after they were already enrolled, plus that type of behavior might be expected of some individuals, but not a large percentage. Based on the love of many motorcyclists for riding, it may be that riders inadvertently report what the mileage they would like to ride, which ends up being more than they actually get to ride. Another factor that might explain some of the apparent overestimation is that the survey question leading to the estimated “previous year” mileage asked for mileage on all motorcycles, whereas the collected data was from the primary, instrumented motorcycle. However, analysis on only riders reporting that they own or lease one motorcycle supported the full-group analysis results.
349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371
C
E
R
347 348
R
345 346
N C O
343 344
U
341 342
374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390
6. Practical applications
408
This initial look at the relationship of motorcyclist self-reported mileage to actual collected mileage provides caution about reliance on survey or rider report of riding habits, specifically related to either current or average miles on the road. Although reported mileage was not indicative of the same time period during which the actual mileage was collected, the very weak correlation (which tended to be an overestimate) suggests that rider surveys should not be relied on to corroborate other methods of obtaining motorcyclist travel data. If self-reported mileage is used as a supplement or check for motorcyclist travel data, the likelihood that riders may tend to overestimate mileage should be taken into account. In the design of rider surveys or reporting methods to obtain annual mileage estimates, a question asking for annual mileage during the previous year may elicit the same response as one asking for an average annual mileage. To obtain the most thoughtful response, the difference between the two questions should be emphasized, and the respondent should be provided with specific instructions to distinguish between the two types of mileage, perhaps even adding an extra prompt such as “During the past year, did you ride more than, less than, or the same as your average mileage?” In addition to providing more guidance when asking motorcyclists to report their mileage, another application of these results is incorporation of the idea that many riders may overestimate their time on the road during activities such as training. Although this study does not provide conclusive evidence that all riders recall riding more than they
409 410
P
391
E
4. Discussion
T
337
372 373
Motorcyclist training efforts such as the MSF Basic Rider Course emphasize the need for rider self-evaluation of skill level in order to weigh the risk of various situations one might encounter. An important component of self-evaluation, particularly of proficiency, is having a realistic picture of the miles ridden. The MSF 100 Naturalistic Study riders' selfevaluation of riding mileage (represented by questions indicating riding distance in the previous year), along with actual collected mileage during study participation, imply that riders do not generally tend to accurately estimate their mileage, and/or riders do not necessarily ride a similar amount from year to year. Based on these findings, use of selfreport of mileage to target training or interpret study results may be unreasonable. In addition, efforts to check or supplement motorcyclist travel data through surveys or other self-reporting means should be used with caution. Certainly any interpretation of self-reported mileage should incorporate the concept that mileage overestimation seems to be the most likely tendency. In general, reported mileage should not be relied upon as an accurate predictor of future actual mileage.
D
Fig. 4. Reported average annual mileage compared to annual mileage collected during the study (90 riders, adjusted for participation time).
340
R O
O
F
Results of this work provide an indication of how predictive selfreport of annual mileage is of subsequent riding mileage. This unique analysis provides information that is useful in understanding whether there is any disconnect between real versus perceived riding amount for a variety of motorcyclists. For some riders, this disconnect could even be indicative of an overestimation of their own riding proficiency, for those motorcyclists who assume that more miles on the road corresponds to an increased skill level. Although specific reasons for the differences between the self-report of the previous year(s) and actual miles the following year are unknown, what is evident in this study is that there is a pattern of decreased mileage from the estimation to subsequent riding for motorcyclists (in this study, as much of a decrease as 3/100ths of the reported previous year mileage). In terms of the phrasing of a question regarding rider mileage, asking about a rider's average annual mileage may provide a result slightly closer to actual future mileage, but will likely still not be a strong indicator. In general, asking about mileage specifically during the previous year does not appear to produce results much different than asking for an average annual mileage.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407
411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433
458
References
459 460 461 462 463 464 465 466 467 468 469
Lyon, C., Persaud, B., & Himes, S. (2017). Investigating total AADT as a surrogate for motorcycle volumes in estimating safety performance functions for motorcycle crashes. Transportation research record: Journal of the transportation research board, no. 2637. (pp. 9–16). Washington, DC: Transportation Research Board of the National Academies. McClafferty, J., Perez, M., & Hankey, J. (2015). Identification of consented driver trips in the SHRP2 naturalistic driving study data set. Prepared for the strategic highway research program 2. Transportation Research Board of The National Academies (Retrieved from http://hdl.handle.net/10919/70848). Middleton, D., Turner, P., Charara, H., Sunkari, S., Geedipally, S., & Scopatz, R. (2013). Improving the quality of motorcycle travel data collection (NCHRP report 760).
447 448 449 450 451 452
C
445 446
Vicki Williams is a Human Factors Engineer working in the Motorcycle Research Group at VTTI, and is a Certified Professional Ergonomist with two degrees in Industrial Engineering and Operations Research (B.S. IEOR and M.S. IEOR/Human Factors). Before joining VTTI, Vicki had 15 years of experience as a human factors engineer and consultant for various industrial corporations, specializing in ergonomic design, evaluation, and training. Since 2001 she has worked for VTTI, performing all phases of research, including literature review, protocol development, study design, data collection (in-vehicle and remote), data reduction, statistical analysis, and report writing. Her research interests include all human factors aspects of transportation research, most currently protocol development and analysis of video data for large-scale naturalistic motorcycle studies.
479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515
Shane McLaughlin is the Motorcycle Safety Research Group Leader. Shane has been involved in vehicle design and driving safety research for over 17 years. He is the principal Investigator on two naturalistic motorcycle studies. His areas of emphasis include motorcycles, driver support systems, vehicle kinematics, and large scale data mining. Prior to coming to VTTI, Shane worked for Ford Motor Company on vehicle programs and complex interfaces. He received his PhD in Industrial Engineering from Virginia Tech in 2007. Shane is also the Director of VTTI's Center for Automated Vehicle Systems. Mac McCall is a research associate with the Motorcycle Research Group at the Virginia Tech Transportation Institute. Since arriving at VTTI 3 years ago he has focused on the analysis of large-scale naturalistic riding datasets for both the Motorcycle Safety Foundation and NHTSA. His research portfolio includes publications on rider differences in exposure based on when, where, and how frequently riders ride. He completed his undergraduate work at Texas Tech University and holds a Master's from the University of South Dakota, where he is currently completing his doctoral dissertation in Human Factors Psychology on the influence of an aging visual system on driver behavior. Tim Buche has served as president and CEO of the Motorcycle Safety Foundation since 1996. Prior to his current role he served as an MSF Trustee for two years, representing American Suzuki. During his thirteen years with Suzuki he held regional and national positions. In addition to his MBA, with a focus on international business, he has served numerous Boards and committees beyond the world of motorcycling. Tim considers the most rewarding aspect of these past twenty years to be his work with the MSF Board to initiate various research projects, including the milestone MSF 100 Motorcyclists Naturalistic Study, which is expected to yield innovative safety initiatives for decades to come.
E
443 444
F
457
This work was sponsored by the Motorcycle Safety Foundation (MSF), and the authors thank the MSF Board for their full support throughout the study.
441 442
O
455 456 Q9
440
R O
Acknowledgment
438 439
470 471 472 473 474 475 476 477 478
P
454
436 437
National cooperative highway research program. Washington DC: Transportation Research Board. Motorcycle Safety Foundation (2014). Basic rider course. Retrieved from https://www. msf-usa.org/downloads/BRCHandbook.pdf. National Highway Traffic Safety Administration (NHTSA) (2016). Traffic safety facts 2014 data (DOT HS 812 292). (Washington, DC). United States Department of Transportation (2015). Appendix I: Data completeness and reliability report. Retrieved from https://www.transportation.gov/mission/budget/ fy-2015-data-completeness-reliability.
T
453
actually do, results do indicate that the tendency to overestimate previous mileage is a strong possibility. Therefore, other methods of obtaining rider experience levels or helping riders to recognize where they stand in terms of training or experience are recommended. In addition, it may be worthwhile to include references during training or informational outlets for motorcyclists regarding the importance of not overestimating one's own skill level because of perceived time on the road. Finally, because these results suggest that rider reporting and actual mileage may not be closely related, it is even more important to obtain accurate travel data for motorcyclists through other methods such as observation. Suggestions such as those provided in the Middleton et al. (2013) report, as well as other efforts toward improving motorcycle travel data, should be studied and built upon. Unless further study of the differences between rider self-report and actual mileage demonstrate definitive higher correlation, or methods of more accurately obtaining self-reported mileage are implemented and proven, other methods of collecting data first-hand (including naturalistic data collection) appear to be the most promising way to collect these necessary transportation statistics.
D
434 435
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
E
6
U
N
C
O
R
R
516
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
Contents lists available at ScienceDirect
Journal of Safety Research journal homepage: www.elsevier.com/locate/jsr
4Q5 5 6
a
8 7
a r t i c l e
Q8 9 10 11 12 13 18 17 16 15 14
Article history: Received 31 January 2017 Received in revised form 17 May 2017 Accepted 9 October 2017 Available online xxxx
Virginia Tech Transportation Institute, United States Motorcycle Safety Foundation, United States
i n f o
D
b
19 20
T
23
1. Problem
27 28
Motorcycle crashes and fatalities are on the rise, and multiple transportation agencies have called for research into this issue. The motorcyclist fatality rate per 100,000 registered motorcycles in 2014 was six times that of the fatality rate for drivers of passenger cars (per 100,000 registered vehicles), and the fatality rate per motorcycle miles traveled was 27 times that of automobile miles traveled (National Highway Traffic Safety Administration, 2016). Many public and private organizations are interested in identifying causes of these crashes and related fatalities and injuries. One factor could be how capable riders are of estimating their own riding experience. The importance of “SelfAwareness” (including awareness of one's own skill level) is emphasized in the Motorcycle Safety Foundation (MSF) Basic Rider Course (2014). Further, in targeting training or considering survey responses, it is necessary to understand how indicative self-reported mileage is of actual future mileage. In addition, understanding the relationship between motorcyclists' estimated mileage and actual mileage is valuable because there continues to be some difficulty in obtaining accurate estimates of exposure (motorcyclist mileage) for measures such as crash and injury statistics as well as in efforts involving funding allocations, infrastructure planning, and financial forecasting (Lyon, Persaud, & Himes, 2017; Middleton et al., 2013). Exposure estimates have been obtained via various methods, but fall short in terms of likely accuracy. Motorcycle registration does not provide a complete tally of riders due to unlicensed riders or licensing
37 38 39 40 41 42 43 44 45 46 47 48 49 50
E
R
R
N C O
35 36
U
33 34
C
Q7 26
31 32
© 2017 National Safety Council and Elsevier Ltd. All rights reserved.
E
24 22 21 25
29 30
F
Vicki Williams, a,⁎ Shane McLaughlin, a Robert McCall, a Tim Buche b
O
Q43Q3
R O
2
Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year
P
1Q2
⁎ Corresponding author. E-mail addresses: [email protected] (V. Williams), [email protected] (S. McLaughlin), [email protected] (R. McCall), [email protected] (T. Buche).
issued to individuals other than the actual rider. As indicated by the U.S. Department of Transportation (USDOT, 2015), the Motorcyclist Fatality Rate includes the “statistical issue” that the Federal Highway Administration (FHWA) likely underestimates the number of motorcycles on the road each year (supported by the finding from organizations such as the Motorcycle Industry Council that not all riders register their motorcycles). Annual motorcycle inspections rely on accurate odometer readings and faithful inspection scheduling. Observational and roadway detector equipment recordings of motorcyclist traffic flow present multiple difficulties related to collection protocol, location and timing choice, and sensor accuracy. Middleton et al. (2013) discuss some of these shortcomings, and offer various guidelines for calculating the accuracy of current methods used to report motorcycle traffic data. The authors provide detailed research methods and recommendations in terms of equipment and collection methods, and note that there are ongoing efforts to improve motorcycle traffic data. Their paper also presents the possibility of supplementing travel data through motorcyclist surveys, such as the National Household Traffic Survey (NHTS), origin and destination (O & D) surveys, and driver exposure surveys. If this additional source of motorcyclist mileage information (in the form of selfreported mileage) appears to be fairly accurate, it could be used to check or supplement mileage estimations collected through other methods. As the first large-scale naturalistic instrumented motorcycle study to collect real-time mileage and self-reported mileage estimates, the MSF 100 study provides a unique data set from which to draw inferences about the characteristics of motorcyclist self-reported mileage and its application. Investigation of the data found in the study provides knowledge useful in answering multiple questions. Is collection of motorcyclists' self-reported mileage a useful method of checking or supplementing motorcycle travel data (which is in need of improved
https://doi.org/10.1016/j.jsr.2017.10.004 0022-4375/© 2017 National Safety Council and Elsevier Ltd. All rights reserved.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123
C
107 108
E
105 106
R
103 104
R
101 102
O
99 100
C
97 98
N
95 96
U
90 91
F
The MSF 100 Motorcyclists Naturalistic Study was sponsored by the Motorcycle Safety Foundation and conducted by the Virginia Tech Transportation Institute (VTTI) to collect real-world riding data from riders in their natural day-to-day experiences while riding their own motorcycles. Individual rider participation in the study ranged from two months to two years. These 100 riders resided in California, Florida, Virginia, and Arizona, and both video data and motorcycle kinematic data were collected for every trip (defined as the time between keyon and key-off, during which the rider travels from one destination to another). These data were collected via unobtrusive instrumentation of a VTTI-developed data acquisition system (DAS) on each motorcycle, which continuously recorded five video views of the rider and the surrounding environment as well as motorcycle data such as GPS (Global Positioning System), acceleration, gyro, and brake activation. The full set of collected data incorporated over 366,000 miles of riding. Participants also completed various questionnaires prior to equipment installation, including riding exposure surveys, indicating riding habits to date such as annual mileage. This paper explores motorcyclists' self-reported annual riding mileage and the actual amount of riding done within the study. For this evaluation, 91 riders (those who had been riding for at least one year before study enrollment and reported an annual mileage for that year) were considered. Self-reported annual mileage was recorded immediately preceding study participation, directly from the survey question “Approximately how many miles have you ridden a motorcycle on public roads in the past 12 months?” This sample of 71 males and 20 females were of various age groups and represented all three types of motorcycles, as indicated in Fig. 1. Two approaches were used for determining the actual miles ridden. For 78 cases, starting and ending odometer readings were used. These
88 89
O
94
87
R O
2. Method
85 86
P
93
83 84
readings were collected by technicians during equipment installation, and again during de-installation (at the close of participation). During the incorporation of the odometer readings for the data analysis phase, 13 cases were observed with recorded odometer reading that were missing or suspect (did not reasonably pair with the corresponding reading). In those 13 cases, integration of GPS-based speed and time data were used to calculate the distance traveled. Missing GPS values within a trip, including the period of time between DAS start and GPS signal acquisition, were replaced with the mean speed of that same trip. In the event that an entire trip was missing GPS speed, then the mean speed across all trips taken by that participant was used in conjunction with the trip duration to estimate mileage. However, the GPS dropout rate was not such that major adjustments were necessary, and thus total mileage estimates were not significantly affected. In general, the riders were not surrounded by high rise buildings or in weather that would interfere with GPS signal, and cellular signals were not used (the GPS was onboard with an external antenna). All riding data (including odometer mileage and GPS-based calculations) were compared within riders to ensure the most accurate actual mileage was being used in the final analysis. The mileage for each rider was then translated to an adjusted annual riding mileage based on their study participation duration. At the beginning of the study, riders completed a survey which included a question about the number of months they tend to ride each year. Because the accuracy of selfreported data is not guaranteed, nor is it necessarily predictive of how many months overall they would have ridden during the study year, annualized mileage calculations were not based on any attempts at adjustment for riding season. However, each case of a reported abbreviated riding season was investigated to consider the potential effect of the riding season on annualized mileage calculations. The majority of riders who reported an abbreviated riding season participated for around a year and/or recorded very low mileage, so the riding season was either taken into account or was likely altered by a relatively small amount. Those riders falling outside of this category tended to ride nearly yearround, so any over- or under-estimation would also be relatively small. During initial quality control efforts related to this MSF 100 study, one aspect of ensuring data integrity involved a video analysis sampling technique to prevent the inclusion of non-participant rider data files in the final data set. Although participants were informed that riding by anyone other than themselves was to be reported so applicable files could be deleted from the data set, this further analysis provided an extra safeguard against including non-participant trips. To verify that the rider was likely the consented study participant for all files included in MSF analyses (since viewing all of the more than 30,000 trip files was time- and cost-prohibitive), a video review sampling technique was conducted. This VTTI-developed tool, the Rapid Driver Identification (RDI) task, was used in the SHRP2 Naturalistic Driving Study, and is described fully for that application in McClafferty, Perez, and Hankey (2015). For the MSF review of each participant's selected files, two snapshots known to be the rider (one with and one without a helmet) were downloaded into the RDI system as a reference for comparison. Then snapshots from the file to be tested were gathered by dividing the trip into three segments of equal length (beginning, middle, and end) and using a face-detection algorithm to select a maximum of 12 snapshots from the trip (4 from each segment). At this point, through the video viewing tool, an analyst compared the known rider reference snapshots to the sample of actual file snapshots to determine whether the rider in the file was the participant. Appropriate quality control checks were also performed. The goal of the rider identification task was to protect against a suspected non-rider contamination of 20% or more of each rider's trips (assuming that such contamination would significantly alter behavioral conclusions about that rider, such as substantially affecting collected mileage). Based on hypotheses testing and Bayes factors testing, if 6 or more out of a random sample of 20 of a rider's video files were contaminated (not the consented rider), then 20% or more
T
92
accuracy)? Can we rely on rider self-reporting, especially reports of previous annual mileage, to estimate current or future mileage? If we survey riders about their previous mileage, can we make any predictions about the upcoming year? Would it be better to phrase this inquiry in terms of mileage during the most recent year, or would we most likely obtain a better estimate if we ask about the rider's overall annual average mileage (perhaps riders, for example, tend to perceive that they ride more now than in previous years)? Although motorcyclists are not expected to ride the same number of miles from year to year, any pattern in mileage estimates for previous years versus actual mileage on the road in the upcoming year can be informative in starting to uncover the actual relationship between estimated and actual rider mileage.
D
81 82
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
E
2
Fig. 1. Description of 91-Rider Sample.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
3
212
3.1. Self-reported mileage (previous year) vs. collected mileage
213
Self-reported mileage during the year immediately preceding study participation for the 91 riders ranged from 100 to 40,000 annual miles (the average was 7868 miles). Annual riding mileage collected during the study for these individuals ranged from 65 to 21,696 miles (mileage was annualized, since some riders participated for less than a year, and others for more). The average of the collected mileage was 4847 miles. Fig. 2 shows individual data points for each of the riders. The best-fit linear relationship between self-reported mileage and adjusted annual riding mileage (blue line) is weak (R2 = 0.3468). The red line indicates a direct linear relationship (x = y), where collected mileage would equal reported mileage. Of the 91 riders, 66 (73%) rode less the following year than reported for the previous year and 25 (27%) rode more. The mean difference between the previous year and the study year was 3022 miles less recorded in the following (study) year. This recorded mileage was, on average, 89% of the reported mileage from the previous year. The range was from 0.03 (collected mileage was 3/100ths of the reported previous annual mileage) to 5.88 (collected mileage was nearly 6 times the self-reported mileage). A chi-square test based on the null hypothesis that the same number of riders report more mileage (compared to collected mileage) as those who report less indicates that there is a significant difference in this sample of 91 riders (significantly more riders reported more mileage than collected, χ2 = 18.473, p b .05). Table 1 includes the estimation tendencies of the 91 riders across Age Group and Gender (whether collected mileage was more or less than the previous year mileage report). The proportion of the self-reported annual mileage that was represented by the collected annual mileage is included as frequencies within “bins” in the table. For the purpose of presentation, collected mileages between 0.9 and 1.1 times the estimate are grouped as being a “close” match. Based on these groupings, the tendency toward riding less than the estimated mileage held regardless of age group, motorcycle type, or gender (in all categories, the majority reported higher mileage than collected values). In other words, for every categorization level (row) for these riders, the total number of riders in the “Rode less than reported” cells is greater than or equal to the number of riders in the “Rode more than reported” cells. However, the tendency was most pronounced for the 31–40 age group (both genders) and the 51– 60 year old female group, with 100% of these groups reporting an annual mileage higher than the mileage collected during the study. Table 2 includes similar data by motorcycle type, and shows that the pattern of riding less than the previously reported mileage held across all categories. Note that, although the majority of riders recorded less mileage than the reported mileage, there were six riders who recorded more than twice the number of miles than they reported riding the previous year (see Tables 1 and 2, column heading “More than 2 ×”). Based on
214 215
R O
O
F
3. Results
201 202 203 204 205 206 207 208 209 210 211
D
E
T
C
199 200
E
197 198
R
195 196
R
193 194
contamination was likely, and a full sample of that rider's video files would be reviewed. This full video review was not necessary, since all rider samples resulted in b 6 non-rider files per sample (84 riders' samples provided no non-rider trips). The few files that were discovered to be a non-consented rider were removed from the database. This structured rider identification task was the minimum check performed for each participant—other types of data analyses were performed on the MSF 100 data set, and every time a video was viewed for analysis, rider identification was the first step. Any time that a non-rider file was discovered, that file was flagged and removed from all subsequent data analyses. Following all data integrity assessments and mileage calculations, the level of association between self-reported mileage and actual recorded riding was evaluated. Differences between these two values were explored in terms of tendencies for actual mileage to be either more or less than the reported mileage for certain demographic groups. In addition, variances between the self-reported mileage when reported as the rider's average annual mileage versus the mileage specific to the year immediately prior to study enrollment were investigated. These analyses were used to discuss the implications of variations in riding behavior or recollection of pre-study riding versus observed riding behavior during study participation.
N C O
191 192
t1:1 t1:2
U
190
P
Fig. 2. Reported annual mileage in previous year compared to actual annual mileage in the following year (91 riders, adjusted for participation time).
Table 1 Mileage estimation by age and gender.
t1:3
Rode less than reported
About the same
Rode more than reported
t1:4
Gender
Age group
0 to 0.2×
0.3× to 0.5×
0.6× to 0.8×
0.9× to 1.1×
1.2× to 1.4×
1.5× to 1.7×
1.8× to 2.0×
N2×
t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15
Female
21–30 31–40 41–50 51–60 21–30 31–40 41–50 51–60 61–70 71–80
1 1 2 0 2 1 4 2 1 0 14
2 0 0 1 2 6 2 6 4 1 24
1 2 1 1 1 2 0 4 3 0 15
0 1 3 1 3 0 3 2 3 1 17
1 0 1 0 2 0 2 2 3 1 12
0 0 0 0 0 0 0 1 1 0 2
0 0 0 0 0 0 0 1 0 0 1
0 0 1 0 2 0 1 2 0 0 6
Male
Q1
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258
4
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
t2:1 t2:2
Table 2 Mileage estimation by motorcycle type.
t2:3
Proportion of reported miles represented by actual collected miles
t2:4
278 279 280 281 282 283 284
N2×
t2:6 t2:7 t2:8 t2:9
Cruiser Sport Touring
6 3 5 14
10 6 8 24
6 4 5 15
5 2 10 17
5 2 5 12
0 0 2 2
0 0 1 1
5 1 0 6
285
Since the most recent travel patterns of participants were expected to be more closely related to the patterns observed during study participation, comparison of self-reported versus collected mileage used the rider's estimate of mileage for the year immediately preceding study participation. However, another question asked of the participants during pre-study survey completion was “On average, how many miles per year do you ride a street motorcycle?”. Perhaps this reported average value was closer to what the rider typically rides, and thus would be more closely correlated with the collected mileage. The relationship of this average annual mileage to the collected annual mileage was investigated to answer these questions. This sample included 90 riders because, of the 91 riders who had been riding for at least a year before study participation, one rider neglected to respond to the question regarding average annual mileage and therefore was removed from this phase of analysis. The first comparison of interest was whether the self-reported previous year mileage was closely related to the reported average annual mileage (whether riders feel that they rode similar miles the previous year as they tend to ride annually). If the mileage tended to be the same, then it would not matter whether previous or average annual mileage was the barometer against which to measure collected mileage. The best-fit linear relationship between self-reported mileage of the previous year and the annual average was fairly strong (R2 = 0.7253). Fig. 3 illustrates this relationship. Many riders (40 of them) tended to report an average annual mileage that was the same as the previous year mileage, whereas the remaining riders were fairly evenly split between riding more than average in the previous year (28 riders) and less than average during the previous year (22 riders). The fairly high overall correlation indicates that most riders are either consistent in their yearly mileage, or they tend to feel that their riding patterns are fairly uniform from year to year. This correlation would support the idea that riders think they ride or want to ride a given number of miles; perhaps they “categorize” themselves as riders of a particular level, regardless of year-to-year variations. For other riders (50 whose previous year and average mileages were different), since the number reporting lower mileage was comparable to those reporting higher mileage than average, these riders may more closely recognize their own variation in riding patterns. However, a t-test performed on the differences between estimated previous year and annual mileage resulted in no significant difference between these two measures (p = 0.2265). Although the relationship between the two types of self-reported mileage indicated similarity, since the estimates were different for 50 of the riders, the correlation of self-reported average annual mileage was tested against the recorded mileage (in the same way that the previous year mileage was tested to produce Fig. 2). Fig. 4 includes the results of this analysis, demonstrating a correlation slightly stronger than that of the previous year versus collected mileage (Fig. 2), but still only moderate to weak (R2 = 0.4017). Again, the pattern tended to be that the estimated annual mileage was more than the actual mileage recorded during the study.
286
P
R O
O
F
3.2. Self-reported mileage (annual average) vs. collected mileage
D
demographic self-reported information, these six riders were all novice (based on self-reported mileage indicating b1600 miles traveled during the previous year, and also taking into account a very rough estimate of the total mileage during the rider's lifetime based on survey responses) and/or newly returning (after taking a break from riding of a year or longer). Within the sample of 91 riders, 20 riders were novice and/or returning. There were 25 riders (27%) who rode in the study for less than one year. To investigate the possibility that the results may be biased by extrapolation of year-long mileage for this sample, data were analyzed with only the 66 riders (73%) who participated longer than a year. Very similar patterns of riding less the following year than reported (whether results were grouped by age, motorcycle type, or gender) were evident in this 73% sample. Another avenue of investigation was to focus only on riders who reported owning or leasing one motorcycle, to reduce the likelihood that substantial additional mileage was being logged on non-instrumented motorcycles. Although the motorcycle used in the study was the primary motorcycle, perhaps additional mileage on a secondary motorcycle provided the “missing” mileage (the discrepancy between reported and recorded mileage). However, the same analyses used to evaluate reported versus recorded mileage for the full set of 91 riders was applied to a subset of 65 riders who owned or leased only the instrumented motorcycle, with similar results. Whether results were stratified by rider age, gender, or motorcycle type, reported mileage was consistently greater than mileage recorded during the study.
E
276 277
1.8× to 2.0×
T
274 275
1.5× to 1.7×
C
272 273
1.2× to 1.4×
E
270 271
0.9× to 1.1×
R
268 269
0.6× to 0.8×
R
266 267
0.3× to 0.5×
O
264 265
0 to 0.2×
C
262 263
Rode less than reported
N
260 261
Rode more than reported
Motorcycle Type
U
259
About the same
t2:5
Fig. 3. Reported annual mileage in previous year compared to reported average annual mileage (90 riders, adjusted for participation time).
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
5
5. Summary
338 339
In general, results indicate that riders may tend to overestimate their annual mileage. Although this pattern is a bit stronger when the estimation is for the most current year of riding, it still occurs regardless of whether the estimation is an average or specific to the previous year. However, when novice or returning riders report annual mileage, some may either significantly underestimate or tend to significantly increase actual mileage as they begin (or continue) to ride. Because of the probability that these riders increase mileage as they learn or continue to ride, what appears to be underestimation is likely to be a true difference related to a change in riding habits. For the general riding population, however, riding less than reported for the previous year(s) seems to be common, and may not be the result of an actual decrease in mileage. Consideration of demographics support these patterns (i.e., these tendencies hold regardless of rider gender, age group, or motorcycle type). There are multiple hypotheses about what may lead to a difference between reported riding distance and actual mileage the subsequent year. For one rider the difference between predicted and actual might be because of a change in work demands. For another, differences might be due to an inability to estimate annual mileage accurately. The impact of weather on riding frequency could certainly change from year to year. It could be that riders over-estimated mileage in the interest of increasing their chances of becoming a study participant, however their response to this question was taken after they were already enrolled, plus that type of behavior might be expected of some individuals, but not a large percentage. Based on the love of many motorcyclists for riding, it may be that riders inadvertently report what the mileage they would like to ride, which ends up being more than they actually get to ride. Another factor that might explain some of the apparent overestimation is that the survey question leading to the estimated “previous year” mileage asked for mileage on all motorcycles, whereas the collected data was from the primary, instrumented motorcycle. However, analysis on only riders reporting that they own or lease one motorcycle supported the full-group analysis results.
349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371
C
E
R
347 348
R
345 346
N C O
343 344
U
341 342
374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390
6. Practical applications
408
This initial look at the relationship of motorcyclist self-reported mileage to actual collected mileage provides caution about reliance on survey or rider report of riding habits, specifically related to either current or average miles on the road. Although reported mileage was not indicative of the same time period during which the actual mileage was collected, the very weak correlation (which tended to be an overestimate) suggests that rider surveys should not be relied on to corroborate other methods of obtaining motorcyclist travel data. If self-reported mileage is used as a supplement or check for motorcyclist travel data, the likelihood that riders may tend to overestimate mileage should be taken into account. In the design of rider surveys or reporting methods to obtain annual mileage estimates, a question asking for annual mileage during the previous year may elicit the same response as one asking for an average annual mileage. To obtain the most thoughtful response, the difference between the two questions should be emphasized, and the respondent should be provided with specific instructions to distinguish between the two types of mileage, perhaps even adding an extra prompt such as “During the past year, did you ride more than, less than, or the same as your average mileage?” In addition to providing more guidance when asking motorcyclists to report their mileage, another application of these results is incorporation of the idea that many riders may overestimate their time on the road during activities such as training. Although this study does not provide conclusive evidence that all riders recall riding more than they
409 410
P
391
E
4. Discussion
T
337
372 373
Motorcyclist training efforts such as the MSF Basic Rider Course emphasize the need for rider self-evaluation of skill level in order to weigh the risk of various situations one might encounter. An important component of self-evaluation, particularly of proficiency, is having a realistic picture of the miles ridden. The MSF 100 Naturalistic Study riders' selfevaluation of riding mileage (represented by questions indicating riding distance in the previous year), along with actual collected mileage during study participation, imply that riders do not generally tend to accurately estimate their mileage, and/or riders do not necessarily ride a similar amount from year to year. Based on these findings, use of selfreport of mileage to target training or interpret study results may be unreasonable. In addition, efforts to check or supplement motorcyclist travel data through surveys or other self-reporting means should be used with caution. Certainly any interpretation of self-reported mileage should incorporate the concept that mileage overestimation seems to be the most likely tendency. In general, reported mileage should not be relied upon as an accurate predictor of future actual mileage.
D
Fig. 4. Reported average annual mileage compared to annual mileage collected during the study (90 riders, adjusted for participation time).
340
R O
O
F
Results of this work provide an indication of how predictive selfreport of annual mileage is of subsequent riding mileage. This unique analysis provides information that is useful in understanding whether there is any disconnect between real versus perceived riding amount for a variety of motorcyclists. For some riders, this disconnect could even be indicative of an overestimation of their own riding proficiency, for those motorcyclists who assume that more miles on the road corresponds to an increased skill level. Although specific reasons for the differences between the self-report of the previous year(s) and actual miles the following year are unknown, what is evident in this study is that there is a pattern of decreased mileage from the estimation to subsequent riding for motorcyclists (in this study, as much of a decrease as 3/100ths of the reported previous year mileage). In terms of the phrasing of a question regarding rider mileage, asking about a rider's average annual mileage may provide a result slightly closer to actual future mileage, but will likely still not be a strong indicator. In general, asking about mileage specifically during the previous year does not appear to produce results much different than asking for an average annual mileage.
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004
392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407
411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433
458
References
459 460 461 462 463 464 465 466 467 468 469
Lyon, C., Persaud, B., & Himes, S. (2017). Investigating total AADT as a surrogate for motorcycle volumes in estimating safety performance functions for motorcycle crashes. Transportation research record: Journal of the transportation research board, no. 2637. (pp. 9–16). Washington, DC: Transportation Research Board of the National Academies. McClafferty, J., Perez, M., & Hankey, J. (2015). Identification of consented driver trips in the SHRP2 naturalistic driving study data set. Prepared for the strategic highway research program 2. Transportation Research Board of The National Academies (Retrieved from http://hdl.handle.net/10919/70848). Middleton, D., Turner, P., Charara, H., Sunkari, S., Geedipally, S., & Scopatz, R. (2013). Improving the quality of motorcycle travel data collection (NCHRP report 760).
447 448 449 450 451 452
C
445 446
Vicki Williams is a Human Factors Engineer working in the Motorcycle Research Group at VTTI, and is a Certified Professional Ergonomist with two degrees in Industrial Engineering and Operations Research (B.S. IEOR and M.S. IEOR/Human Factors). Before joining VTTI, Vicki had 15 years of experience as a human factors engineer and consultant for various industrial corporations, specializing in ergonomic design, evaluation, and training. Since 2001 she has worked for VTTI, performing all phases of research, including literature review, protocol development, study design, data collection (in-vehicle and remote), data reduction, statistical analysis, and report writing. Her research interests include all human factors aspects of transportation research, most currently protocol development and analysis of video data for large-scale naturalistic motorcycle studies.
479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515
Shane McLaughlin is the Motorcycle Safety Research Group Leader. Shane has been involved in vehicle design and driving safety research for over 17 years. He is the principal Investigator on two naturalistic motorcycle studies. His areas of emphasis include motorcycles, driver support systems, vehicle kinematics, and large scale data mining. Prior to coming to VTTI, Shane worked for Ford Motor Company on vehicle programs and complex interfaces. He received his PhD in Industrial Engineering from Virginia Tech in 2007. Shane is also the Director of VTTI's Center for Automated Vehicle Systems. Mac McCall is a research associate with the Motorcycle Research Group at the Virginia Tech Transportation Institute. Since arriving at VTTI 3 years ago he has focused on the analysis of large-scale naturalistic riding datasets for both the Motorcycle Safety Foundation and NHTSA. His research portfolio includes publications on rider differences in exposure based on when, where, and how frequently riders ride. He completed his undergraduate work at Texas Tech University and holds a Master's from the University of South Dakota, where he is currently completing his doctoral dissertation in Human Factors Psychology on the influence of an aging visual system on driver behavior. Tim Buche has served as president and CEO of the Motorcycle Safety Foundation since 1996. Prior to his current role he served as an MSF Trustee for two years, representing American Suzuki. During his thirteen years with Suzuki he held regional and national positions. In addition to his MBA, with a focus on international business, he has served numerous Boards and committees beyond the world of motorcycling. Tim considers the most rewarding aspect of these past twenty years to be his work with the MSF Board to initiate various research projects, including the milestone MSF 100 Motorcyclists Naturalistic Study, which is expected to yield innovative safety initiatives for decades to come.
E
443 444
F
457
This work was sponsored by the Motorcycle Safety Foundation (MSF), and the authors thank the MSF Board for their full support throughout the study.
441 442
O
455 456 Q9
440
R O
Acknowledgment
438 439
470 471 472 473 474 475 476 477 478
P
454
436 437
National cooperative highway research program. Washington DC: Transportation Research Board. Motorcycle Safety Foundation (2014). Basic rider course. Retrieved from https://www. msf-usa.org/downloads/BRCHandbook.pdf. National Highway Traffic Safety Administration (NHTSA) (2016). Traffic safety facts 2014 data (DOT HS 812 292). (Washington, DC). United States Department of Transportation (2015). Appendix I: Data completeness and reliability report. Retrieved from https://www.transportation.gov/mission/budget/ fy-2015-data-completeness-reliability.
T
453
actually do, results do indicate that the tendency to overestimate previous mileage is a strong possibility. Therefore, other methods of obtaining rider experience levels or helping riders to recognize where they stand in terms of training or experience are recommended. In addition, it may be worthwhile to include references during training or informational outlets for motorcyclists regarding the importance of not overestimating one's own skill level because of perceived time on the road. Finally, because these results suggest that rider reporting and actual mileage may not be closely related, it is even more important to obtain accurate travel data for motorcyclists through other methods such as observation. Suggestions such as those provided in the Middleton et al. (2013) report, as well as other efforts toward improving motorcycle travel data, should be studied and built upon. Unless further study of the differences between rider self-report and actual mileage demonstrate definitive higher correlation, or methods of more accurately obtaining self-reported mileage are implemented and proven, other methods of collecting data first-hand (including naturalistic data collection) appear to be the most promising way to collect these necessary transportation statistics.
D
434 435
V. Williams et al. / Journal of Safety Research xxx (2017) xxx–xxx
E
6
U
N
C
O
R
R
516
Please cite this article as: Williams, V., et al., Motorcyclists' self-reported riding mileage versus actual riding mileage in the following year, Journal of Safety Research (2017), https://doi.org/10.1016/j.jsr.2017.10.004