- Email: [email protected]
Sources of uncertainty in estimated benefits of road safety programmes
Accident Analysis and Prevention 42 (2010) 2171–2178
Contents lists available at ScienceDirect
Accident Analysis and Prevention journal homepage: www.elsevier.com/locate/aap
Sources of uncertainty in estimated benefits of road safety programmes Rune Elvik Institute of Transport Economics, Gaustadalléen 21, NO-0349 Oslo, Norway
a r t i c l e
i n f o
Article history: Received 21 December 2009 Received in revised form 23 February 2010 Accepted 30 March 2010 Keywords: Road safety programme Estimates of effect Uncertainty Propagation of errors
a b s t r a c t National road safety programmes have been developed in many motorised countries. Some of these programmes contain estimates of the safety benefits that were expected to be realised if the programmes were fully implemented. When these estimates are compared to actual outcomes, it is not uncommon to find large differences. This paper argues that differences between the predicted and actual results of road safety programmes could be the result of a large, but generally unrecognised, uncertainty inherent in estimates of the effects of such programmes. Ten sources of uncertainty are identified and briefly described. The possibility of describing these sources of uncertainty numerically, and of estimating their joint contribution, is discussed. It is concluded that at the current state of knowledge, it is not possible to meaningfully estimate the total uncertainty inherent in road safety programmes. The prospects of reducing uncertainty by means of research are discussed. © 2010 Elsevier Ltd. All rights reserved.
1. Introduction Quantified targets for improving road safety and programmes designed to realise these targets have been adopted by many motorised countries in recent years. These programmes do not always produce the expected results. On the contrary, it is commonly found that the actual number of people killed or injured in road accidents exceeds the numbers that were predicted according to the road safety programme. Road safety is influenced by very many factors and no road safety programme can hope to influence all of them. Road safety is therefore only to some extent controllable by means of government programmes. Besides, the effects of a programme on road safety are highly uncertain—more so than is generally recognised. This paper is a first attempt to study the following questions: 1. What are the principal sources of uncertainty in the estimated benefits of national road safety programmes? 2. Is it possible to quantify all sources of uncertainty? 3. Is it possible to estimate the joint contribution of all sources of uncertainty, i.e. the total uncertainty associated with estimates of the benefits of a road safety programme? The term “benefits” refers to the monetary benefits of a road safety programme in terms of reduced costs of accidents and other favourable impacts. Very often, the principal focus of interest is on effects stated in terms of changes in the number of people killed
E-mail address: [email protected]. 0001-4575/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.aap.2010.03.022
or injured. These impacts are the principal focus of interest in this paper as well. The reason for including the monetary valuation of impacts in the analysis will be explained later. 2. Examples of erroneous estimates of effects of road safety programmes It is easy to give examples of road safety programmes that have not resulted in the safety benefits they were expected to produce. Fig. 1 gives one example. It shows road accident fatalities in Denmark from 1989 to 2000 compared to the targeted development according to a road safety programme proposed in 1988 (Færdselssikkerhedskommissionen, 1988). During the first years, outcomes were close the road safety programme, but then progress slowed down. At the end of the period, the actual number of road accident fatalities clearly exceeded the number expected to occur according to the road safety programme. Similar examples can be given for Finland (Trafiksäkerhetsdelegationen, 1982) and Sweden (Vägverket, 1999). For Norway, on the other hand, the number of fatalities has developed more favourably than predicted in the National Transport Plan for the 2002–2011 term (Samferdselsdepartementet, 2000). Thus, although too optimistic predictions appear to be more common than too pessimistic, examples of both these errors can be found. Since erroneous predictions of the impacts of road safety policy presumably are not intentional, these errors must be attributable to sources of uncertainty in estimates of programme impacts that were either unrecognised or erroneously estimated when the programme was prepared. Wishful thinking about the impacts of road safety measures cannot be ruled out. However, wishful thinking is not deliberate. In general, victims of it are
2172
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
unaware of the fact that they are being too optimistic. A victim of wishful thinking may nevertheless be inclined to resist corrections, since these imply that the outcome wished for is more difficult to attain. In the following, sources of uncertainty in the estimated benefits of road safety programmes are identified, the possibilities of analysing them statistically are discussed and examples are given of how to estimate uncertainty. The prospects of reducing uncertainty are discussed. 3. A taxonomy of sources of uncertainty There are several sources of uncertainty in the estimated benefits of national road safety programmes. The following ten sources of uncertainty have been identified: 1. Random variation in the recorded number of accidents or injuries in the target group of a road safety measure; 2. Incomplete or inaccurate reporting of accidents or injuries in official accident statistics; 3. Uncertainty about the definition of the target group of accidents or injuries influenced by a road safety measure; 4. Random variation in the effects of a road safety measure on accidents or injuries; 5. Unknown sources of systematic variation in the effects of a road safety measure on accidents or injuries; 6. Unknown duration or stability over time of the effects of a road safety measure; 7. Uncertainty with respect to if and how the first order effects of a road safety measure are modified when it is combined with other road safety measures; 8. Uncertainty with respect to assumptions made about the effects of exogenous factors influencing road safety; 9. Uncertainty about the degree to which planned road safety measures will actually be implemented; 10. Uncertain monetary valuation of the benefits of road safety measures. Each of these sources of uncertainty will be briefly described. 3.1. Random variation in the target number of accidents or injuries influenced by each road safety measure A road safety programme will normally consist of a number of road safety measures. Each of these measures is intended to influence a certain target group of accidents or injuries. As an example, converting junctions to roundabouts will influence the number of accidents at junctions. To estimate the impacts of this on road safety, one must first estimate how many accidents or injuries will be influenced by converting a certain number of junctions to roundabouts. One way of doing this is to apply the following model: Expected number of injured road users influenced = Number of junctions converted × Mean traffic volume × Mean injury rate Thus, in a recent road safety impact assessment for Norway (Elvik, 2007), it was assumed that conversion of junctions to roundabouts would start by converting junctions with a high traffic volume and continue with junctions with less traffic. It was estimated that it would be cost-effective (i.e. marginal benefits would exceed marginal costs) to convert 460 three leg junctions to roundabouts. The number of injured road users expected to be influenced by this was estimated as: 460 × 12349 × 365 × 10−6 × 0.091 = 189
Fig. 1. Planned and actual number of fatalities according to Danish road safety programme 1988–2000.
The first four terms represent exposure; the last term is the mean injury rate per million entering vehicles in three leg junctions. In principle, there are at least three sources of uncertainty in this estimate: 1. Uncertainty with respect to how junctions that are candidates for conversion to roundabouts are distributed according to traffic volume, 2. Uncertainty with respect to the mean injury rate for three leg junctions before conversion to roundabouts, 3. Random variation in the actual number of injured road users in junctions that are converted to roundabouts. If the number of injured road users is assumed to be a Poisson variable, the variance equals the expected number (in this case estimated as 189); the standard error is the square root of the variance. However, as shown by Fridstrøm et al. (1995) and Bijleveld (2005), the Poisson assumption is strictly speaking only valid for accidents, not for injured road users, since there must by definition be at least one injured person in each injury accident, and since injuries are not independent. Hence, the number of injured road users in a sample of accidents that occur at random will display over-dispersion. 3.2. Incomplete or inaccurate accident reporting in official statistics The estimate given above of the number of injured road users that could be expected to be influenced by the introduction of a road safety measure applied an injury rate that was based on official accident statistics. However, accident reporting is incomplete and inaccurate in all countries (Elvik and Mysen, 1999). The true number of injured road users influenced by a road safety measure will therefore be greater than the recorded number. Hauer and Hakkert (1988) have shown how one can estimate the true number of accidents and the uncertainty of this estimate, provided the level of accident reporting and the variance of the level of accident reporting are known. Unfortunately, this knowledge is rarely likely to be available at the level of detail that is required for meaningful use of the corrections described by Hauer and Hakkert. Besides, the purpose of the models developed by Hauer and Hakkert is to estimate the true number of accidents, whereas what we would like to know is how incomplete accident reporting adds to the uncertainty of the recorded number of accidents. The reason why this is what we need to know, is that an assessment of the success or failure of a road safety programme will most likely be based on the recorded number of
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
accidents, not on an estimate of the unknown true number of accidents. As an approximation, one may assume that the addition to uncertainty in the recorded number of accidents attributable to incomplete accident reporting is proportional to the ratio: (variance of estimated true number of accidents/estimated true number of accidents). 3.3. Uncertain definition of target group of accidents or injuries Most road safety measures are intended to influence a clearly defined target group of accidents or injuries. In some cases, however, the group of accidents actually influenced by a measure will not be identical to the formally defined target group. In such cases, prediction of effect based on a formally defined target group may deviate from the effect actually observed, normally by being too optimistic. A case in point is the study of changes in motorcycle legislation in Great Britain by Broughton (1987). He evaluated the effects of banning learner motorcyclists from using motorcycles with an engine volume of more than 125 cubic centimetres. Before the ban was introduced, accidents involving learner motorcyclists riding large motorcycles represented about 13% of all injured motorcyclists and about 31% of all learner motorcyclists injured. If the ban was perfectly complied with, one would expect these accidents to be eliminated. What happened was that learner motorcyclists switched to smaller motorcycles. Accidents involving learner motorcyclists riding small motorcycles increased. As a result of this, the total number of accidents among learner motorcyclists was reduced by only 8%. This reduction was actually smaller than the reduction of accidents among experienced motorcyclists in the same period (11%). A related case is presented by Wanvik (2009). His study evaluated the effect on accidents of road lighting. It has traditionally been assumed that road lighting only influences accidents in darkness; hence, accidents in daylight may serve as a comparison group in a before-and-after study. Relying on this assumption and using data for Dutch motorways, Wanvik estimated the effect of road lighting to an impressive 58% reduction of accidents in darkness. Lighting poles represent a new traffic hazard on lit roads. This particular hazard does not exist on unlit roads. Accidents in which vehicles strike lighting poles may occur all day and therefore influence the number of daytime accidents. When the estimate of effect was adjusted for this, the effect attributed to road lighting on Dutch motorways was reduced to a 51% reduction of accidents in darkness. 3.4. Random variation in effects of road safety measures For most road safety measures that are included in national road safety programmes, there will be multiple estimates of effect based on several evaluation studies. Summary estimates of effect based on meta-analysis will sometimes be available. In that case, uncertainty attributable to random variation in the effects of a road safety measure can be estimated by taking the inverse value of the fixedeffects statistical weight of the summary estimate of effect. This will indicate the purely random variation in effects around the summary effect. If summary estimates of effect based on meta-analysis are not available, one may estimate the variance of an estimate of effect based on a single study by applying the techniques explained by Hauer (1997). These approaches will not always be applicable, in particular not with respect to new road safety measures for which there may not be any evaluation studies that show effects on accidents and the uncertainty of those effects.
2173
3.5. Unknown sources of systematic variation in effects of road safety measures The effects of many road safety measures are likely to vary, depending on, for example, the standard of the measure (good road lighting is likely to be more effective than poor road lighting), the extent of its use (more police enforcement may be more effective than a little police enforcement), or the context in which it is introduced (a bypass road may be more effective in a small town than in a large town). Unfortunately, little is known about the sources and patterns of systematic variation in the effects of road safety measures. A fairly common situation may therefore be that there is known to be systematic variation in the effects of a road safety measure, as indicated for example by statistical tests made as part of meta-analyses, but that sources of this variation, and how best to model it, are unknown. This may lead to erroneous estimates of the effects of a road safety measure. Fig. 2 shows a case in point. It is based on a paper by Elvik (2009a) in which accident modification functions for the effects of bypass roads were developed. These functions were based on three evaluation studies reported in Denmark, Norway and Sweden, employing broadly speaking the same study design. The effects of bypass roads were found to vary according to the size of the population of the bypassed town. If the variation in the effects of bypass roads had been unknown, the effect of building a new bypass road would have been predicted by applying a constant accident modification factor, shown by the straight line in Fig. 2. It can be seen that this would have given too pessimistic predictions for small towns and to optimistic predictions for larger towns. Fitting an accident modification function allows the predicted effect of a new bypass road to vary according to the size of the town. Yet, even these predictions are uncertain, as indicated in Fig. 2 by the dotted curves showing the most optimistic and the most pessimistic of the accident modification functions that were developed in the study. 3.6. Unknown duration or stability over time of the effects of a road safety measure Do the effects of a road safety measure gradually disappear as time goes by after its introduction? Or do the effects of introducing a road safety measure change over time? This may sound as two versions of the same question, but it is actually two different questions. The first question is whether the effect on accidents of, say, a roundabout are the same 10 years after it was built as during the first year. The second question is whether new roundabouts built today have larger or smaller effects during their first year than roundabouts built fifteen years ago. Very little is known about these questions. There is, however, scattered evidence from a few studies suggesting that the effects of a certain road safety measure may indeed change over time. Elvik et al. (2003) reviewed studies of the effects of daytime running lights on cars. Five studies permitted these effects to be evaluated for different lengths of the after-period (i.e. after the use of daytime running lights became mandatory). Three of the five studies indicated that effects declined over time; one study was inconclusive; one study indicated that effects increased over time. On balance, these studies suggest that it is more likely that effects are weakened over time than that they are strengthened. A second example is given in Fig. 3. Fig. 3 shows the summary estimates of converting junctions to roundabouts based on evaluation studies reported in Norway. The first estimate is based on the first study reported in Norway; the second on the first two studies, and so on. At first, the effects on injury accidents of converting junctions to roundabouts seemed to increase. From the first five studies
2174
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
Fig. 2. Systematic variation in effects on accidents of bypass roads. Source: Elvik (2009a).
onwards, however, effects have become smaller as each new study has been reported. The addition of the ninth study to the sample of studies was even associated with an increase of the size of the confidence interval for the summary estimate of effect. The width of the 95% confidence interval for the summary estimate of effect based on the first eight studies was 0.157 (0.557 minus 0.400). When the ninth study was added, the width of the confidence interval increased to 0.166. Apparently, it is not always the case that adding new studies makes estimates of effect more precise. To statistically estimate and account for the uncertainty resulting from variation of the effects of road safety measures over time, it is necessary to predict the future effects of road safety measures. This cannot be done with any confidence, as shown by Elvik (1996). 3.7. Uncertainty regarding the combined effects of several measures A national road safety programme normally consists of a large number of road safety measures. A recent road safety impact assessment for Norway, for example (Elvik, 2007), included 45 road safety measures. A problem that needs to be addressed in any road safety programme is how best to estimate the combined effects of the
measures that form a programme. This cannot be done simply by adding the estimates of effect developed for each road safety measure. If three road safety measures influence the same target group of accidents, and are expected to reduce accidents by 50%, 40% and 30%, their combined effects cannot possibly be 120%. Effects combine multiplicatively, not additively. In a recent paper, Elvik (2009b) reviewed the very few studies that provide evidence about the combined effects of several road safety measures. Two models were developed to statistically estimate the combined effects of road safety measures. One of these models, labelled the common residuals model, assumes that the percentage effect of a road safety measure remains unchanged when the measure is combined with one or more other measures. Another model, labelled the dominant common residuals model, assumed that in any set of road safety measures, there will be one or more measures that are “dominant” in the sense that once they have been introduced, the effects of the remaining measures are weakened. Exploratory analyses indicated that both these models can be defended and that no model appears to be clearly superior to the other. In general, the dominant common residuals model will give more conservative estimates of the combined effects of road safety measures than the common residuals model. There will, accord-
Fig. 3. Summary estimates of effects on injury accidents of converting junctions to roundabouts.
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
ingly, always be uncertainty regarding the combined effects of measures that form a road safety programme. 3.8. Uncertainty regarding the effects of exogenous factors A road safety programme will typically apply to a period of 5–15 years. Although the programme may significantly influence road safety during this period, it will never be the only factor that influences road safety and may not be the most important factor. The outcomes that are observed in terms of the count of fatalities, injured road users and accidents, will be the result of all factors that influence road safety, not just the road safety programme. When developing a road safety programme, it is essential to identify as precisely as possible the influence it will have on road safety, given the effects of all other factors influencing road safety. This is sometimes done by developing a so called baseline scenario, in which road safety is predicted assuming that the programme is not introduced. Once a baseline forecast has been made, the difference a road safety programme can make is estimated. Although this sounds analytically straightforward, it is actually almost impossible to do in practice. Ideally speaking, the baseline scenario is intended to show what will happen if no road safety programme is introduced. Past trends in road safety give no information about this. Past trends reflect all factors that have influenced road safety, including all road safety measures that were introduced in the past. Hence, projecting past trends to the future does not show what might happen in a baseline scenario; it rather shows what might happen if road safety measures (and other factors) continue to be introduced at the same rate as in the past and continue to be equally effective. Projecting past trends is a notoriously unreliable method for predicting future changes (Elvik, 2010). Trying to identify factors that explain past trends does not help very much; in fact it makes prediction considerably more complex, since the explanatory factors have to be predicted in order to predict safety outcomes. To add to complexity, it is even conceivable that the effects of some exogenous factors changes over time. As an example, in the analysis reported by Elvik (2010), the sign of the coefficient representing the effect of GDP per capita on traffic fatalities changed in Great Britain and the Netherlands during the period 1970–2007, depending on which years were included in the analysis (1970–1989, 1970–1999 or 1970–2007). While some factors that have influenced past trends may be identified (Elvik et al., 2009), this cannot be done in a very rigorous manner and may be of little help in predicting future changes. 3.9. Uncertain degree of implementation of road safety measures Road safety programmes are almost never fully implemented. Estimates of their effects assume full implementation. When implementation is less than one hundred percent, effects on road safety will be reduced accordingly. In Norway, a system for long-term planning of road investments was created around 1970; the basic elements of this system are still in place. Fairly detailed investment programmes are developed every four years. At the end of the period, an assessment is made of the extent to which the investment programme has been implemented. The first of these assessments, referring to the 1970–1973 planning term, was reported in 1973 (Samferdselsdepartementet, 1973). It showed that 96.6% of planned investments had been implemented. The degree of implementation has subsequently declined. For the 1978–1981 term (Samferdselsdepartementet, 1981), it was down to 92.9%. For the 1994–1997 planning term (Samferdselsdepartementet, 1997), 92.7% of planned investments actually took place, and only 81% of the targeted effect on accidents was realised.
2175
Estimates of effect obviously tend to be based on an assumption of 100% implementation. This will in nearly all cases be too optimistic, although the actual degree of implementation of a plan can never be predicted with much confidence. 3.10. Uncertain monetary valuation of impacts of road safety measures All sources of uncertainty discussed so far, refer to uncertainty about the impacts of a road safety programme stated in terms of changes in the number of accidents or the number of road users killed or injured. The title of the paper, however, refers to the benefits of road safety programmes, not to their effects on accidents or injuries. What is the difference, and why has the term benefits been introduced? The difference between the impacts of a road safety programme and the benefits of the programme is that the latter term refers to the conversion of all impacts of the programme to monetary terms. It is not uncommon that cost–benefit analyses of road safety measures are performed as part of the development of a road safety programme, in order to identify the road safety measures that are most cost-effective in improving road safety. The priorities given to the various road safety measures may not always be based strictly on the results of cost–benefit analyses. In principle, however, cost–benefit analyses can be used to determine the optimal use of each road safety measure and the optimal overall level of road safety. If policy makers have an ambition of using cost–benefit analyses to determine the optimal level of safety, uncertainty about the monetary valuation of impacts becomes crucial. The monetary valuation both of road safety impacts, as well as other relevant impacts of a programme, like impacts on travel time, is highly uncertain (de Blaeij et al., 2003). It is therefore not possible to determine the optimal level of safety very precisely, if at all. Depending on how uncertainty with respect to monetary valuation is treated analytically, there can be a narrow or a broad range of outcomes that will be regarded as representing an optimal level of safety. The optimal size of the budget may also vary considerably depending on the monetary valuations adopted. 4. The basic model for the propagation of errors The problem facing a policy maker who wants to quantify the uncertainty associated with the impacts and benefits of a road safety programme is how to estimate the combined contribution of several sources of uncertainty. A general answer to this problem can be found in statistics, in which the basic model for the propagation of errors has been developed (Rasmussen, 1964; Sverdrup, 1964). Assume that the outcome of interest can be modelled as a function of several variables: Y = f (X, Z, . . . , W )
(1)
Please note that all elementary arithmetical operations (adding, subtracting, multiplying, dividing), can be treated as functions. A random variable is also, by definition, a function. If the variance of each term entering a function (e.g. each of two numbers that are added) is independent of the variance of the other terms, the variance of the outcome equals:
Var(R) =
∂R ∂X
2
Var(X) +
∂R ∂Y
2
Var(Y ) + · · · +
∂R ∂W
2
Var(W )
(2)
In Eq. (2), ∂R/∂X. . . denote the partial derivatives of the function R with respect to the variables X, Y, . . ., W. Var(X, Y, . . ., W) is the variance of each of the variables entering the function. It is well known (Strand, 1987) that if the variances of the independent variables entering a function are correlated, the model for the propagation of errors becomes considerably more complicated. In this
2176
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
paper, only the elementary form of the model will be considered. This does not necessarily mean that it is the most correct model, simply that current knowledge does not justify the use of models that include terms representing the correlations between variables. Estimating the benefits of a road safety programme can be modelled as a set of successive multiplications. At the first stage, the number of road users influenced by a road safety measure is obtained by multiplying exposure and injury rate for each level of implementation of each road safety measure. That level could, as an example, be to convert all junctions to roundabouts if the benefits of doing so exceed the costs. The next stage is to estimate the number of injuries prevented by a measure, by multiplying the number of injured road users in the target group by an accident modification factor or accident modification function. The economic benefit associated with each measure is obtained by multiplying the number of injuries prevented at each level of severity by the appropriate monetary valuation of preventing those injuries. Finally, the combined effects of a set of measures can also be obtained by multiplying the “residual factors” of the measures, i.e. the fraction of injuries in the target group that is not prevented by a measure. In the multiplication R = A × B, the partial derivatives are: ∂R =B ∂A
and
∂R =A ∂B
A and B are interpreted as random variables. This makes estimation of compound uncertainty easy, by proceeding in stages term by term. The statistical software @RISK (Palisade Corporation, 2009) permits the combined uncertainty resulting from multiple sources to be estimated based on the specification of a probability distribution applying to each of the sources of uncertainty. Users of the software can choose from a total of 37 probability distributions. An informative use of this software presumes, however, that each source of uncertainty is sufficiently well known to be described in terms of a certain probability distribution. 5. Quantifying each source of uncertainty To what extent is it possible to meaningfully quantify each of the sources of uncertainty discussed in Section 3 of the paper? Table 1 gives a summary of the possibilities of quantifying the various sources of uncertainty. As can be seen from the table, it is possible to quantify five of the ten sources of uncertainty discussed in this paper. For the other five sources, quantification of uncertainty is not possible at the current stage of knowledge. This means that any estimate of the uncertainty associated with the benefits of a road safety programme will be incomplete and understate the true uncertainty. Uncertainty attributable to random variation of the number of injured road users influenced by a road safety measure can be quantified according to closed-form expressions developed by Bijleveld (2005). Hauer and Hakkert (1988) give formulas for estimating the uncertainty associated with incomplete accident reporting. Random variation in the effects of a road safety measure can be quantified either by relying on statistical weights assigned to studies in meta-analysis or by applying formulas given by Hauer (1997). If there is reason to believe that there are unknown sources of systematic variation in effects, uncertainty attributable to this can be quantified either by relying on random effects weights in metaanalysis, or by applying the approach of Ye and Lord (2009). Finally, uncertainty with respect to the monetary valuation of safety benefits can be estimated by relying on, for example, the standard errors reported by de Blaeij et al. (2003). Uncertainty with respect to the definition of the target group of accidents or injuries influenced by a certain road safety measure
is basically not of a statistical nature and is not sufficiently wellknown at present to be quantified. While examples can be given of this source of uncertainty, no systematic knowledge exists. A similar point of view applies to uncertainty regarding the duration or stability over time of the effects of a road safety measure. Again, examples of a lack of stability can be given, but there is no systematic knowledge. As far as the combined effects of several measures are concerned, there is reason to believe that the effects of road safety measures may indeed be modified when combined with other measures. However, knowledge is too sketchy and incomplete to support any meaningful quantification of uncertainty. Uncertainty regarding the effects of exogenous factors and forecasts of safety is likely to be large, but not possible to quantify with any confidence. The same point of view applies to uncertainty regarding the extent to which road safety programmes are implemented. 6. Case illustrations of quantifying uncertainty 6.1. Variance of the number of victims Three examples will be given to show how uncertainty can be quantified. The first example deals with variance of the number of accident victims. In the numerical example given in Section 3, 189 road users were injured annually in a total of 460 junctions that were considered for conversion to roundabouts. On the average, 1.4 road users are injured in each injury accident in junctions. It can thus be estimated that the 189 injured road users were injured in 189/1.4 = 135 injury accidents. It may be convenient to represent the excessive number of victims per accident, 0.4 (1.4–1.0) as a Poisson variable with a mean of 0.4. It can then be estimated that there will be a single victim (i.e. no excessive victims) in 90.2 accidents, 2 victims in 36.4 accidents, 3 victims in 7.3 accidents and 4 victims in 1 accident. 90 victims are expected to be injured in accidents involving only 1 victim, 73 victims in accidents involving 2 victims, 22 victims in accidents involving 3 victims and 4 victims in accidents involving 4 victims. According to Bijleveld (2005), the variance of the number of victims can be estimated by: Var(V ) =
N
v21
i=1
N is the number of accidents, vi is the number of victims in the ith accident. In the present case, variance becomes: (90.2 × 1) + (36.4 × 4) + (7.3 × 9) + (1.0 × 16) = 317 The first parenthesis is the number of accidents with 1 victim per accident. The second parenthesis is the number of accidents with 2 victims per accident; 4 = 22 . The third parenthesis represents accidents with 3 victims; the fourth parenthesis is accidents with 4 victims. The number of victims in total is 189 and the variance of this number is 317—rather more than what the simple Poisson assumption would imply (189). 6.2. Accounting for sources of variance in estimate of effect of a road safety measure In an analysis of road safety policy in Sweden, Elvik and Amundsen (2000) compared three summary estimates of the effect on accidents of traffic calming: 1. One estimate accounting only for random variation in effect, obtained by using fixed-effects statistical weights in a metaanalysis of 32 studies that evaluated the effects in accidents of traffic calming,
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
2177
Table 1 Possibilities of quantifying sources of uncertainty in estimated benefits of road safety programmes. Source of uncertainty
Possibility for quantification
1: Random variation in target group
Can be quantified using Poisson assumption for accidents and the estimator developed by Bijleveld (2005) for accident victims Can be quantified using estimators developed by Hauer and Hakkert (1988) Cannot be quantified at the current stage of knowledge Can be quantified by relying on fixed-effects statistical weights for summary estimates based on meta-analysis or by applying estimators developed by Hauer (1997) for single studies Can be quantified by relying on random-effects statistical weights for summary estimates based on meta-analysis or by applying estimators developed by Ye and Lord (2009) for single studies Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Can be quantified by relying on standard errors reported in meta-analysis by de Blaeij et al. (2003)
2: Incomplete accident reporting 3: Uncertain definition of target group 4: Random variation in effect of measures
5: Systematic variation in effect of measures
6: Uncertain duration of effects 7: Uncertainty in combined effects 8: Uncertain effects of exogenous factors 9: Uncertain degree of implementation 10: Uncertain monetary valuation
2. One estimate accounting both for random variation and unknown sources of systematic variation in effects, obtained by relying on random-effects weights the same meta-analysis of 32 studies, 3. One estimate accounting for (a) random variation in effects; (b) unknown sources of systematic variation in effects and (c) incomplete accident reporting, obtained by relying on a study (Elvik and Mysen, 1999) showing the mean level of accident reporting in the countries that had contributed the studies that were included in the meta-analysis. Accounting for random variation in effects only, the summary accident modification factor for injury accidents was 0.855 (i.e. 14.5% accident reduction) with a 95% confidence interval from 0.881 to 0.830 (length 0.051). When in addition systematic variation in effects was accounted for, the summary accident modification factor remained 0.855, but the 95% confidence interval now spanned from 0.900 to 0.813(length 0.087). Finally, accounting for incomplete accident reporting resulted in a summary accident modification factor of 0.853 and a 95% confidence interval from 0.935 to 0.778 (length 0.157). 6.3. Uncertainty in the summary estimate of effect of a road safety programme
Final number of fatalities (“after treatment”) = 138 Statistical weight =
1 = 93.55 (1/285) + (1/138)
Inverse of statistical weight = variance =
1 1 + = 0.0108 285 138
We now have the following input values to estimate uncertainty with respect to the effect of this policy option on the number of fatalities: Number of fatalities affected by policy option (A): 285 Variance of number of fatalities affected by policy option (Var(A)): 335 Estimated effect of policy option (B): 0.5128 Variance of effect of policy option (Var(B)): 0.0108 Number of fatalities prevented by policy option (A × B): 147 Estimate of variance of number of fatalities prevented: [(0.5128 × 0.5128) × 335] + [(285 × 285) × 0.0108] = 962.71 The standard error of the number of fatalities prevented is the square root of the variance, which equals 31.0. Thus, for policy option 1, a 95% prediction interval for the number of fatalities prevented is: 147 ± 1.96 × 31.0 = 147 ± 60.8 = lower limit = 86; upper limit = 208 7. Discussion
The uncertainty of the summary effect of a road safety programme is a function of all the sources of uncertainty discussed in this paper. As noted above, it is at the current stage of knowledge not possible to quantify the contribution of all these sources. It is, however, possible to provide a lower bound for the uncertainty of the summary effect of a road safety programme. Such an estimate was provided in a recent road safety impact assessment for Norway (Elvik, 2007). The reference value for the number of fatalities – the number expected to occur in 2020 if no new road safety measures are introduced – is 285. The variance of this number is about 335. The variance of the estimated reduction of the number of fatalities in each policy option can be estimated by applying a fixed effects weight to the estimate, the way this is done in meta-analysis. The variance of an estimate of effect is the inverse value of the statistical weight assigned to that estimate. If, for the purposes of gaining an impression of the uncertainty, the estimated, predicted numbers of fatalities are treated as if they were observed numbers, we get for policy option 1, optimal use of road safety measures: Initial number of fatalities (“before treatment”) = 285
Road safety policy is made in great uncertainty. It is not always perfectly clear which group of accidents or road users a road safety measure influences. Estimates of the effects of road safety measures are always uncertain; the more so when several measures are combined in a programme. The monetary valuation of the benefits of road safety measures is highly uncertain; thus it may not be possible to determine the optimal level of safety. No road safety programme can influence road safety more than marginally; road safety is influenced by a host of factors that are beyond the control of any national, not to say local, government. All these observations are perfectly obvious. It may therefore seem surprising that uncertainty is hardly mentioned in road safety programmes and that no attempt is made to identify the most important sources of uncertainty. Yet, on closer reflection, the aversion to an explicit treatment of uncertainty is perfectly understandable, as an adequate treatment of uncertainty renders decision making considerably more complex. Consider the case of a planner faced by the choice between two measures: a bypass road and road lighting. Assume that both measures pass a benefit–cost test, with benefit–cost ratios of, respectively, 1.05 and
2178
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
1.25. The planner may opt for road lighting, since it has the higher benefit–cost ratio. However, once road lighting is introduced, the number of accidents is reduced and the benefit–cost ratio of the bypass road now drops below 1.0. Besides, the 95% confidence interval for the effect of road lighting is wider than for the effect of a bypass road. Hauer and Geva (1984) show how the value of obtaining more precise estimates of the effect of road safety measures can be estimated to support decisions about whether to start research or not. The analytical framework they present is very useful, but in practice it is impossible to predict the result of a new study: if it differs greatly from previous studies, it could make the confidence interval wider, not smaller. It is a lot easier to ignore all these complexities and proceed as if they did not exist. The simplicity of doing so is, however, entirely illusory. Uncertainties exist even if ignored. Uncertainties that are ignored may turn up as nasty surprises later on. The first step towards avoiding this is to explicitly recognise uncertainty and determine how it can be reduced. Five of the ten sources of uncertainty that have been discussed can be quantified at present; the other five remain difficult to quantify. Is there any prospect of quantifying these sources of uncertainty? It should be possible to quantify at least three of the five sources of uncertainty so far not quantified. Uncertainty with respect to the effects of exogenous factors (i.e. all other factors influencing road safety in addition to the programme) can be approximately quantified by analysing the prediction errors associated with past predictions of changes in road safety. Uncertain duration or stability over time of the effects of a road safety measure is also a topic which is amenable to research. As an example, so called cumulative meta-analysis studies how summary estimates of effect change over time as new studies are added. This makes it possible to determine whether there is a trend over time for effects to become larger or smaller. Uncertainty about the degree to which a road safety programme is implemented can be estimated by studying historical records showing the degree to which programmes were implemented in the past and variability with respect to the degree of implementation. The two remaining sources of uncertainty, about the definition of the target group of accidents, and about the combined effects of measures, present greater challenges. The first of these two is not statistical in its nature. Still, a properly designed evaluation study can test for spill-over effects or migration effects suggesting that a measure may have an effect – often adverse – on accidents it was not intended to influence. Research on the combined effects of measures is still in its infancy. It may be difficult to perform very rigorous studies about the combined effects of measures. It requires quite detailed records of which measures have been implemented; often such records are not kept. Moreover, if implementation of a programme takes time, confounding factors increasingly influence safety outcomes and need to be controlled for.
8. Conclusions The main results of the study reported in this paper can be summarised as follows: 1. The targets for improving road safety as part of national road safety programmes are not always realised. One reason for this may be that sources of uncertainty in the estimated impacts of such programmes are overlooked.
2. Ten sources of uncertainty in estimated benefits of national road safety programmes are identified and described. It is concluded that five of these sources can be quantified, whereas too little is known to quantify the other five sources of uncertainty. 3. Uncertainty with respect to the impacts of road safety programmes is rarely estimated, but is likely to be considerable. 4. More research on sources of uncertainty in road safety programmes may lead to the quantification of more sources of uncertainty, but it is unlikely that all sources of uncertainty can ever be meaningfully quantified. References Bijleveld, F.D., 2005. The covariance between the number of accidents and the number of victims in multivariate analysis of accident related outcomes. Accident Analysis and Prevention 37, 591–600. de Blaeij, A., Florax, R.J.G.M., Rietveld, P., Verhoef, E., 2003. The value of statistical life in road safety: a meta-analysis. Accident Analysis and Prevention 35, 973–986. Broughton, J. 1987. The effect on motorcycling of the 1981 Transport Act. TRRL Research Report 106. Transport and Road Research Laboratory, Crowthorne, Berkshire. Elvik, R., 1996. Does prior knowledge of safety effect help to predict how effective a measure will be? Accident Analysis and Prevention 28, 339–347. Elvik, R., Amundsen, A., 2000. Improving road safety in Sweden. Main Report. Report 490. Institute of Transport Economics, Oslo. Elvik, R., 2007. Prospects for Improving Road Safety in Norway. Report 897. Institute of Transport Economics, Oslo. Elvik, R., 2009a. Developing accident modification functions. An exploratory study. Transportation Research Record 2103, 18–24. Elvik, R., 2009b. An exploratory analysis of models for estimating the combined effects of road safety measures. Accident Analysis and Prevention 41, 876–880. Elvik, R., 2010. The stability of long-term trends in the number of traffic fatalities in a sample of highly motorised countries. Accident Analysis and Prevention 42, 245–260. Elvik, R., Mysen, A.B., 1999. Incomplete accident reporting. A meta-analysis of studies made in 13 countries. Transportation Research Record 1665, 131–140. Elvik, R., Christensen, P., Fjeld-Olsen, S., 2003. Daytime Running Lights. A Systematic Review of Effects on Road Safety. Report 688. Institute of Transport Economics, Oslo. Elvik, R., Kolbenstvedt, M., Elvebakk, B., Hervik, A., Bræin, L., 2009. Costs and benefits to Sweden of Swedish road safety research. Accident Analysis and Prevention 41, 387–392. Færdselssikkerhedskommissionen, 1988. Færdselssikkerhedspolitisk handlingsplan. Betænkning 1157. Statens Informationstjeneste, København. Fridstrøm, L., Ifver, J., Ingebrigtsen, S., Kulmala, R., Thomsen, L.K., 1995. Measuring the contribution of randomness, exposure, weather, and daylight to the variation in road accident counts. Accident Analysis and Prevention 27, 1–20. Hauer, E., 1997. Observational Before–After Studies in Road Safety. Pergamon Press, Oxford. Hauer, E., Geva, I., 1984. A decision analytic approach to the design of experiments in research on traffic safety. In: Yagar, S. (Ed.), Transport Risk Assessment, vol. 155–173. University of Waterloo Press, Waterloo, Ontario. Hauer, E., Hakkert, A.S., 1988. Extent and some implications of incomplete accident reporting. Transportation Research Record 1185, 1–10. Palisade Corporation, 2009. Guide to Using @RISK. Risk Analysis and Simulation Add-in for Microsoft Excel. Palisade Corporation, Ithaca. Rasmussen, R.E.H., 1964. Elementær Måleteori. Fjerde Oplag. Jul Gjellerups Forlag, København. Samferdselsdepartementet, 1973. Stortingsmelding 107 (1972–1973). Om revidert Norsk Vegplan. Statens forvaltningstjeneste, Oslo. Samferdselsdepartementet, 1981. Stortingsmelding 80 (1980–1981). Om Norsk Vegplan 1982–1985. Statens Forvaltningstjeneste, Oslo. Samferdselsdepartementet, 1997. Stortingsmelding 37 (1996–1997). Norsk veg- og vegtrafikkplan 1998–2007. Statens Forvaltningstjeneste, Oslo. Samferdselsdepartementet, 2000. Stortingsmelding 46 (1999–2000). Nasjonal transportplan 2002–2011. Statens Forvaltningstjeneste, Oslo. Strand, S., 1987. Usikkerhet i Data og Modeller. Transportøkonomisk Institutt, Oslo. Sverdrup, E., 1964. Lov og Tilfeldighet. Bind 1. Universitetsforlaget, Oslo. Trafiksäkerhetsdelegationen, 1982. Plan för Trafiksäkerhetsarbetet. Kommittébetänkande 1981:50. Statens Tryckericentral, Helsingfors. Vägverket, 1999. Särskild Trafiksäkerhetsplan. Vägverket, Borlänge. Wanvik, P.O., 2009. Road lighting and traffic safety. Do we need road lighting? Doctoral Theses at NTNU, 2009:66. Norwegian University of Science and Technology, Trondheim. Ye, Z., Lord, D., 2009. Estimating the variance in before–after studies. Journal of Safety Research 40, 257–263.
Contents lists available at ScienceDirect
Accident Analysis and Prevention journal homepage: www.elsevier.com/locate/aap
Sources of uncertainty in estimated benefits of road safety programmes Rune Elvik Institute of Transport Economics, Gaustadalléen 21, NO-0349 Oslo, Norway
a r t i c l e
i n f o
Article history: Received 21 December 2009 Received in revised form 23 February 2010 Accepted 30 March 2010 Keywords: Road safety programme Estimates of effect Uncertainty Propagation of errors
a b s t r a c t National road safety programmes have been developed in many motorised countries. Some of these programmes contain estimates of the safety benefits that were expected to be realised if the programmes were fully implemented. When these estimates are compared to actual outcomes, it is not uncommon to find large differences. This paper argues that differences between the predicted and actual results of road safety programmes could be the result of a large, but generally unrecognised, uncertainty inherent in estimates of the effects of such programmes. Ten sources of uncertainty are identified and briefly described. The possibility of describing these sources of uncertainty numerically, and of estimating their joint contribution, is discussed. It is concluded that at the current state of knowledge, it is not possible to meaningfully estimate the total uncertainty inherent in road safety programmes. The prospects of reducing uncertainty by means of research are discussed. © 2010 Elsevier Ltd. All rights reserved.
1. Introduction Quantified targets for improving road safety and programmes designed to realise these targets have been adopted by many motorised countries in recent years. These programmes do not always produce the expected results. On the contrary, it is commonly found that the actual number of people killed or injured in road accidents exceeds the numbers that were predicted according to the road safety programme. Road safety is influenced by very many factors and no road safety programme can hope to influence all of them. Road safety is therefore only to some extent controllable by means of government programmes. Besides, the effects of a programme on road safety are highly uncertain—more so than is generally recognised. This paper is a first attempt to study the following questions: 1. What are the principal sources of uncertainty in the estimated benefits of national road safety programmes? 2. Is it possible to quantify all sources of uncertainty? 3. Is it possible to estimate the joint contribution of all sources of uncertainty, i.e. the total uncertainty associated with estimates of the benefits of a road safety programme? The term “benefits” refers to the monetary benefits of a road safety programme in terms of reduced costs of accidents and other favourable impacts. Very often, the principal focus of interest is on effects stated in terms of changes in the number of people killed
E-mail address: [email protected]. 0001-4575/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.aap.2010.03.022
or injured. These impacts are the principal focus of interest in this paper as well. The reason for including the monetary valuation of impacts in the analysis will be explained later. 2. Examples of erroneous estimates of effects of road safety programmes It is easy to give examples of road safety programmes that have not resulted in the safety benefits they were expected to produce. Fig. 1 gives one example. It shows road accident fatalities in Denmark from 1989 to 2000 compared to the targeted development according to a road safety programme proposed in 1988 (Færdselssikkerhedskommissionen, 1988). During the first years, outcomes were close the road safety programme, but then progress slowed down. At the end of the period, the actual number of road accident fatalities clearly exceeded the number expected to occur according to the road safety programme. Similar examples can be given for Finland (Trafiksäkerhetsdelegationen, 1982) and Sweden (Vägverket, 1999). For Norway, on the other hand, the number of fatalities has developed more favourably than predicted in the National Transport Plan for the 2002–2011 term (Samferdselsdepartementet, 2000). Thus, although too optimistic predictions appear to be more common than too pessimistic, examples of both these errors can be found. Since erroneous predictions of the impacts of road safety policy presumably are not intentional, these errors must be attributable to sources of uncertainty in estimates of programme impacts that were either unrecognised or erroneously estimated when the programme was prepared. Wishful thinking about the impacts of road safety measures cannot be ruled out. However, wishful thinking is not deliberate. In general, victims of it are
2172
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
unaware of the fact that they are being too optimistic. A victim of wishful thinking may nevertheless be inclined to resist corrections, since these imply that the outcome wished for is more difficult to attain. In the following, sources of uncertainty in the estimated benefits of road safety programmes are identified, the possibilities of analysing them statistically are discussed and examples are given of how to estimate uncertainty. The prospects of reducing uncertainty are discussed. 3. A taxonomy of sources of uncertainty There are several sources of uncertainty in the estimated benefits of national road safety programmes. The following ten sources of uncertainty have been identified: 1. Random variation in the recorded number of accidents or injuries in the target group of a road safety measure; 2. Incomplete or inaccurate reporting of accidents or injuries in official accident statistics; 3. Uncertainty about the definition of the target group of accidents or injuries influenced by a road safety measure; 4. Random variation in the effects of a road safety measure on accidents or injuries; 5. Unknown sources of systematic variation in the effects of a road safety measure on accidents or injuries; 6. Unknown duration or stability over time of the effects of a road safety measure; 7. Uncertainty with respect to if and how the first order effects of a road safety measure are modified when it is combined with other road safety measures; 8. Uncertainty with respect to assumptions made about the effects of exogenous factors influencing road safety; 9. Uncertainty about the degree to which planned road safety measures will actually be implemented; 10. Uncertain monetary valuation of the benefits of road safety measures. Each of these sources of uncertainty will be briefly described. 3.1. Random variation in the target number of accidents or injuries influenced by each road safety measure A road safety programme will normally consist of a number of road safety measures. Each of these measures is intended to influence a certain target group of accidents or injuries. As an example, converting junctions to roundabouts will influence the number of accidents at junctions. To estimate the impacts of this on road safety, one must first estimate how many accidents or injuries will be influenced by converting a certain number of junctions to roundabouts. One way of doing this is to apply the following model: Expected number of injured road users influenced = Number of junctions converted × Mean traffic volume × Mean injury rate Thus, in a recent road safety impact assessment for Norway (Elvik, 2007), it was assumed that conversion of junctions to roundabouts would start by converting junctions with a high traffic volume and continue with junctions with less traffic. It was estimated that it would be cost-effective (i.e. marginal benefits would exceed marginal costs) to convert 460 three leg junctions to roundabouts. The number of injured road users expected to be influenced by this was estimated as: 460 × 12349 × 365 × 10−6 × 0.091 = 189
Fig. 1. Planned and actual number of fatalities according to Danish road safety programme 1988–2000.
The first four terms represent exposure; the last term is the mean injury rate per million entering vehicles in three leg junctions. In principle, there are at least three sources of uncertainty in this estimate: 1. Uncertainty with respect to how junctions that are candidates for conversion to roundabouts are distributed according to traffic volume, 2. Uncertainty with respect to the mean injury rate for three leg junctions before conversion to roundabouts, 3. Random variation in the actual number of injured road users in junctions that are converted to roundabouts. If the number of injured road users is assumed to be a Poisson variable, the variance equals the expected number (in this case estimated as 189); the standard error is the square root of the variance. However, as shown by Fridstrøm et al. (1995) and Bijleveld (2005), the Poisson assumption is strictly speaking only valid for accidents, not for injured road users, since there must by definition be at least one injured person in each injury accident, and since injuries are not independent. Hence, the number of injured road users in a sample of accidents that occur at random will display over-dispersion. 3.2. Incomplete or inaccurate accident reporting in official statistics The estimate given above of the number of injured road users that could be expected to be influenced by the introduction of a road safety measure applied an injury rate that was based on official accident statistics. However, accident reporting is incomplete and inaccurate in all countries (Elvik and Mysen, 1999). The true number of injured road users influenced by a road safety measure will therefore be greater than the recorded number. Hauer and Hakkert (1988) have shown how one can estimate the true number of accidents and the uncertainty of this estimate, provided the level of accident reporting and the variance of the level of accident reporting are known. Unfortunately, this knowledge is rarely likely to be available at the level of detail that is required for meaningful use of the corrections described by Hauer and Hakkert. Besides, the purpose of the models developed by Hauer and Hakkert is to estimate the true number of accidents, whereas what we would like to know is how incomplete accident reporting adds to the uncertainty of the recorded number of accidents. The reason why this is what we need to know, is that an assessment of the success or failure of a road safety programme will most likely be based on the recorded number of
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
accidents, not on an estimate of the unknown true number of accidents. As an approximation, one may assume that the addition to uncertainty in the recorded number of accidents attributable to incomplete accident reporting is proportional to the ratio: (variance of estimated true number of accidents/estimated true number of accidents). 3.3. Uncertain definition of target group of accidents or injuries Most road safety measures are intended to influence a clearly defined target group of accidents or injuries. In some cases, however, the group of accidents actually influenced by a measure will not be identical to the formally defined target group. In such cases, prediction of effect based on a formally defined target group may deviate from the effect actually observed, normally by being too optimistic. A case in point is the study of changes in motorcycle legislation in Great Britain by Broughton (1987). He evaluated the effects of banning learner motorcyclists from using motorcycles with an engine volume of more than 125 cubic centimetres. Before the ban was introduced, accidents involving learner motorcyclists riding large motorcycles represented about 13% of all injured motorcyclists and about 31% of all learner motorcyclists injured. If the ban was perfectly complied with, one would expect these accidents to be eliminated. What happened was that learner motorcyclists switched to smaller motorcycles. Accidents involving learner motorcyclists riding small motorcycles increased. As a result of this, the total number of accidents among learner motorcyclists was reduced by only 8%. This reduction was actually smaller than the reduction of accidents among experienced motorcyclists in the same period (11%). A related case is presented by Wanvik (2009). His study evaluated the effect on accidents of road lighting. It has traditionally been assumed that road lighting only influences accidents in darkness; hence, accidents in daylight may serve as a comparison group in a before-and-after study. Relying on this assumption and using data for Dutch motorways, Wanvik estimated the effect of road lighting to an impressive 58% reduction of accidents in darkness. Lighting poles represent a new traffic hazard on lit roads. This particular hazard does not exist on unlit roads. Accidents in which vehicles strike lighting poles may occur all day and therefore influence the number of daytime accidents. When the estimate of effect was adjusted for this, the effect attributed to road lighting on Dutch motorways was reduced to a 51% reduction of accidents in darkness. 3.4. Random variation in effects of road safety measures For most road safety measures that are included in national road safety programmes, there will be multiple estimates of effect based on several evaluation studies. Summary estimates of effect based on meta-analysis will sometimes be available. In that case, uncertainty attributable to random variation in the effects of a road safety measure can be estimated by taking the inverse value of the fixedeffects statistical weight of the summary estimate of effect. This will indicate the purely random variation in effects around the summary effect. If summary estimates of effect based on meta-analysis are not available, one may estimate the variance of an estimate of effect based on a single study by applying the techniques explained by Hauer (1997). These approaches will not always be applicable, in particular not with respect to new road safety measures for which there may not be any evaluation studies that show effects on accidents and the uncertainty of those effects.
2173
3.5. Unknown sources of systematic variation in effects of road safety measures The effects of many road safety measures are likely to vary, depending on, for example, the standard of the measure (good road lighting is likely to be more effective than poor road lighting), the extent of its use (more police enforcement may be more effective than a little police enforcement), or the context in which it is introduced (a bypass road may be more effective in a small town than in a large town). Unfortunately, little is known about the sources and patterns of systematic variation in the effects of road safety measures. A fairly common situation may therefore be that there is known to be systematic variation in the effects of a road safety measure, as indicated for example by statistical tests made as part of meta-analyses, but that sources of this variation, and how best to model it, are unknown. This may lead to erroneous estimates of the effects of a road safety measure. Fig. 2 shows a case in point. It is based on a paper by Elvik (2009a) in which accident modification functions for the effects of bypass roads were developed. These functions were based on three evaluation studies reported in Denmark, Norway and Sweden, employing broadly speaking the same study design. The effects of bypass roads were found to vary according to the size of the population of the bypassed town. If the variation in the effects of bypass roads had been unknown, the effect of building a new bypass road would have been predicted by applying a constant accident modification factor, shown by the straight line in Fig. 2. It can be seen that this would have given too pessimistic predictions for small towns and to optimistic predictions for larger towns. Fitting an accident modification function allows the predicted effect of a new bypass road to vary according to the size of the town. Yet, even these predictions are uncertain, as indicated in Fig. 2 by the dotted curves showing the most optimistic and the most pessimistic of the accident modification functions that were developed in the study. 3.6. Unknown duration or stability over time of the effects of a road safety measure Do the effects of a road safety measure gradually disappear as time goes by after its introduction? Or do the effects of introducing a road safety measure change over time? This may sound as two versions of the same question, but it is actually two different questions. The first question is whether the effect on accidents of, say, a roundabout are the same 10 years after it was built as during the first year. The second question is whether new roundabouts built today have larger or smaller effects during their first year than roundabouts built fifteen years ago. Very little is known about these questions. There is, however, scattered evidence from a few studies suggesting that the effects of a certain road safety measure may indeed change over time. Elvik et al. (2003) reviewed studies of the effects of daytime running lights on cars. Five studies permitted these effects to be evaluated for different lengths of the after-period (i.e. after the use of daytime running lights became mandatory). Three of the five studies indicated that effects declined over time; one study was inconclusive; one study indicated that effects increased over time. On balance, these studies suggest that it is more likely that effects are weakened over time than that they are strengthened. A second example is given in Fig. 3. Fig. 3 shows the summary estimates of converting junctions to roundabouts based on evaluation studies reported in Norway. The first estimate is based on the first study reported in Norway; the second on the first two studies, and so on. At first, the effects on injury accidents of converting junctions to roundabouts seemed to increase. From the first five studies
2174
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
Fig. 2. Systematic variation in effects on accidents of bypass roads. Source: Elvik (2009a).
onwards, however, effects have become smaller as each new study has been reported. The addition of the ninth study to the sample of studies was even associated with an increase of the size of the confidence interval for the summary estimate of effect. The width of the 95% confidence interval for the summary estimate of effect based on the first eight studies was 0.157 (0.557 minus 0.400). When the ninth study was added, the width of the confidence interval increased to 0.166. Apparently, it is not always the case that adding new studies makes estimates of effect more precise. To statistically estimate and account for the uncertainty resulting from variation of the effects of road safety measures over time, it is necessary to predict the future effects of road safety measures. This cannot be done with any confidence, as shown by Elvik (1996). 3.7. Uncertainty regarding the combined effects of several measures A national road safety programme normally consists of a large number of road safety measures. A recent road safety impact assessment for Norway, for example (Elvik, 2007), included 45 road safety measures. A problem that needs to be addressed in any road safety programme is how best to estimate the combined effects of the
measures that form a programme. This cannot be done simply by adding the estimates of effect developed for each road safety measure. If three road safety measures influence the same target group of accidents, and are expected to reduce accidents by 50%, 40% and 30%, their combined effects cannot possibly be 120%. Effects combine multiplicatively, not additively. In a recent paper, Elvik (2009b) reviewed the very few studies that provide evidence about the combined effects of several road safety measures. Two models were developed to statistically estimate the combined effects of road safety measures. One of these models, labelled the common residuals model, assumes that the percentage effect of a road safety measure remains unchanged when the measure is combined with one or more other measures. Another model, labelled the dominant common residuals model, assumed that in any set of road safety measures, there will be one or more measures that are “dominant” in the sense that once they have been introduced, the effects of the remaining measures are weakened. Exploratory analyses indicated that both these models can be defended and that no model appears to be clearly superior to the other. In general, the dominant common residuals model will give more conservative estimates of the combined effects of road safety measures than the common residuals model. There will, accord-
Fig. 3. Summary estimates of effects on injury accidents of converting junctions to roundabouts.
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
ingly, always be uncertainty regarding the combined effects of measures that form a road safety programme. 3.8. Uncertainty regarding the effects of exogenous factors A road safety programme will typically apply to a period of 5–15 years. Although the programme may significantly influence road safety during this period, it will never be the only factor that influences road safety and may not be the most important factor. The outcomes that are observed in terms of the count of fatalities, injured road users and accidents, will be the result of all factors that influence road safety, not just the road safety programme. When developing a road safety programme, it is essential to identify as precisely as possible the influence it will have on road safety, given the effects of all other factors influencing road safety. This is sometimes done by developing a so called baseline scenario, in which road safety is predicted assuming that the programme is not introduced. Once a baseline forecast has been made, the difference a road safety programme can make is estimated. Although this sounds analytically straightforward, it is actually almost impossible to do in practice. Ideally speaking, the baseline scenario is intended to show what will happen if no road safety programme is introduced. Past trends in road safety give no information about this. Past trends reflect all factors that have influenced road safety, including all road safety measures that were introduced in the past. Hence, projecting past trends to the future does not show what might happen in a baseline scenario; it rather shows what might happen if road safety measures (and other factors) continue to be introduced at the same rate as in the past and continue to be equally effective. Projecting past trends is a notoriously unreliable method for predicting future changes (Elvik, 2010). Trying to identify factors that explain past trends does not help very much; in fact it makes prediction considerably more complex, since the explanatory factors have to be predicted in order to predict safety outcomes. To add to complexity, it is even conceivable that the effects of some exogenous factors changes over time. As an example, in the analysis reported by Elvik (2010), the sign of the coefficient representing the effect of GDP per capita on traffic fatalities changed in Great Britain and the Netherlands during the period 1970–2007, depending on which years were included in the analysis (1970–1989, 1970–1999 or 1970–2007). While some factors that have influenced past trends may be identified (Elvik et al., 2009), this cannot be done in a very rigorous manner and may be of little help in predicting future changes. 3.9. Uncertain degree of implementation of road safety measures Road safety programmes are almost never fully implemented. Estimates of their effects assume full implementation. When implementation is less than one hundred percent, effects on road safety will be reduced accordingly. In Norway, a system for long-term planning of road investments was created around 1970; the basic elements of this system are still in place. Fairly detailed investment programmes are developed every four years. At the end of the period, an assessment is made of the extent to which the investment programme has been implemented. The first of these assessments, referring to the 1970–1973 planning term, was reported in 1973 (Samferdselsdepartementet, 1973). It showed that 96.6% of planned investments had been implemented. The degree of implementation has subsequently declined. For the 1978–1981 term (Samferdselsdepartementet, 1981), it was down to 92.9%. For the 1994–1997 planning term (Samferdselsdepartementet, 1997), 92.7% of planned investments actually took place, and only 81% of the targeted effect on accidents was realised.
2175
Estimates of effect obviously tend to be based on an assumption of 100% implementation. This will in nearly all cases be too optimistic, although the actual degree of implementation of a plan can never be predicted with much confidence. 3.10. Uncertain monetary valuation of impacts of road safety measures All sources of uncertainty discussed so far, refer to uncertainty about the impacts of a road safety programme stated in terms of changes in the number of accidents or the number of road users killed or injured. The title of the paper, however, refers to the benefits of road safety programmes, not to their effects on accidents or injuries. What is the difference, and why has the term benefits been introduced? The difference between the impacts of a road safety programme and the benefits of the programme is that the latter term refers to the conversion of all impacts of the programme to monetary terms. It is not uncommon that cost–benefit analyses of road safety measures are performed as part of the development of a road safety programme, in order to identify the road safety measures that are most cost-effective in improving road safety. The priorities given to the various road safety measures may not always be based strictly on the results of cost–benefit analyses. In principle, however, cost–benefit analyses can be used to determine the optimal use of each road safety measure and the optimal overall level of road safety. If policy makers have an ambition of using cost–benefit analyses to determine the optimal level of safety, uncertainty about the monetary valuation of impacts becomes crucial. The monetary valuation both of road safety impacts, as well as other relevant impacts of a programme, like impacts on travel time, is highly uncertain (de Blaeij et al., 2003). It is therefore not possible to determine the optimal level of safety very precisely, if at all. Depending on how uncertainty with respect to monetary valuation is treated analytically, there can be a narrow or a broad range of outcomes that will be regarded as representing an optimal level of safety. The optimal size of the budget may also vary considerably depending on the monetary valuations adopted. 4. The basic model for the propagation of errors The problem facing a policy maker who wants to quantify the uncertainty associated with the impacts and benefits of a road safety programme is how to estimate the combined contribution of several sources of uncertainty. A general answer to this problem can be found in statistics, in which the basic model for the propagation of errors has been developed (Rasmussen, 1964; Sverdrup, 1964). Assume that the outcome of interest can be modelled as a function of several variables: Y = f (X, Z, . . . , W )
(1)
Please note that all elementary arithmetical operations (adding, subtracting, multiplying, dividing), can be treated as functions. A random variable is also, by definition, a function. If the variance of each term entering a function (e.g. each of two numbers that are added) is independent of the variance of the other terms, the variance of the outcome equals:
Var(R) =
∂R ∂X
2
Var(X) +
∂R ∂Y
2
Var(Y ) + · · · +
∂R ∂W
2
Var(W )
(2)
In Eq. (2), ∂R/∂X. . . denote the partial derivatives of the function R with respect to the variables X, Y, . . ., W. Var(X, Y, . . ., W) is the variance of each of the variables entering the function. It is well known (Strand, 1987) that if the variances of the independent variables entering a function are correlated, the model for the propagation of errors becomes considerably more complicated. In this
2176
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
paper, only the elementary form of the model will be considered. This does not necessarily mean that it is the most correct model, simply that current knowledge does not justify the use of models that include terms representing the correlations between variables. Estimating the benefits of a road safety programme can be modelled as a set of successive multiplications. At the first stage, the number of road users influenced by a road safety measure is obtained by multiplying exposure and injury rate for each level of implementation of each road safety measure. That level could, as an example, be to convert all junctions to roundabouts if the benefits of doing so exceed the costs. The next stage is to estimate the number of injuries prevented by a measure, by multiplying the number of injured road users in the target group by an accident modification factor or accident modification function. The economic benefit associated with each measure is obtained by multiplying the number of injuries prevented at each level of severity by the appropriate monetary valuation of preventing those injuries. Finally, the combined effects of a set of measures can also be obtained by multiplying the “residual factors” of the measures, i.e. the fraction of injuries in the target group that is not prevented by a measure. In the multiplication R = A × B, the partial derivatives are: ∂R =B ∂A
and
∂R =A ∂B
A and B are interpreted as random variables. This makes estimation of compound uncertainty easy, by proceeding in stages term by term. The statistical software @RISK (Palisade Corporation, 2009) permits the combined uncertainty resulting from multiple sources to be estimated based on the specification of a probability distribution applying to each of the sources of uncertainty. Users of the software can choose from a total of 37 probability distributions. An informative use of this software presumes, however, that each source of uncertainty is sufficiently well known to be described in terms of a certain probability distribution. 5. Quantifying each source of uncertainty To what extent is it possible to meaningfully quantify each of the sources of uncertainty discussed in Section 3 of the paper? Table 1 gives a summary of the possibilities of quantifying the various sources of uncertainty. As can be seen from the table, it is possible to quantify five of the ten sources of uncertainty discussed in this paper. For the other five sources, quantification of uncertainty is not possible at the current stage of knowledge. This means that any estimate of the uncertainty associated with the benefits of a road safety programme will be incomplete and understate the true uncertainty. Uncertainty attributable to random variation of the number of injured road users influenced by a road safety measure can be quantified according to closed-form expressions developed by Bijleveld (2005). Hauer and Hakkert (1988) give formulas for estimating the uncertainty associated with incomplete accident reporting. Random variation in the effects of a road safety measure can be quantified either by relying on statistical weights assigned to studies in meta-analysis or by applying formulas given by Hauer (1997). If there is reason to believe that there are unknown sources of systematic variation in effects, uncertainty attributable to this can be quantified either by relying on random effects weights in metaanalysis, or by applying the approach of Ye and Lord (2009). Finally, uncertainty with respect to the monetary valuation of safety benefits can be estimated by relying on, for example, the standard errors reported by de Blaeij et al. (2003). Uncertainty with respect to the definition of the target group of accidents or injuries influenced by a certain road safety measure
is basically not of a statistical nature and is not sufficiently wellknown at present to be quantified. While examples can be given of this source of uncertainty, no systematic knowledge exists. A similar point of view applies to uncertainty regarding the duration or stability over time of the effects of a road safety measure. Again, examples of a lack of stability can be given, but there is no systematic knowledge. As far as the combined effects of several measures are concerned, there is reason to believe that the effects of road safety measures may indeed be modified when combined with other measures. However, knowledge is too sketchy and incomplete to support any meaningful quantification of uncertainty. Uncertainty regarding the effects of exogenous factors and forecasts of safety is likely to be large, but not possible to quantify with any confidence. The same point of view applies to uncertainty regarding the extent to which road safety programmes are implemented. 6. Case illustrations of quantifying uncertainty 6.1. Variance of the number of victims Three examples will be given to show how uncertainty can be quantified. The first example deals with variance of the number of accident victims. In the numerical example given in Section 3, 189 road users were injured annually in a total of 460 junctions that were considered for conversion to roundabouts. On the average, 1.4 road users are injured in each injury accident in junctions. It can thus be estimated that the 189 injured road users were injured in 189/1.4 = 135 injury accidents. It may be convenient to represent the excessive number of victims per accident, 0.4 (1.4–1.0) as a Poisson variable with a mean of 0.4. It can then be estimated that there will be a single victim (i.e. no excessive victims) in 90.2 accidents, 2 victims in 36.4 accidents, 3 victims in 7.3 accidents and 4 victims in 1 accident. 90 victims are expected to be injured in accidents involving only 1 victim, 73 victims in accidents involving 2 victims, 22 victims in accidents involving 3 victims and 4 victims in accidents involving 4 victims. According to Bijleveld (2005), the variance of the number of victims can be estimated by: Var(V ) =
N
v21
i=1
N is the number of accidents, vi is the number of victims in the ith accident. In the present case, variance becomes: (90.2 × 1) + (36.4 × 4) + (7.3 × 9) + (1.0 × 16) = 317 The first parenthesis is the number of accidents with 1 victim per accident. The second parenthesis is the number of accidents with 2 victims per accident; 4 = 22 . The third parenthesis represents accidents with 3 victims; the fourth parenthesis is accidents with 4 victims. The number of victims in total is 189 and the variance of this number is 317—rather more than what the simple Poisson assumption would imply (189). 6.2. Accounting for sources of variance in estimate of effect of a road safety measure In an analysis of road safety policy in Sweden, Elvik and Amundsen (2000) compared three summary estimates of the effect on accidents of traffic calming: 1. One estimate accounting only for random variation in effect, obtained by using fixed-effects statistical weights in a metaanalysis of 32 studies that evaluated the effects in accidents of traffic calming,
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
2177
Table 1 Possibilities of quantifying sources of uncertainty in estimated benefits of road safety programmes. Source of uncertainty
Possibility for quantification
1: Random variation in target group
Can be quantified using Poisson assumption for accidents and the estimator developed by Bijleveld (2005) for accident victims Can be quantified using estimators developed by Hauer and Hakkert (1988) Cannot be quantified at the current stage of knowledge Can be quantified by relying on fixed-effects statistical weights for summary estimates based on meta-analysis or by applying estimators developed by Hauer (1997) for single studies Can be quantified by relying on random-effects statistical weights for summary estimates based on meta-analysis or by applying estimators developed by Ye and Lord (2009) for single studies Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Cannot be quantified at the current stage of knowledge Can be quantified by relying on standard errors reported in meta-analysis by de Blaeij et al. (2003)
2: Incomplete accident reporting 3: Uncertain definition of target group 4: Random variation in effect of measures
5: Systematic variation in effect of measures
6: Uncertain duration of effects 7: Uncertainty in combined effects 8: Uncertain effects of exogenous factors 9: Uncertain degree of implementation 10: Uncertain monetary valuation
2. One estimate accounting both for random variation and unknown sources of systematic variation in effects, obtained by relying on random-effects weights the same meta-analysis of 32 studies, 3. One estimate accounting for (a) random variation in effects; (b) unknown sources of systematic variation in effects and (c) incomplete accident reporting, obtained by relying on a study (Elvik and Mysen, 1999) showing the mean level of accident reporting in the countries that had contributed the studies that were included in the meta-analysis. Accounting for random variation in effects only, the summary accident modification factor for injury accidents was 0.855 (i.e. 14.5% accident reduction) with a 95% confidence interval from 0.881 to 0.830 (length 0.051). When in addition systematic variation in effects was accounted for, the summary accident modification factor remained 0.855, but the 95% confidence interval now spanned from 0.900 to 0.813(length 0.087). Finally, accounting for incomplete accident reporting resulted in a summary accident modification factor of 0.853 and a 95% confidence interval from 0.935 to 0.778 (length 0.157). 6.3. Uncertainty in the summary estimate of effect of a road safety programme
Final number of fatalities (“after treatment”) = 138 Statistical weight =
1 = 93.55 (1/285) + (1/138)
Inverse of statistical weight = variance =
1 1 + = 0.0108 285 138
We now have the following input values to estimate uncertainty with respect to the effect of this policy option on the number of fatalities: Number of fatalities affected by policy option (A): 285 Variance of number of fatalities affected by policy option (Var(A)): 335 Estimated effect of policy option (B): 0.5128 Variance of effect of policy option (Var(B)): 0.0108 Number of fatalities prevented by policy option (A × B): 147 Estimate of variance of number of fatalities prevented: [(0.5128 × 0.5128) × 335] + [(285 × 285) × 0.0108] = 962.71 The standard error of the number of fatalities prevented is the square root of the variance, which equals 31.0. Thus, for policy option 1, a 95% prediction interval for the number of fatalities prevented is: 147 ± 1.96 × 31.0 = 147 ± 60.8 = lower limit = 86; upper limit = 208 7. Discussion
The uncertainty of the summary effect of a road safety programme is a function of all the sources of uncertainty discussed in this paper. As noted above, it is at the current stage of knowledge not possible to quantify the contribution of all these sources. It is, however, possible to provide a lower bound for the uncertainty of the summary effect of a road safety programme. Such an estimate was provided in a recent road safety impact assessment for Norway (Elvik, 2007). The reference value for the number of fatalities – the number expected to occur in 2020 if no new road safety measures are introduced – is 285. The variance of this number is about 335. The variance of the estimated reduction of the number of fatalities in each policy option can be estimated by applying a fixed effects weight to the estimate, the way this is done in meta-analysis. The variance of an estimate of effect is the inverse value of the statistical weight assigned to that estimate. If, for the purposes of gaining an impression of the uncertainty, the estimated, predicted numbers of fatalities are treated as if they were observed numbers, we get for policy option 1, optimal use of road safety measures: Initial number of fatalities (“before treatment”) = 285
Road safety policy is made in great uncertainty. It is not always perfectly clear which group of accidents or road users a road safety measure influences. Estimates of the effects of road safety measures are always uncertain; the more so when several measures are combined in a programme. The monetary valuation of the benefits of road safety measures is highly uncertain; thus it may not be possible to determine the optimal level of safety. No road safety programme can influence road safety more than marginally; road safety is influenced by a host of factors that are beyond the control of any national, not to say local, government. All these observations are perfectly obvious. It may therefore seem surprising that uncertainty is hardly mentioned in road safety programmes and that no attempt is made to identify the most important sources of uncertainty. Yet, on closer reflection, the aversion to an explicit treatment of uncertainty is perfectly understandable, as an adequate treatment of uncertainty renders decision making considerably more complex. Consider the case of a planner faced by the choice between two measures: a bypass road and road lighting. Assume that both measures pass a benefit–cost test, with benefit–cost ratios of, respectively, 1.05 and
2178
R. Elvik / Accident Analysis and Prevention 42 (2010) 2171–2178
1.25. The planner may opt for road lighting, since it has the higher benefit–cost ratio. However, once road lighting is introduced, the number of accidents is reduced and the benefit–cost ratio of the bypass road now drops below 1.0. Besides, the 95% confidence interval for the effect of road lighting is wider than for the effect of a bypass road. Hauer and Geva (1984) show how the value of obtaining more precise estimates of the effect of road safety measures can be estimated to support decisions about whether to start research or not. The analytical framework they present is very useful, but in practice it is impossible to predict the result of a new study: if it differs greatly from previous studies, it could make the confidence interval wider, not smaller. It is a lot easier to ignore all these complexities and proceed as if they did not exist. The simplicity of doing so is, however, entirely illusory. Uncertainties exist even if ignored. Uncertainties that are ignored may turn up as nasty surprises later on. The first step towards avoiding this is to explicitly recognise uncertainty and determine how it can be reduced. Five of the ten sources of uncertainty that have been discussed can be quantified at present; the other five remain difficult to quantify. Is there any prospect of quantifying these sources of uncertainty? It should be possible to quantify at least three of the five sources of uncertainty so far not quantified. Uncertainty with respect to the effects of exogenous factors (i.e. all other factors influencing road safety in addition to the programme) can be approximately quantified by analysing the prediction errors associated with past predictions of changes in road safety. Uncertain duration or stability over time of the effects of a road safety measure is also a topic which is amenable to research. As an example, so called cumulative meta-analysis studies how summary estimates of effect change over time as new studies are added. This makes it possible to determine whether there is a trend over time for effects to become larger or smaller. Uncertainty about the degree to which a road safety programme is implemented can be estimated by studying historical records showing the degree to which programmes were implemented in the past and variability with respect to the degree of implementation. The two remaining sources of uncertainty, about the definition of the target group of accidents, and about the combined effects of measures, present greater challenges. The first of these two is not statistical in its nature. Still, a properly designed evaluation study can test for spill-over effects or migration effects suggesting that a measure may have an effect – often adverse – on accidents it was not intended to influence. Research on the combined effects of measures is still in its infancy. It may be difficult to perform very rigorous studies about the combined effects of measures. It requires quite detailed records of which measures have been implemented; often such records are not kept. Moreover, if implementation of a programme takes time, confounding factors increasingly influence safety outcomes and need to be controlled for.
8. Conclusions The main results of the study reported in this paper can be summarised as follows: 1. The targets for improving road safety as part of national road safety programmes are not always realised. One reason for this may be that sources of uncertainty in the estimated impacts of such programmes are overlooked.
2. Ten sources of uncertainty in estimated benefits of national road safety programmes are identified and described. It is concluded that five of these sources can be quantified, whereas too little is known to quantify the other five sources of uncertainty. 3. Uncertainty with respect to the impacts of road safety programmes is rarely estimated, but is likely to be considerable. 4. More research on sources of uncertainty in road safety programmes may lead to the quantification of more sources of uncertainty, but it is unlikely that all sources of uncertainty can ever be meaningfully quantified. References Bijleveld, F.D., 2005. The covariance between the number of accidents and the number of victims in multivariate analysis of accident related outcomes. Accident Analysis and Prevention 37, 591–600. de Blaeij, A., Florax, R.J.G.M., Rietveld, P., Verhoef, E., 2003. The value of statistical life in road safety: a meta-analysis. Accident Analysis and Prevention 35, 973–986. Broughton, J. 1987. The effect on motorcycling of the 1981 Transport Act. TRRL Research Report 106. Transport and Road Research Laboratory, Crowthorne, Berkshire. Elvik, R., 1996. Does prior knowledge of safety effect help to predict how effective a measure will be? Accident Analysis and Prevention 28, 339–347. Elvik, R., Amundsen, A., 2000. Improving road safety in Sweden. Main Report. Report 490. Institute of Transport Economics, Oslo. Elvik, R., 2007. Prospects for Improving Road Safety in Norway. Report 897. Institute of Transport Economics, Oslo. Elvik, R., 2009a. Developing accident modification functions. An exploratory study. Transportation Research Record 2103, 18–24. Elvik, R., 2009b. An exploratory analysis of models for estimating the combined effects of road safety measures. Accident Analysis and Prevention 41, 876–880. Elvik, R., 2010. The stability of long-term trends in the number of traffic fatalities in a sample of highly motorised countries. Accident Analysis and Prevention 42, 245–260. Elvik, R., Mysen, A.B., 1999. Incomplete accident reporting. A meta-analysis of studies made in 13 countries. Transportation Research Record 1665, 131–140. Elvik, R., Christensen, P., Fjeld-Olsen, S., 2003. Daytime Running Lights. A Systematic Review of Effects on Road Safety. Report 688. Institute of Transport Economics, Oslo. Elvik, R., Kolbenstvedt, M., Elvebakk, B., Hervik, A., Bræin, L., 2009. Costs and benefits to Sweden of Swedish road safety research. Accident Analysis and Prevention 41, 387–392. Færdselssikkerhedskommissionen, 1988. Færdselssikkerhedspolitisk handlingsplan. Betænkning 1157. Statens Informationstjeneste, København. Fridstrøm, L., Ifver, J., Ingebrigtsen, S., Kulmala, R., Thomsen, L.K., 1995. Measuring the contribution of randomness, exposure, weather, and daylight to the variation in road accident counts. Accident Analysis and Prevention 27, 1–20. Hauer, E., 1997. Observational Before–After Studies in Road Safety. Pergamon Press, Oxford. Hauer, E., Geva, I., 1984. A decision analytic approach to the design of experiments in research on traffic safety. In: Yagar, S. (Ed.), Transport Risk Assessment, vol. 155–173. University of Waterloo Press, Waterloo, Ontario. Hauer, E., Hakkert, A.S., 1988. Extent and some implications of incomplete accident reporting. Transportation Research Record 1185, 1–10. Palisade Corporation, 2009. Guide to Using @RISK. Risk Analysis and Simulation Add-in for Microsoft Excel. Palisade Corporation, Ithaca. Rasmussen, R.E.H., 1964. Elementær Måleteori. Fjerde Oplag. Jul Gjellerups Forlag, København. Samferdselsdepartementet, 1973. Stortingsmelding 107 (1972–1973). Om revidert Norsk Vegplan. Statens forvaltningstjeneste, Oslo. Samferdselsdepartementet, 1981. Stortingsmelding 80 (1980–1981). Om Norsk Vegplan 1982–1985. Statens Forvaltningstjeneste, Oslo. Samferdselsdepartementet, 1997. Stortingsmelding 37 (1996–1997). Norsk veg- og vegtrafikkplan 1998–2007. Statens Forvaltningstjeneste, Oslo. Samferdselsdepartementet, 2000. Stortingsmelding 46 (1999–2000). Nasjonal transportplan 2002–2011. Statens Forvaltningstjeneste, Oslo. Strand, S., 1987. Usikkerhet i Data og Modeller. Transportøkonomisk Institutt, Oslo. Sverdrup, E., 1964. Lov og Tilfeldighet. Bind 1. Universitetsforlaget, Oslo. Trafiksäkerhetsdelegationen, 1982. Plan för Trafiksäkerhetsarbetet. Kommittébetänkande 1981:50. Statens Tryckericentral, Helsingfors. Vägverket, 1999. Särskild Trafiksäkerhetsplan. Vägverket, Borlänge. Wanvik, P.O., 2009. Road lighting and traffic safety. Do we need road lighting? Doctoral Theses at NTNU, 2009:66. Norwegian University of Science and Technology, Trondheim. Ye, Z., Lord, D., 2009. Estimating the variance in before–after studies. Journal of Safety Research 40, 257–263.