Improving crash predictability of the Highway Safety Manual through optimizing local calibration process

Accident Analysis and Prevention 136 (2020) 105393

Contents lists available at ScienceDirect

Accident Analysis and Prevention journal homepage: www.elsevier.com/locate/aap

Improving crash predictability of the Highway Safety Manual through optimizing local calibration process

T

Seyedehsan Dadvara, Young-Jae Leeb,*, Hyeon-Shic Shinc a

National Research Council (NRC) Research Associate, Turner-Fairbank Highway Research Center, Federal Highway Administration, 6300 Georgetown Pike, McLean, VA 22101, United States b Department of Transportation and Urban Infrastructure Studies, School of Engineering, Morgan State University, 1700 E. Cold Spring Lane, Baltimore, MD 21251, United States c City & Regional Planning Program, Department of Graduate Built Environment Studies, School of Architecture and Planning, Morgan State University, 1700 E. Cold Spring Lane, Baltimore, MD 21251, United States

ARTICLE INFO

ABSTRACT

Keywords: Highway Safety Manual (HSM) Local calibration Crash prediction Prediction quality Highway Safety Information System (HSIS)

The predictive method of the Highway Safety Manual (HSM) estimates crash frequency by applying an uncalibrated safety performance function (SPF) and a set of uncalibrated crash modification factors (CMFs) to each location individually; then the predicted crashes must be adjusted by a local calibration factor (LCF) at the aggregate level for at least 30–50 sites per SPF. Although this calibration procedure assures total predicted crashes will be localized, still the prediction of crashes for individual locations suffers from the aggregate localization process. An alternative approach of locally calibrating the HSM predictive method is proposed to improve prediction quality at individual locations while maintaining equality of total observed and total predicted crashes. The methodology incorporates multiple calibration factors for different components of the predictive method (SPF parameters and CMFs) rather than a single calibration factor as recommended by the HSM that only calibrates at the aggregate level. In the proposed method, the application of calibration factors expressed in both weight and power function better reflects the local conditions while still ensuring calibration at the aggregate level. The parameters are estimated through an optimization process of five different methods. Rural two-lane, two-way roads (R2U) data was used from the states of Maryland, Illinois, and Washington. A tool named “Roadway Safety Data Integrator (RSDI)” was developed for data preparation. Different Goodness-ofFit measures along with CURE plots indicated that the proposed method performed significantly better than the HSM calibration method, calibration function (that will most likely be calibration process in the HSM 2nd edition), calibrated Washington models (for the case of Washington data), and some alternative calibration methods suggested by past studies. Moreover, the results indicated that the additional parameters for CMFs could improve the prediction significantly; a previous study did not find this to be so due to data limitations, but we have improved the methodology and are not so limited. Application of the proposed approach can lead to more accurate identification of hot-spots and site-specific strategies. Considering the limitations of this study, some avenues for further research are discussed.

1. Introduction The HSM predictive method for a facility type (either roadway segment or intersection) consists of three main components (Equation 1): safety performance functions (SPF), crash modification factors (CMF)1, and local calibration factors (LCF) (American Association of

State Highway and Transportation Officials AASHTO, 2010, American Association of State Highway and Transportation Officials AASHTO, 2014). Equation 1. The HSM Predictive Method

NPredicted (Adjusted) = NSPF × (CMF1 × CMF2 × …× CMFn) × LCF

Corresponding author. E-mail addresses: [email protected] (S. Dadvar), [email protected] (Y.-J. Lee), [email protected] (H.-S. Shin). 1 The HSM Part C CMFs will be called SPF Adjustment Factors (AFs) in the forthcoming 2nd edition of the HSM based on an approved decision at the TRB ANB25 Highway Safety Performance Committee (HSPC). However, the term CMF was used throughout this paper to be consistent with the 1st edition of the HSM Part C which is currently (2019) the official release of the manual. ⁎

https://doi.org/10.1016/j.aap.2019.105393 Received 2 July 2019; Received in revised form 25 September 2019; Accepted 2 December 2019 0001-4575/ © 2019 Elsevier Ltd. All rights reserved.

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Where:

2.1. Summary of calibration & transferability of the predictive method of the HSM

NPredicted (Adjusted) = Adjustedtotal predicted crash frequency

While the HSM calibration procedure recommended by the manual is simple and straightforward, there have been many research efforts due to ambiguity, confusion, and theoretical challenges. Questioning the calibration methodology of the HSM started several years prior to its official first edition release; Persaud et al. (2002) investigated the then-proposed calibration procedure in the IHSDM for urban intersections using data from Toronto, Vancouver, and California. The results suggested a single calibration factor might be inappropriate and a disaggregation by traffic volume might be better option. Sawalha and Sayed (2006) also presented some methods for recalibrating negative binomial models before transferring them to another jurisdiction or time period. Their emphasis was on recalibration of the shape parameter (over-dispersion parameter) of the transferred model based on local jurisdiction data. A moment method and a maximum likelihood model were presented for recalibration of the shape parameter (when the constant term is not recalibrated) and both the shape parameter and the constant term of the transferred model, respectively. The proposed methods were compared with the current calibration method of the HSM, and the results showed the superiority of their proposed method. Bahar and Hauer (2014) recommended developing LCFs by regional subsets of a jurisdiction, different segment lengths, and AADT ranges, which may improve the calibration and eventually predictive quality. Some studies applied this recommendation (Srinivasan et al., 2011; Dixon et al., 2012; Geedipally et al., 2017; Smith et al., 2017; Claros et al., 2018). Nevertheless, developing local calibration factors by region, AADT ranges, and other categorizations would not necessarily improve the quality of calibration (Srinivasan et al., 2016). In a study on R2U in Arizona, Srinivasan et al. (2016) developed HSM LCF for R2U for total crashes. Due to very poor fit based on a significant portion of the CURE plot outside the confidence limits, they developed LCFs by region, highway functional class, AADT category, segment length, alignment, curve radius, and year. However, a significant portion of the CURE plots was still outside the confidence limits. So, they finally decided to try some calibration functions that could improve the results significantly. The functions (six types) included the HSM prediction value as a power function (NPredicted = a × (HSMPrediction)b ) and/or power values for the R2U SPF parameters and/or the product of CMFs as a power function. However, from the aspect of application, usage of a simple calibration function (the one with HSM prediction value as a power function) was recommended on Arizona R2U roadways (Colety et al., 2016). The same methodology was also applied on R2U in North Carolina (Smith et al., 2017), and the methodology was further examined in a recent NCHRP study (Ivan et al., 2017) on U2U and U4D roadway segments and R23ST intersections using data from several states (Illinois, Ohio, Minnesota, Texas, and Washington). The calibration function method will most likely be calibration process in the HSM 2nd Edition. Generally, the Arizona studies (Colety et al., 2016; Srinivasan et al., 2016), which were the basis for the following studies (Smith et al., 2017; Ivan et al., 2017), had two limitations: one methodological and one data-related. They sampled 196 sites (509 homogeneous R2U segments) and developed the HSM local calibration factor and proposed local calibration functions (Type 1–6), but they did not use any kind of validation method besides checking some statistics (log likelihood and over-dispersion parameter) and CURE plots. They pointed out that the estimated sample size was for a single calibration factor and the same sample size may not provide reasonable results for the more complicated functional forms; thus, it seemed necessary that if they could they should apply the results on a testing dataset. Also, the data limitation caused them to use the default values for 10 out of 12 CMFs of R2U. While the HSM does have some default values for some CMFs, the actual data is needed for the rest. So, using the base condition values for nearly all CMFs does not seem a reasonable assumption, and perhaps

NSPF = Averagecrashfrequency under basecondition CMF1, …, CMFn = Crash ModificationFactors LCF = Local Calibration Factor To predict crash frequency for a given facility type, NSPF is calculated first to estimate the average crash frequency for base conditions. Then a set of CMFs are multiplied to each other to produce a combined CMF (CMFCombined). A CMF is a multiplicative factor or function for evaluating changes in crashes with a given countermeasure or existing condition at the study location (Federal Highway Administration (FHWA), 2016). The product of NSPF and CMFCombined becomes uncalibrated predicted crash frequency. The last task is to compute an LCF, a factor to adjust crash frequency estimated from the HSM predictive method to local conditions (e.g., traffic variation, climate, weather, population, and other contributing factors of crashes) at an aggregate level. An LCF, by the HSM recommended procedure, is the ratio of total observed crashes to total uncalibrated predicted crashes (i.e., NSPF * CMFCombined) of a minimum 30–50 sites. One issue is that the predictive method that is adjusted at an aggregate level may suffer from an average-over effect from aggregating individual locations, resulting in large errors for individual locations when applied to a study site. The calibration of the HSM models and, in general, negative binomial crash models (which are the basis of the HSM predictive models) are required when the models are to be used in other jurisdictions (spatial transferability) or in the same jurisdiction in other time periods (temporal transferability). Roadway SPFs in the HSM are dealing with crash prediction based on two parameters: a traffic volume variable and a linear relationship with roadway segment length (American Association of State Highway and Transportation Officials AASHTO, 2010). As was also mentioned in some recent studies (Venkataraman et al., 2016; Park and Abdel-Aty, 2017), this approach is still at the aggregate level because each facility type contains many individual sites that have varying geometrics and other conditions. Limiting the SPF prediction to traffic volume variations leaves many effects unobserved, and this is an issue if prediction is needed at the regional level because the model is not sensitive to local effects in anything beyond traffic volume (Venkataraman et al., 2016). Moreover, the CMFs are not universally constant values or variables that can always be applied everywhere. The spatial and temporal transferability of CMFs should be examined appropriately, which will lead to a more confident decisionmaking process (Hauer et al., 2012). In addition, it should be noted that there is significant research and work on developing CMFs that are costly (typically $50,000 to $100,000), laborious, and highly dependent on data and its quality, which challenge jurisdictions at different levels all around the world. Thus, sharing and transferring CMFs is of interest to and needs the attention of involved major research parties. The goal of this research is to improve the predictive quality of the HSM calibration through a proposed calibration method. In this regard, the predictive quality of the proposed method will be compared to the HSM calibration, calibration function, and some alternative calibration methods suggested by past studies. The scope of study is limited to the HSM Part C SPFs and CMFs which adjust NSPF prediction for base conditions. 2. Literature review In this section the existing literature on the calibration and transferability of the predictive method of the HSM and CMFs (including different methods of combining multiple CMFs) is reviewed and discussed. 2

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

because of this issue the calibration function type 5 (CMFs as a power function) did not improve the calibration quality. A follow-up study using more data and available CMFs would shed some light on the proposed methodology. In a recent study (Farid et al., 2018), the authors developed statespecific SPFs for Ohio, Illinois, Minnesota, and California and examined their transferability among other states using different calibration methods such as the HSM calibration method, calibration function, and a local regression method (by minimizing the sum of the squares of the differences between the observed crash frequencies and the fitted curve in the neighborhood). Claros et al. (2018) examined different calibration schemes on freeway segments (UF4) in Missouri including the HSM calibration, calibration by AADT and segment length ranges, calibration functions, and developing new SPFs. The results were somewhat unexpected because the Missouri-specific SPF had accuracy similar to a fully loaded and calibrated HSM SPF, while only relying on two variables (AADT and segment length). It was recommended that agencies should consider developing jurisdiction-specific SPFs because of ease of implementation and less data collection efforts and statistical modeling complexity. On the other hand, a single correction factor of 1.85 was used to transfer the inertial consistency model developed in Spain to North Carolina R2U. The inertial consistency models are SPFs with a consistency parameter in addition to the HSM SPFs for which the main risk exposures are traffic volume (AADT) and length. (Llopis-Castelló et al., 2019). Some alternative calibration schemes have been proposed and examined in the literature as well, such as the calibration factor estimation as a special case of SPF estimation proposed by Mehta and Lou (2013) and presented in Equation 2. Equation 2. The Mehta Calibration Method (Mehta and Lou, 2013)

the calibration procedures of the HSM, they defined and established new base conditions for new SPF models, and then they converted CMF values accordingly, which can be considered as a way of calibrating the CMFs to local conditions. Their study results also supported the importance of substituting the HSM crash severity level distributions with the jurisdiction-specific ones for CMFs, which for their study was drastically different from the HSM. This method tends to account for local differences and their impacts on the performance of individual CMFs in overall prediction but it still may not address the potential effect of combining multiple CMFs. The study was expanded by Qin et al. (2016) by establishing new base conditions, developing jurisdiction-specific SPFs, and converting CMFs to new base conditions. The comparison results for R2U, R4U (Rural Four-Lane Undivided Roads), and R4D (Rural Four-Lane Divided Roads) showed that the customized models outperformed the HSM models in predicting sites with crashes. 4 Developing new CMFs based on jurisdiction-specific conditions: Thousands such CMFs are available in the CMF Clearinghouse. This approach is similar to the third category accounting for individual CMFs but not the effect of combining multiple CMFs. 5 Developing new CMFs for multiple treatments and/or multiple existing conditions: A limited number of past studies investigated the development of new CMFs for multiple treatments and conditions like the New Zealand study (Roberts and Turner, 2007). However, this topic has gained some popularity in recent years. Several studies have been published based on Florida experiments. Park et al. (2014) stated that there is no CMF for multiple treatments in the HSM and the HSM suggests combining treatments by multiplying CMFs which may over- or under-estimate impacts. Park and Abdel-Aty (2015a) used Florida data for R2U and two treatments (shoulder rumble strips (SRS) and widening shoulder width (WSW)) to validate the HSM caution of over- or under-estimation by using a multiplicative manner of combining CMFs. The analysis revealed that combined safety effects were overestimated by 4–10% when the HSM manner was used. In another study, Park and Abdel-Aty (2015b) used Florida data for R4U to develop CMFs for four roadside elements (driveway density, poles density, distance to poles, and distance to trees). The results of developing CMF for combined safety effects were consistent with an earlier study by the same authors (Park and Abdel-Aty, 2015a) and it was observed that the HSM procedure overestimated the real effects by 8–10%.

µ = exp(LN (LCFMehta) + LN (base SPF for Sitei )) Where: μ = Mean of Negative Binomial distribution LCFMehta = Local Calibration Factor based on Mehta and Lou (2013) method 2.2. Summary of calibration & transferability of CMFs Table 1 shows the available CMFs for the facility types discussed in chapters 10–12 of the 1st edition of the HSM, and Table 2 exhibits the available CMFs for the facility types discussed in chapters 18 and 19 of the supplement to the 1st edition of the HSM. Since the HSM predictive methods were developed over time for different roadway and intersection types, some facility types have more CMFs than others do. Currently, R2U (Rural Two-Lane, Two-Way Roads) has 12 CMFs, the most among the HSM facility types, followed by Freeway Segments with 11 CMFs. On the other hand, some facilities have fewer numbers of CMFs (3–5 CMFs), and no CMF is available for RM4SG (Rural Multilane, Four-leg Signalized Intersections). Broadly speaking, five categories of methods dealing with localization/calibration of the CMFs to improve prediction quality have been discussed in the literature:

In summary, little research currently exists regarding the independence of the countermeasures/existing conditions and an appropriate method of accounting for the impacts of combining multiple CMFs to reduce prediction errors. However, the current approach seems reasonable due to current findings. The Empirical Bayes (EB) method that involves the use of historical observed crash frequency data may compensate for bias caused by the lack of independence among CMFs (Hagen, 2015). At the time of writing this manuscript, a “Limited Use Document” has been released for “Guidance for the Development and Application of Crash Modification Factors NCHRP Project 17–63” (Carter et al., 2017). It provides supplemental grounds regarding the calibration of existing CMFs, quantifying the effect of multiple treatments, and development of CMFunctions for inclusion in the ongoing research for the second edition of the HSM. However, since the official release was not yet published, the results of that report were not discussed in this study.

1 Indirectly locally calibrating CMFs as part of developing LCFs by following calibration procedures that are available in the HSM (Part C; Appendix A) or alternative calibration schemes: Many states, such as Florida, Maryland, Michigan, Oregon, and North Carolina, have developed LCFs for the HSM (some or all facility types) based on the HSM calibration guideline (Shin et al., 2015a,b). 2 Indirectly locally calibrating CMFs by applying alternative methods of combining CMFs: Table 3 summarizes some of the existing methods 3 Directly locally calibrating CMFs based on jurisdiction-specific conditions: Few studies examined the effect of calibrating CMFs based on jurisdiction-specific conditions. Qin et al. (2014) calibrated the HSM predictive methods for R2U in South Dakota. While they followed

3. Methodology Although improving the HSM calibration method has been the subject of many research efforts, there is no clear guidance on acceptability of crash prediction error. Crash prediction error acceptability relies on many different factors such as statistical tests, agency’s preference, resources, and engineering judgement and in some cases is 3

4

CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF CMF

for for for for for for for for for for for for for for for for for for for for for for for for for

Lane Width Shoulder Width and Type Horizontal Curves Super-elevation Grades Driveway Density Centerline Rumble Strips Passing Lanes Two-Way Left-Turn Lane Roadside Design Lighting Automated Speed Enforcement Sideslopes Median Width On-Street Parking Roadside Fixed Objects Intersection Skew Angle Intersection Left-Turn Lanes Intersection Right-Turn Lanes Intersection Left-Turn Phasing Right-Turn-on-Red Red-Light Cameras Bus Stops Schools Alcohol Sales Establishments 12

* * * * * * * * * * * *

R2U

4

* * *

* * *

4

*

R24ST

*

R23ST

Facility Types

Source: Highway Safety Manual, 1st Edition (2010).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Total

Crash Modification Factor

Table 1 CMFs for facility types of the HSM 1st edition.

3

* *

*

R24SG

5

* * *

* *

R4U

5

*

* *

* *

R4D

4

* * *

*

RM3ST

4

* * *

*

RM4ST

0

RM4SG

4

* *

* *

U2U

4

* *

* *

U3T

4

* *

* *

U4U

5

* * *

* *

U4D

4

* *

* *

U5T

3

* *

*

U3ST

3

* *

*

U4ST

* * * * * * * * 9

*

U3SG

* * * * * * * * 9

*

U4SG

3 3 1 1 1 1 1 1 1 1 17 8 1 2 5 5 4 9 9 2 2 2 2 2 2 86

Total

S. Dadvar, et al.

Accident Analysis and Prevention 136 (2020) 105393

5

11

* * * * * * * * * * *

Freeway Segments

Facility Types

7

*

* * * * * *

Ramp Entrance (Speed-Change Lanes)

Source: Highway Safety Manual, 1st Edition, Supplement (2014).

CMF for Horizontal Curves CMF for Lane Width CMF for Inside Shoulder Width CMF for Median Width CMF for Median Barrier CMF for High Volume CMF for Lane Change CMF for Outside Shoulder Width CMF for Shoulder Rumble Strip CMF for Outside Clearance CMF for Outside Barrier CMF for Ramp Entrances CMF for Ramp Exits CMF for Right Shoulder Width CMF for Left Shoulder Width CMF for Right Side Barrier CMF for Left Side Barrier CMF for Lane Add or Drop CMF for Ramp Speed-Change Lane CMF for Weaving Section CMF for Exit Ramp Capacity CMF for Crossroad Left-Turn Lane CMF for Crossroad Right-Turn Lane CMF for Access Point Frequency CMF for Segment Length CMF for Median Width CMF for Protected Left-Turn Operation CMF for Channelized Right Turn on Crossroad 29 CMF for Channelized Right Turn on Exit Ramp 30 CMF for Non-Ramp Public Street Leg 31 CMF for Skew Angle Total

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Crash Modification Factor

Table 2 CMFs for facility types of the HSM 1st edition - Supplement.

7

*

* * * * * *

Ramp Exit (Speed-Change Lanes)

8

* * * * * *

* *

Ramps

9

* * * * * * *

* *

Collector-Distributor Roads

10

1 1 59 * 7

*

5 5 3 3 3 3 1 1 1 1 1 1 1 2 2 2 2 2 2 1 2 2 2 2 2 2 1 1 1

* * * * * *

Ramp Terminals (StopControlled)

*

* * * * * * * *

Ramp Terminals (Signalized)

Total

S. Dadvar, et al.

Accident Analysis and Prevention 136 (2020) 105393

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 3 Selected existing methods for combining CMFs. Method

State / Country

CMFCombined = CMF1 × CMF2 × …×CMFn

The HSM, Florida, Maine, Maryland, Minnesota, Ohio, Oregon, Pennsylvania, Utah, Vermont, Washington Alabama

CMFCombined = CMF1

1

CMF2 2

…

1

CMFn 2

Only one treatment can be applied in the analysis to any one crash. Only the lowest CMF is applied (i.e., treatment with the greatest expected crash reduction). NCHRP 17-25: Systematic reduction of safety effects of less effective treatment:

CMF2, Reduced = CMFCombined = 1

1

CMF2 + CMF2 ➔ CMFCombined 2 2 (1 (CMF1 × CMF2)) 3

Michigan South Carolina –

= CMF1 × CMF2, Reduced New Zealand

Elvik (2009) proposed methods:

The method of dominant common residuals CMFCombined = (CMF1 × CMF2×…× CMFn)CMFD1

The method of double dominant common residuals CMFCombined = (CMF1 × CMF2×…× CMFn) (CMFD1× CMFD2) Where: CMFD1 = The dominant CMF; CMFD2 = the second dominant CMF. Meta-Analysis (Bahar, 2010):

CMFCombined =

n CMFUnbiased, i 2 Si i=1 n 1 2 i = 1 Si

and S =

–

–

1 n 1 2 i = 1 Si

Park and Abdel-Aty (2017) proposed methods: Average best two existing methods Average best three existing methods

–

Sources: (Harwood et al., 2000; Roberts and Turner, 2007; Elvik, 2009; Bahar, 2010; Gross and Hamidi, 2011).

subjective. Depending on the considered measure, different definitions of LCFs may outperform others (Rajabi et al., 2018). However, when applying predictive models in the real world, the use of an incorrect LCF can result in systematic misallocation of resources but the magnitude of error that can be tolerated is a matter of “opinion” (Bahar and Hauer, 2014). Thus, the application of calibration methods that satisfy multiple measures reasonably would be better than those that perform differently by different measures. To improve crash predictions, some alternative calibration methods are proposed and presented in Table 4. The main goal is to adjust the HSM crash predictions that provide better fit of the data than does the use of the HSM calibration method. As was mentioned, chapters 10–12 of the HSM contain 25 CMFs with 86 applications and chapters 18–19 provide 31 different CMFs with 59 applications. Therefore, developing all of them at the jurisdictional level may not be possible in terms of available data and resources; however, on the other hand, their application potentially can contribute in prediction errors at individual sites. Also, in the ideal world, as Lacy (2001) noted, the research should provide crash modification factors for the combined treatments as well. While this approach would account for interactions, it would also be very costly and labor-intensive. Thus, there is the need for a good method to combine and transfer existing CMFs. Based on the reviewed literature, two issues seem related to how CMFs affect the individual site crash prediction quality: first, whether the CMFs were locally developed / calibrated, and second, how multiple CMFs for a facility type are being combined. The proposed method aims to address both issues simultaneously. The proposed calibration methods are categorized by a calibration focus area (Table 4): CMFs (proposed methods C-1 through C-4), SPF parameters (methods S-1 through S-3), and SPF parameters and CMFs (proposed methods SC-1 through SC-4). However, it should be noted that all methods, no matter what the focus area is, would calibrate the HSM predictive method as a whole. While the proposed methods C-1 through C-4 are only focused on CMFs by weights and/or powers, the adjustment coefficient (noted as Cadj in Table 4) technically represents the HSM calibration factor. The assumption behind the first proposed method “Proposed-C-1,” which weighs each CMF (or a combined weight of all CMFs and eventually a whole prediction method (Cadj) because of the multiplicative manner of CMF combinations in the HSM), is to adjust the CMFs to area conditions, both treatments and/or existing

conditions. The same assumption is amplified in the third method “Proposed-C-3” by applying a power to each CMF in addition to the weight. A power value greater than 1.0 means that the CMF is more effective/important in the study area, and less effective/important CMFs for the study area will be assigned a power value smaller than 1.0. The power of 1.0 indicates that the base CMF can be applied to the study area. Generally, the first method is the same as a specific case of the third method when all powers are equal to 1.0. The second series of proposed methods (i.e., S-1 through S-3) are targeting SPF parameters (AADT and length) by weights and/or powers. In the third series of proposed methods (i.e., SC-1 through SC-4) SPF parameters and CMFs are all targeted in the calibration process by weights and powers. The rationale of the proposed methods in which estimation of parameters and their application are based on CMFs ≠ 1 is the fact that when a CMF = 1 then that facility in that particular location meets the exact same conditions as the HSM base condition, so theoretically manipulation of that CMF with weight is meaningless and with power is useless. In these proposed methods, during the process of estimation of parameters, only the CMFs ≠ 1 of each individual site of training datasets were included in the estimation process, and the estimated parameters (“weights” for Proposed-C-2 and “weights and powers” for Proposed-C-4, Proposed-SC-3, and Proposed-SC-4) were only applied to CMFs ≠ 1 of each individual site of testing datasets. If data is available then predictive models can be developed at the state or regional level, but the level of data needed is much higher than calibrating the HSM SPFs (Srinivasan et al., 2013). While developing SPFs or CMFs based on available state data seems possible and promising, there are conditions that threaten the reliability of state-specific models. A state may not have sufficient data to develop a meaningful single CMF for a countermeasure or characteristic; however, in the proposed method if a particular existing CMF (i.e., of the HSM) has different characteristics in a particular state – for example, only matching certain conditions of associated HSM CMF, such as most of the roadway segments for a particular roadway type having similar lane widths, which technically makes the development of that CMF in that state meaningless – then it can be captured in a way by weights and powers. Therefore, different factors play in the decision making process for development or calibration of predictive models in different jurisdictions, which should be taken into account. The proposed methods in Table 4 are estimated by using five 6

Weights & powers for SPF parameters (Power only for AADT)

Proposed-S-2

7

Weights & powers for SPF parameters and CMFs

Weights & powers for SPF parameters (Power only for AADT) and CMFs ≠ 1 Weights & powers for SPF parameters and CMFs ≠ 1

Proposed-SC-2

Proposed-SC-3

Proposed-SC-4

Weights & powers for SPF parameters (Power only for AADT) and CMFs

Proposed-SC-1

6

× e(

0.312)

× e(

0.312)

× CMFC

×(N ×

=

6

× e( 6

× e(

0.312)

0.312)

× (CMF1a × CMF2b×…×CMFnn)

× (A × CMF1a) × (B × CMF2b) × …

× L × 365 × 10

×

6

× e(

× (A × CMF1a) × (B × CMF2b) × …

0.312) ]

× (A × CMF1a) × (CMF1a × CMF2b×…×CMFnn)

× e(

× (CMF1a × CMF2b×…× CMFnn)

0.312) ]

0.312)

× e(

× (CMF1 × CMF2×… ×CMFn) =

× CMFC

0.312) ]

0.312)

× e(

× e(

6

Calculating weights and powers based on “Proposed-SC-2” method only for CMFs ≠ 1 and applying to only CMFs ≠ 1

Calculating weights and powers based on “Proposed-SC-1” method only for CMFs ≠ 1 and applying to only CMFs ≠ 1

Cadj

L ) P2

6

6

× (CMF1 × CMF2×… ×CMFn) =

× CMFC

0.312) ]

e ( 0.312)

× (w 2 × × 365 × 10 6 b (B × CMF2 ) × …×(N × CMFnn) = × AADT p1 × L p2 × 365 × 10 6 × e ( 0.312)

NPredicted = [(w1 ×

AADT ) P1

Cadj × AADT p1 × L × 365 × 10

×(N × CMFnn) =

[(w1 × AADT ) P1 × (w 2 × L ) × 365 × 10

NPredicted =

Cadj × AADT p1 × L p2 × 365 × 10

× (CMF1 × CMF2×… ×CMFn) =

× CMFC

0.312) ] 0.312)

× e(

×

6

[(w1 × AADT ) P1 × (w 2 × L ) P2 × 365 × 10

NPredicted =

Cadj ×

AADT p1

[(w1 × AADT ) P1 × (w 2 × L ) × 365 × 10 6

Cadj × AADT × L × 365 × 10 NPredicted =

× e( × e(

6 6

[(w1×AADT) × (w 2 × L) × 365 × 10

NPredicted =

Calculating weights and powers based on “Proposed-C-3” method only for CMFs ≠ 1 and applying to only CMFs ≠ 1

Cadj × AADT × L × 365 × 10

CMFnn)

NPredicted = AADT× L× 365 × 10

Calculating weights based on “Proposed-C-1” method only for CMFs ≠ 1 and applying to only CMFs ≠ 1

6

× (A × CMF1) × (B × CMF2) × …

×(N × CMFn) = Cadj × AADT × L × 365 × 10

NPredicted = AADT× L× 365 × 10

Formulation

Notes: CMFC = Combined CMF, Cadj = Adjustment coefficient, A, B … N = weights for CMFs, a, b… n = powers for CMFs, w1&p1 = weight and power for AADT parameter of the HSM SPF, w2&p2 = weight and power for length parameter of the HSM SPF.

SPF Parameters & CMFs

Proposed-S-3

Weights & powers for SPF parameters

Weights & powers for CMFs ≠1 Weights for SPF parameters

Proposed-C-4

Proposed-S-1

Weights for CMFs ≠ 1 Weights & powers for CMFs

Proposed-C-2 Proposed-C-3

SPF Parameters

Weights for CMFs

Proposed-C-1

CMFs

Description

Method

Calibration Focus Area

Table 4 Proposed calibration methods.

S. Dadvar, et al.

Accident Analysis and Prevention 136 (2020) 105393

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 5 Estimation methods for proposed calibration methods. Method Sum of Squared Deviation (SSD)

Median Absolute Deviation (MAD)

Root-Mean-Squared Error (RMSE)

Poisson Regression (Loglikelihood Poisson)

Negative Binomial Regression (Log-likelihood NB)

Formulation

SSD =

MAD =

RMSE =

LLP =

n i

Goal

(NOi

n |N Oi i

Objective Function: Min. SSD Constraint:

NPi) 2

N Pi |

n (N i Oi

n

N Pi ) 2

n 1 Noi

=

n 1 Noi

=

n 1 Noi

=

n 1 Noi

=

n 1 Noi

=

n 1

N pi

n 1

N pi

n 1

N pi

n 1

N pi

n 1

N pi

Objective Function: Min. MAD Constraint: Objective Function: Min. RMSE Constraint:

n

Objective Function: Max. LLP Constraint:

NPi + NOi × LN (NPi)

LLNB = GAMMALN (NOi + L )

GAMMALN ( L) + L × LN ( L) + NOi × LN (NPi) =

( L + NOi) × LN ( L + NPi)

Objective Function: Max. LLNB Constraint:

Notes: Noi = Number of observed crashes, Npi = Number of predicted crashes, GAMMALN = log gamma function in Microsoft Excel, and φ = inverse overdispersion parameter for the NB regression model.

different optimization methods (Table 5), all under the constraint that total observed crashes is equal to total predicted crashes. The reason for inclusion of the constraint (NObserved = NPredicted) is bypassing the HSM local calibration process that requires summation of observed crashes equal summation of predicted crashes. As part of the estimation process, some performance measures were also estimated across all proposed calibration methods (e.g., MAD, RMSE, residual, Chi Square, and R2). The models are estimated by using the Solver module (Generalized Reduced Gradient (GRG) Nonlinear method) in Microsoft Excel. The details of using Solver are like those that were explained in Srinivasan et al. (2016). The application of an optimization procedure was also used in literature such as Sawalha and Sayed (2006) and Srinivasan et al. (2016). Application of least square and also maximum likelihood methods were used by Srinivasan et al. (2016) and Rajabi et al. (2018). Hauer (2015) also used and recommended Microsoft Excel Solver. For the purpose of comparison, the five measures of SSD, RMSE, MAD, LLP, and LLNB were estimated for all proposed and selected existing methods of calibration (the HSM calibration method, SPF-only, LCFMehta (Mehta and Lou, 2013), and calibration function method (Srinivasan et al., 2016)) and some of the CMF combining methods (Alabama, Michigan, South Carolina, NCHRP 17–25, New Zealand, and average best methods that were examined by (Park and Abdel-Aty, 2017)). Moreover, previously developed SPFs and CMFs based on Washington data (Banihashemi, 2011) were examined among other existing methods for Washington data. The CURE (Cumulative Residuals) plots with ± 2σ (Standard Deviation) bounds were also developed for all compared methods; based on the literature, CURE plots can be helpful to make decisions regarding transferability of the HSM predictive method parameters (Hauer and Bamfo, 1997; Hauer et al., 2012; Srinivasan et al., 2013). In general, a good CURE plot is one that oscillates around 0.0. Thus, a good fit is given when the residuals do not stray beyond the ± 2σ boundaries. Mean and variance were also estimated for all training and testing datasets to generate the CURE plots for different methods. As a note the preliminary version of the proposed methodology (the estimation method was limited only to min. SSD) was presented as a poster (Dadvar et al., 2016) and a hybrid session with a 3-minute presentation and poster (Dadvar, 2018a,b) at two Transportation Research Board (TRB) annual meetings (95th and 97th).

4. Data Upon a preliminary analysis based on Maryland R2U data that were collected through a research study to develop LCFs for Maryland (Shin et al., 2014, Shin et al., 2015a), the Highway Safety Information System (HSIS) data were reviewed and data from Illinois and Washington states were selected and requested (Federal Highway Administration (FHWA), 2018). The data were reviewed, cleaned, and processed to be ready for the analysis. Depending on the selected state, the HSIS data consist of several different datasets with common fields to join (such as roadway information, grade, curve, and crash data). Some data cleanings were done and the AADT values were interpolated for some roadway segments with missing AADT values. To combine different datasets and also select homogeneous roadway segments through the years of study, a tool titled “Roadway Safety Data Integrator (RSDI)”2 was designed and developed (Dadvar and Khodaparasti, 2018). After creating the homogeneous datasets, the datasets were organized to match the required data by the HSM and some data cleanings were done. Then the Interactive Highway Safety Design Model (IHSDM) was used to estimate the HSM predicted values and CMF values, and eventually the final analysis-ready datasets were created for selected states. Since the data were sufficiently available (American Association of State Highway and Transportation Officials AASHTO, 2010), the locally derived crash distributions were also computed and used in the IHSDM wherever applicable. Data were divided into two subsets (equal number of segments in each part and controlled for other important attributes like number of crashes and AADT values for study period); the first set was used as training dataset for developing weights and/or powers and the second set was used as testing dataset for validation of the developed weights and/or powers. Three years of data (2008-10) were used in Maryland (50% for training and 50% for testing) but six years of data were used in Illinois (2005-10; 2005-07 for training and 2008-10 for testing) and Washington (2010-15; 2010–12 for training and 2013-15 for testing). The following tables (Table 6, 7 and 8) present the details of the datasets. It should be noted that in Maryland a sample of statewide data was used; however, in Illinois and Washington, statewide roadway data 2 The tool can be downloaded and tried from https://www.ehsandadvar.com/ 2018/10/roadway-safety-data-integrator-rsdi-tool.html.

8

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 6 Maryland R2U final datasets (2008–2010). Dataset

#

Item

Min.

Max.

Average

StdDev

Sum

Training

213

Testing

213

AADT (2008) AADT (2009) AADT (2010) Crashes (2008) Crashes (2009) Crashes (2010) Crashes (2008-10) Length (Mile) AADT (2008) AADT (2009) AADT (2010) Crashes (2008) Crashes (2009) Crashes (2010) Crashes (2008-10) Length (Mile)

99 99 99 0 0 0 0 0.1 270 281 282 0 0 0 0 0.1

23721 24202 24640 6 8 5 8 4 23721 24202 24640 8 9 7 9 3.4

4551.117 4592.911 4582.014 0.643 0.629 0.554 0.609 0.490 5793.549 5875.455 5795.272 0.629 0.582 0.535 0.582 0.471

4102.345 4145.699 4141.327 1.084 1.081 0.913 1.027 0.614 4379.730 4444.637 4387.183 1.063 1.141 1.066 1.089 0.549

969388 978290 975969 137 134 118 389 104.375 1234026 1251472 1234393 134 124 114 372 100.354

Table 7 Illinois R2U final datasets (2005–2010). Dataset

#

Item

Min.

Max.

Average

StdDev

Sum

Training

16712

AADT (2005) AADT (2006) AADT (2007) Crashes (2005) Crashes (2006) Crashes (2007) Crashes (2005-07) AADT (2008) AADT (2009) AADT (2010) Crashes (2008) Crashes (2009) Crashes (2010) Crashes (2008-10) Length (Mile)

9 9 9 0 0 0 0 9 9 9 0 0 0 0 0.01

13800 14000 14000 21 21 22 57 13300 13300 13500 21 17 16 46 7.07

3021.380 3008.499 3005.914 0.534 0.541 0.579 1.653 2970.999 2965.120 2905.121 0.587 0.463 0.455 1.505 0.357

2170.357 2205.270 2211.078 1.261 1.304 1.358 3.432 2189.539 2189.559 2148.117 1.384 1.118 1.098 3.116 0.598

50493310 50278039 50234839 8919 9036 9670 27625 49651331 49553082 48550380 9810 7743 7602 25155 5958.19

Testing

Both

Table 8 Washington R2U final datasets (2008–2010). Dataset

#

Item

Min.

Max.

Average

StdDev

Sum

Training

40141

AADT (2010) AADT (2011) AADT (2012) Crashes (2010) Crashes (2011) Crashes (2012) Crashes (2010–12) AADT (2013) AADT (2014) AADT (2015) Crashes (2013) Crashes (2014) Crashes (2015) Crashes (2013-15) Length (Mile)

61 66 40 0 0 0 0 38 57 40 0 0 0 0 0.01

25435 25204 25023 7 10 7 19 25023 25868 26543 7 7 8 16 9.16

2922.174 2840.788 2823.525 0.073 0.074 0.069 0.217 2817.952 2915.032 3023.703 0.065 0.072 0.077 0.214 0.087

3116.796 3018.788 3063.711 0.315 0.323 0.305 0.672 3063.357 3153.594 3295.524 0.298 0.324 0.331 0.678 0.147

117298975 114032076 113339106 2948 2981 2768 8697 113115419 117012311 121374472 2596 2898 3090 8584 3505.69

Testing

Both

were used. All homogeneous roadway segments throughout six years of acquired HSIS data were identified, and then the first three-year crash data were used for training and the second three-year crash data were used for testing; therefore, the length information for Illinois and Washington data is provided for both training and testing data in a single row in the following tables (Table 7 and 8).

methods along with the HSM calibration method, some alternative calibration methods, and some alternatives combining CMF methods. Moreover, Washington models (SPFs and CMFs) were also examined. Maryland data were used as preliminary analysis of all existing and proposed methods. Some counterintuitive results were observed; among the existing methods, the power of “HSMPrediction” in calibration function (NPredicted = a × (HSMPrediction)b ) recommended by Srinivasan et al. (2016) became zero (b = 0.0) and the NPredicted = a for each roadway segment; thus, an arbitrary power of 0.6 was used for a somewhat meaningful comparison with other methods. This

5. Analysis This section summarizes the analysis of the proposed calibration 9

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

conditioned version of calibration function was named “Calibration Function (With Constraint).” Also, the power of “Length” parameter became zero in “Proposed-SC-2: Weights & powers for SPF parameters and CMFs” and “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″; therefore, those proposed methods with power function only for AADT parameter of the SPF were proposed and evaluated (“Proposed SC-1: Weights & powers for SPF parameters (Power only for AADT) and CMFs” and “Proposed SC-3: Weights & powers for SPF parameters (Power only for AADT) and CMFs ≠ 1″). However, none of these unexpected conditions were observed in Illinois and Washington; therefore, they are not reported in the following sections. Moreover, due to the size of the Illinois datasets (more than 16,000 roadway segments) and Washington datasets (more than 40,000 roadway segments) and the required processing time, the analysis on the proposed calibration methods was done on a selection of the methods as follows. The selected proposed method performed better in preliminary analysis based on Maryland data.

was the target of each estimation method for training and testing datasets and CURE plot percentages within ± 2σ boundaries. For example, for the “Proposed-S-3: Weights & powers for SPF parameters” method, the rankings of the method for the testing dataset when Max. LLNB was the estimation method were 1st (for SSD), 1st (for RMSE), 1st (for MAD), 2nd (for LLP), 1st (for LLNB), and 6th (for CURE plot percentage). Based on the table, the SPF-only approach had the poorest performance in comparison with other methods. If the performance of all methods is compared and those with somewhat counterintuitive cases are excluded, then “Proposed-S-3: Weights & powers for SPF parameters” could perform better than the rest. Examples of the excluded counterintuitive cases are “Proposed-SC-2: Weights & powers for SPF parameters and CMFs” and “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ because of the power of “Length” parameter became zero 0.0, and calibration function because of arbitrary assignment of a power value of 0.6 for “HSMPrediction.” 5.2. Illinois

• Proposed-C-3: Weights & powers for CMFs • Proposed-C-4: Weights & powers for CMFs ≠ 1 • Proposed-S-3: Weights & powers for SPF parameters • Proposed-SC-2: Weights & powers for SPF parameters and CMFs • Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1

Table 10 presents a comprehensive matrix of rankings of all methods based on the applied estimation method according to the measure that was the target of each estimation method for training and testing datasets and CURE plot percentages within ± 2σ boundaries. For example, for the “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ method, the rankings of the method for the testing dataset when Max. LLNB was the estimation method were 1st (for SSD), 1st (for RMSE), 1st (for MAD), 1st (for LLP), 1st (for LLNB), and 3rd (for CURE plot percentage). Based on the table, the SPF-only approach had the poorest performance in comparison with other methods. Comparing the performance of all methods the “Proposed-SC4: Weights & powers for SPF parameters and CMFs ≠ 1″ could perform better than the rest.

Also, the “SPF Only” method in the following sections refers to the predicted crashes based on only the HSM SPF (excluding all CMFs from the formulation). 5.1. Maryland Table 9 presents a comprehensive matrix of rankings of all methods based on the applied estimation method according to the measure that Table 9 Maryland: Comparison of rankings by method.

Notes: Total number of ranks was 37: 12 existing methods and five proposed methods by five different estimation methods each (12 + 5 * 5 = 37). When multiple methods shared the same value of measures then the same rank was assigned to all of them (e.g., Proposed-SC-1 had the same values of SSD measure for Min. SSD, Min. RMSE, and Min. MAD methods (training data); thus the same rank of 22 was assigned to all of them and the next rank was 25 of Proposed-SC-3: Min. MAD method. Some methods were excluded: the calibration function method because power of “HSMPrediction” was zero (0.0) and those proposed methods that power of “Length” parameter was zero (“Proposed-SC-2″ and “Proposed-SC-4″). 10

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 10 Illinois: Comparison of rankings by method.

Note: Total number of ranks were 42: twelve existing methods, calibration function and five proposed methods by five different estimation methods each (12 + (1 + 5) * 5 = 42).

5.3. Washington

51.2%, respectively; however, the “Proposed-S-3: Weights & powers for SPF parameters” and Max. LLNB as an estimation method could improve them significantly: 93% and 75.1%, respectively. In Illinois, the HSM calibration method had 2.7% within CURE plot boundaries for the training dataset and the testing dataset had a slightly better performance and could reach to 7.3% within the boundaries. These values for the calibration function method and Max. LLNB as an estimation method remained in the same ranges: 5.3% and 8.1%, respectively; however, the “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ and Max. LLNB as an estimation method could improve them significantly: 50.6% and 25.7%, respectively. In Washington, the HSM calibration method had 18.5% within CURE plot boundaries for the training dataset and the testing dataset performed better and could reach to 31.6% within the boundaries. These values for the calibration function method and Max. LLNB as an estimation method were 28.9% and 37%, respectively. The “ProposedSC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ and Max. LLNB as an estimation method had similar ranges: 31.16% and 37.3%, respectively. The calibrated Washington SPF and CMFs (Model 1) had the best value for the training dataset with 63.5%, but it was 37.3% for the testing dataset that was very close to the values of the calibration function and proposed method. Table 13 and 14 summarize the percent within CURE plot boundaries ( ± 2σ) and Log-likelihood for Negative Binomial for different calibration methods. The tables also show relative differences in percent within CURE plot boundaries ( ± 2σ) and Log-likelihood for Negative Binomial between different methods. In Maryland, the proposed method could improve the percent within CURE plot boundaries by 78.4% (vs. the HSM calibration method) and by 46.8% (vs. calibration function (with constraint)). Moreover, the proposed method could improve the Log-Likelihood by 18.6% (vs. the HSM calibration method) and by 4.5% (vs. calibration function (with constraint for power of

The previously developed Washington SPFs and CMFs based on HSIS data (2002-04) by Banihashemi (2011) are also included in the analysis: uncalibrated and calibrated using the training dataset (201012). Table 11 presents a comprehensive matrix of rankings of all methods based on the applied estimation method according to the measure that was the target of each estimation method for training and testing datasets and CURE plot percentages within ± 2σ boundaries. For example, for the “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ method, the rankings of the methods for the testing dataset when Max. LLNB was the estimation method were 7th (for SSD), 7th (for RMSE), 23rd (for MAD), 1st (for LLP), 1st (for LLNB), and 5th (for CURE plot percentage). Based on the table, the uncalibrated Washington models had the poorest performances, even worse than that of the SPF-only approach that had the poorest performance in Maryland and Illinois. Comparing the performance of all methods the “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ could perform better than the rest (by giving a priority to LLNB ranking) followed by the Washington models. 5.4. Comparison Table 12 summarizes the CURE plots developed based on training and testing datasets for the states of Maryland, Illinois, and Washington. In Maryland, the HSM calibration method had 3.3% within CURE plot boundaries for the training dataset; however, the testing dataset had a relatively better performance and it could reach to 42.1% within the boundaries. These values for the calibration function method (with constraint) and Max. LLNB as an estimation method were 5.2% and 11

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 11 Washington: Comparison of rankings by method.

Note: Total number of ranks was 46: 16 existing methods, calibration function and five proposed methods by five different estimation methods each (16 + (1 + 5) * 5 = 46).

HSMPrediction)). In Illinois, the proposed method could improve the percent within CURE plot boundaries by 250.9% (vs. the HSM calibration method) and by 67% (vs. calibration function). Moreover, the proposed method could improve the Log-Likelihood by 12.1% (vs. the HSM calibration method) and by 3.7% (vs. calibration function). In Washington, the proposed method could improve the percent within CURE plot boundaries by 18.3% (vs. the HSM calibration method); however, it was very close to the performance of the calibration function (only 0.8% improvement), and calibrated Washington SPF and CMFs (Model 1) were slightly better than the proposed method by 4%. However, the proposed method had minimal improvement for the LogLikelihood by 2.5% (vs. the HSM calibration method), by 0.6% (vs. calibration function), and by 2.5% (vs. calibrated Washington SPF and CMFs (Model 1)).

6. Findings The main findings based on the data from Maryland, Illinois, and Washington are summarized as follows:

• The proposed method of “Proposed-SC-4: Weights & powers for SPF • •

5.5. Case study

•

While the previous sections demonstrated the performance of proposed calibration methods at the aggregate level (statewide data), Table 15 shows the comparative performance of different calibration methods on subset data (corridor-level) from Illinois and Washington testing datasets. The availability of statewide data enabled the authors to conduct this case study. All roadway segments of a corridor highway were selected, and the performance of different calibration methods were examined. The “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ predicted crashes closer to observed crashes in comparison to the HSM calibration factor method and also calibration function methods in both states. Moreover, the proposed method outperformed calibrated Washington models in Washington State. The proposed method also had better performance measures compared to other methods in both states.

• • •

12

parameters and CMFs ≠ 1 “could perform better in terms of considered GOF measures in comparison with the HSM calibration method, calibration function, and calibrated Washington models (for the case of Washington data). Since the Max. LLNB method also estimates the inverse over-dispersion parameter, which can be used in the Empirical Bayes (EB) method, application of the “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ method and Max. LLNB as an estimation method is recommended. The estimation method had a significant impact on the CURE plots; those of Max. LLP and Max. LLNB had significantly higher percentages within recommended boundaries ( ± 2σ). The “SPF-only” approach performed poorly in comparison with other methods in all three states. Therefore, application of the CMFs can improve the prediction quality. The uncalibrated Washington models had the poorest performances; therefore, calibration is a “Must Do” for HSM application. The HSM method of combining CMFs (multiplicative) outperformed all alternative methods of combining CMFs in all performance measures in Maryland; however, its performance was mixed in the other two states and calls for further investigation. The CURE plots may visually mislead since the number of points on the plots differ at different AADT (or selected variable) intervals. Therefore, the CURE plots may not reveal much due to the size of data; thus, summary tables and statistical measures should be the primary basis of comparison.

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 12 CURE plots.

Notes: Blue line is ∑residuals and green and red lines are upper and lower limits ( ± 2σ). The estimation method of parameters for calibration function and proposed methods is Max. LLNB. Due to zero (0.0) as the power of “HSMPrediction” in the calibration function method for Maryland, an arbitrary power of 0.6 (as constraint) was used in the analysis.

Table 13 Comparison of percent within cure plot boundaries ( ± 2σ) for different calibration methods.

13

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 14 Comparison of log-likelihood for different calibration methods.

Table 15 Corridor-level case study. Case Study

Illinois

Change to HSM

Washington

Change to HSM

•

HSIS Road Inventory ID # Segments Total Length (Mile) Crash Data Period Observed Crashes HSM Calibration Predicted Factor Crashes SSD RMSE MAD LLP LLNB Calibration Predicted Function Crashes SSD RMSE MAD LLP LLNB Proposed-SC-4: Predicted Crashes Weights & powers for SPF SSD parameters and RMSE CMFs ≠ 1 MAD LLP LLNB Calibrated Predicted Washington Crashes Model 1 SSD RMSE MAD LLP LLNB Calibrated Predicted Washington Crashes Model 2 SSD RMSE MAD LLP LLNB

175033 105 26.35 2008-10 242 142.38

– – – – – –

525 205 20.45 2013-15 123 142.62

– – – – – –

•

576.092 2.342 1.452 31.520 45.056 150.45

– – – – – –

159.992 0.883 0.635 −155.608 −158.580 137.82

– – – – – –

554.099 2.297 1.426 49.173 59.356 151.84

−3.8% −1.9% −1.7% 56.0% 31.7% –

156.766 0.874 0.626 −154.757 −149.933 128.70

−2.0% −1.0% −1.4% 0.5% 5.5% –

530.957 2.249 1.368 61.904 69.803

−7.8% −4.0% −5.8% 96.4% 54.9%

156.492 0.874 0.608 −151.365 −147.909 132.78

−2.2% −1.1% −4.2% 2.7% 6.7% –

154.924 0.869 0.607 −152.653 −157.978 135.08

−3.2% −1.6% −4.4% 1.9% 0.4% –

157.188 0.876 0.613 −153.238 −158.163

−1.8% −0.9% −3.6% 1.5% 0.3%

type 5 (CMFs as a power function) did not improve the calibration quality. As discussed earlier, we improved the methodology, reducing such limitations. The results of the corridor-level case study indicate that applying the proposed calibration method can produce improved prediction estimations at the aggregate and disaggregate levels which are certainly of interest to both researchers and practitioners. The estimated parameters are provided in Table 16 and formulation of different methods is shown in Table 17. More information can be found in (Dadvar, 2018b).

7. Conclusions Although the HSM calibration procedure assures total predicted crashes be localized, still prediction of crashes for individual locations suffers from an aggregate localization process. The proposed calibration method incorporates multiple calibration factors (i.e., a calibration function) for different components of the predictive method (SPF parameters and CMFs) rather than a single calibration factor as recommended by the HSM that only calibrates at the aggregate level. In the proposed method, the application of calibration factors expressed in both weight and power function better reflects the local conditions while ensuring calibration at the aggregate level. The parameters are estimated through an optimization process of five different methods (Min. SSD, Min. RMSE, Min. MAD, Max. Log Likelihood for Poisson Regression, and Max. Log Likelihood for Negative Binomial Regression) using Microsoft Excel Solver module. The proposed method has at least two main benefits: First, the local jurisdictions will benefit from more reliable crash estimates. Second, the improved calibration process keeps the HSM from being reduced to a guideline and all models in Part C chapters from becoming useless due to unreliable calibration. The proposed approach was validated using the rural two-lane, twoway roads (R2U) roadway data from the states of Maryland (three years; 2008-10), Illinois (six years; 2005-10), and Washington (six years, 2010-15). Comparing different GOF measures along with CURE plots showed that the proposed method of “Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1″ could perform significantly better than the HSM calibration method and calibration function (that will most likely be calibration process in the HSM 2nd edition) while keeping the total number of predicted crashes equal to the total number of observed crashes. Moreover, the results indicated that the additional parameters for CMFs could improve the prediction significantly; Srinivasan et al. (2016) did not find this to be so since data limitations caused them to use the default values for 10 out of 12 R2U CMFs. We improved the methodology and reduced these limitations. Although the proposed method in the study may not be the most comprehensive and

• The results indicated that the additional parameters for CMFs could

improve the performance significantly; Srinivasan et al. (2016) did not find this to be so since data limitation caused them to use the default values for 10 out of 12 CMFs of R2U, and using the base condition values for nearly all CMFs did not seem a reasonable assumption. Perhaps because of this issue their calibration function 14

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

Table 16 Summary of estimated parameters for different methods.

Note: Due to zero (0.0) as the power of “HSMPrediction” in the calibration function method for Maryland, an arbitrary power of 0.6 (as constraint) was used in the analysis.

sound, it can provide more reliable results in comparison with the HSM calibration and calibration function methods. Application of the proposed approach can lead to more accurate identification of hot-spots and site-specific strategies in terms of funding allocation. Moreover, a corridor-level case study was conducted on data from Illinois and Washington and the results indicated that applying the proposed calibration method produced improved prediction estimations at the aggregate and disaggregate levels in comparison with other calibration methods; researchers and practitioners from other jurisdictions can implement the proposed calibration method and benefit from more reliable predictions. The developed “Road Safety Data Integrator (RSDI)” tool can be helpful for combining different safety-related datasets such as roadway inventory (including grade, curve, and other subsets), traffic volume, and crash data; moreover, it can do required segmentations and identify the homogeneous roadway segments over the desired years of study that are the basis for development and calibration of the HSM predictive models. The RSDI tool can be used for similar purposes and is not only limited to the HSIS data. It can be used for segmentation and finding homogeneous segments of any datasets that follow linear referencing. With that said, there are several different avenues for further research. The HSM SPF of R2U, which was the basis for developing and comparing the proposed calibration methods, is the only SPF without power for AADT parameter of the SPF (all other SPFs in the HSM have power value for AADT in the format of “× LN (AADT ) = LN (AADT b ) ”), while the proposed calibration methods involving power for AADT parameter can still be applied as “[(b + p1 ) × LN (w1 × AADT ) = LN [(w1 × AADT )(b + p1) ]” ; further investigations are recommended. The methodology can be transferred to any study area with a currently available sampling data for the

HSM calibration method, and sample size adjustments can be done following the guidelines that were discussed in our previous works (Shin et al., 2013, Shin et al., 2015b). The methodology also can be applied during the process of developing LCFs for the HSM and eventually it can also be applied to future crash predictions; however, due to the issues that were observed based on somewhat limited Maryland data for both proposed methods (zero (0.0) power for “Length” parameter of SPF) and also calibration function (zero (0.0) power for “HSMPrediction”), a follow-up study on the sample size requirements of proposed methods and also calibration function is recommended. The proposed methods were applied on the HSM method (i.e., multiplicative) of combining CMFs (i.e., applying weights and/or powers to CMFCombined =CMF1 × CMF2 × …×CMFn ); however, in further research, alternative methods, some of which were examined in this study, also can be of interest. Trying other formats of combining/calibrating multiple CMFs like the development of a regression model based on the optimal values for each individual CMF may be worthwhile. Moreover, this study only targeted the CMFs that apply to the total crashes; other similar or dissimilar approaches may be followed for other crash severity levels. While the proposed method may be a reasonable substitution for developing local CMFs, which may be cost-prohibitive, further investigations are warranted. Also, in this study, the authors did not modify the base conditions for HSM Part C CMFs; however, some studies (Qin, et al. (2014; 2016)) suggested benefits from defining new base conditions for new locally estimated SPF models and adjusting the base conditions. Further investigation is recommended. The validation task was carried out based on a conventional validation method (dividing the data into two subsets); however, in future research, more robust validation methods like cross-validation can be considered. The estimations in this study were done with Microsoft Excel Solver; although the results supported the hypotheses, using more robust optimization tools is

15

16

Calibrated Washington Model 1 Calibrated Washington Model 2

Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1

Local Calibration Factor Local Calibration Function

Proposed-SC-4: Weights & powers for SPF parameters and CMFs ≠ 1 6

× e(

× e(

0.312)

6

0.312)

× CMFC

6

× e(

0.312)

Note: Calculating weights and powers based on CMFs ≠ 1 and applying to only CMFs ≠ 1. NPredicted = 0.6495 × (WA Model 1Prediction ) NPredicted = 0.6464 × (WA Model 2Prediction)

1.0000 1.0000 1.0000 × (1.0000 × CMF10 ) × (1.0000 × CMF11 ) × (1.0000 × CMF12 ))

) × (1.0000 × CMF81.0000) × (1.1015 × CMF90.9976)

× CMF50.9927) × (1.0000 × CMF61.0000) × (1.0000 × CMF71.0000

×(1.0000 × CMF41.0000) × (1.0654

× ((1.1447 × CMF11.0033) × (1.0366 × CMF21.5740) × (0.9379 × CMF30.6561)

1.4010 × AADT 0.9422 × L0.9929 × 365 × 10

NPredicted =

NPredicted = 0.8440 × (HSMPrediction )0.9521

Note: Calculating weights and powers based on CMFs ≠ 1 and applying to only CMFs ≠ 1. NPredicted = 0.9195 × (HSMPrediction)

1.0000 1.0000 1.0034 × (1.0000 × CMF10 ) × (1.0000 × CMF11 ) × (1.0000 × CMF12 ))

) × (2.5255 × CMF80.3444) × (0.7319 × CMF91.0074 )

× CMF51.0000) × (1.0000 × CMF61.0006) × (0.4912 × CMF71.0133

×(1.0000 × CMF41.0000) × (1.0000

× ((1.1701 × CMF10.0000) × (2.1154 × CMF20.8443) × (1.2633 × CMF30.0000)

16.0307 × AADT 0.6324 × L0.8925 × 365 × 10

NPredicted =

NPredicted = 1.6901 × (HSMPrediction )0.8218

NPredicted = 1.8068 × (HSMPrediction)

NPredicted = 278.87 × AADT 0.1895 × L0.1232 × 365 × 10

NPredicted = 0.8228 × (HSMPrediction )0.6000

NPredicted = 0.8116 × (HSMPrediction)

Formulation

Notes: HSMPrediction = Uncalibrated HSM predicted crashes, CMFC = Combined CMF, WA Model 1Prediction = Uncalibrated predicted crashes of Washington Model 1. WA Model 2Prediciton = Uncalibrated predicted crashes of Washington Model 2.

Washington

Illinois

Local Calibration Factor Local Calibration Function

Maryland

Proposed-S-3: Weights & powers for SPF parameters Local Calibration Factor Local Calibration Function

Method

State

Table 17 Summary of formulation of different methods.

S. Dadvar, et al.

Accident Analysis and Prevention 136 (2020) 105393

Accident Analysis and Prevention 136 (2020) 105393

S. Dadvar, et al.

recommended for further research.

Administration, U.S. Department of Transportation: Federal Highway Administration, Washington, D.C. Hauer, E., 2015. The Art of Regression Modeling in Road Safety. Springer. Hauer, E., Bamfo, J., 1997. Two Tools for Finding What Function Links the Dependent Variable to the Explanatory Variables. International Cooperation on Theories and Concepts in Traffic Safety, Lund, Sweden. Hauer, E., Bonneson, J.A., Council, F., Srinivasan, R., Zegeer, C., 2012. Crash modification factors foundational issues. Transp. Res. Record: J. Transp. Res. Board 2279, 67–74. Ivan, J.N., Persaud, B., Srinivasan, R., Abdel-Aty, M., Lyon, C., Al Mamun, S., et al., 2017. Improved Prediction Models for Crash Types and Crash Severities [Draft Final Report]. Transportation Research Board of the National Academies., Washington, D.C. Lacy, J.K., 2001. Recommended Procedure for Combining Crash Reduction Factors. North Carolina Department of Transportation, Raleigh, North Carolina. Llopis-Castelló, D., Findley, D.J., Camacho-Torregrosa, F.J., García, A., 2019. Calibration of inertial consistency models on North Carolina two-lane rural roads. Accid. Anal. Prev. 127, 236–245. https://doi.org/10.1016/j.aap.2019.03.013. Mehta, G., Lou, Y., 2013. Calibration and development of safety performance functions for Alabama - two-lane, two-way rural roads and four-lane divided highways. Transp. Res. Record: J. Transp. Res. Board 2398, 75–82. https://doi.org/10.3141/2398-09. Park, J., Abdel-Aty, M., 2015a. Development of adjustment functions to assess combined safety effects of multiple treatments on rural two-lane roadways. Accid. Anal. Prev. 75 (February), 310–319. Park, J., Abdel-Aty, M., 2015b. Assessing the safety effects of multiple roadside treatments using parametric and nonparametric approaches. Accid. Anal. Prev. 83 (October), 203–213. Park, J., Abdel-Aty, M.A., 2017. Alternative approach for combining multiple crash modification factors using adjustment function and analytic hierarchy process. Transp. Res. Record: J. Transp. Res. Board 2636, 15–22. https://doi.org/10.3141/ 2636-03. Park, J., Abdel-Aty, M., Lee, C., 2014. Exploration and comparison of crash modification factors for multiple treatments on rural multilane roadways. Accid. Anal. Prev. 70, 167–177. Persaud, B., Lord, D., Palmisano, J., 2002. Calibration and transferability of accident prediction models for urban intersections. Transp. Res. Record: J. Transp. Res. Board 1784, 57–64. Qin, X., Chen, Z., Vachal, K., 2014. Calibration of highway safety manual predictive methods for rural local roads. In: 93rd Annual Meeting of the Transportation Research Board. Washington, D.C. Qin, X., Shaon, M.R., Chen, Z., 2016. Developing analytical procedures for calibrating the highway safety manual predictive methods. Transp. Res. Record: J. Transp. Res. Board 2583, 91–98. https://doi.org/10.3141/2583-12. Rajabi, M., Ogle, J.H., Gerard, P., 2018. Highway safety manual calibration: assessing alternate definitions of the calibration factor. In: Transportation Research Board 97th Annual Meeting. Washington, D.C.: Transportation Research Board (TRB). Roberts, P., Turner, B., 2007. Estimating the crash reduction factor from multiple road engineering countermeasures. In: 3rd International Road Safety Conference. Perth, Australia. Sawalha, Z., Sayed, T., 2006. Transferability of accident prediction models. Saf. Sci. 44, 209–219. Shin, H.-S., Dadvar, S., Lee, Y.-J., 2015a. Results and lessons from the local calibration process of the highway safety manual for the state of Maryland. Transp. Res. Record: J. Transp. Res. Board 2515, 104–114. Shin, H.-S., Dadvar, S., Lee, Y.-J., Tao, R., 2013. Impacts of segment length and different sampling strategies on local calibration factors for the highway safety manual. In: Transportation Research Board, 92nd Annual Meeting. Washington, D.C.. Shin, H.-S., Lee, Y.-J., Dadvar, S., 2014. The Development of Local Calibration Factors for Implementing the Highway Safety Manual in Maryland. Baltimore. Maryland State Highway Administration., MD. Shin, H.-S., Yu, J.W., Dadvar, S., Lee, Y.-J., 2015b. Statistical evaluation of different sample sizes for local calibration process in the highway safety manual. Transp. Res. Record: J. Transp. Res. Board 2515, 94–103. Smith, S., Carter, D., Srinivasan, R., 2017. Updated and Regional Calibration Factors for Highway Safety Manual Crash Prediction Models. Highway Safety Research Center University of North Carolina at Chapel Hill. Raleigh: North Carolina Department of Transportation, North Carolina. Srinivasan, R., Carter, D., Bauer, K., 2013. Safety Performance Function Decision Guide: SPF Calibration Vs SPF Development. Federal Highway Administration Office of Safety, Washington, D.C. Srinivasan, R., Colety, M., Bahar, G., Crowther, B., Farmen, M., 2016. Estimation of calibration functions for predicting crashes on rural two-lane roads in Arizona. Transp. Res. Record: J. Transp. Res. Board 2583, 17–24. https://doi.org/10.3141/2583-03. Srinivasan, S., Haas, P., Dhakar, N.S., Hormel, R., Torbic, D., Harwood, D., 2011. Development and Calibration of Highway Safety Manual Equations for Florida Conditions. Florida Department of Transportation. Venkataraman, N., Shankar, V., Blum, J., Hariharan, B., Hong, J., 2016. Transferability analysis of heterogeneous overdispersion parameter negative binomial crash models. Transp. Res. Record: J. Transp. Res. Board 2583, 99–109. https://doi.org/10.3141/ 2583-13.

Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements The authors would like to thank Morgan State University’s National Transportation Center for its funding support and continued encouragement of student participation in research projects. The authors also would like to thank the Maryland Department of Transportation State Highway Administration (SHA) for funding and providing assistance in data collection. References American Association of State Highway and Transportation Officials, 2010. Highway Safety Manual, 1st ed. AASHTO., Washington, D.C. American Association of State Highway and Transportation Officials, 2014. Highway Safety Manual, 1st ed. AASHTO., Washington, D.C. Bahar, G.B., 2010. Methodology for the Development and Inclusion of Crash Modification Factors in the First Edition of the Highway Safety Manual. Transportation Research Board (TRB), Washington, D.C. Bahar, G.B., Hauer, E., 2014. User’s Guide to Develop Highway Safety Manual Safety Performance Function Calibration Factors. Transportation Research Board of the National Academies., Washington, D.C. Banihashemi, M., 2011. Highway safety manual, new model parameters vs. Calibration of crash prediction models. In: Transportation Research Board 90th Annual Meeting. Washington, D.C. Carter, D., Srinivasan, R., Gross, F., Himes, S., Le, T., Persaud, B., et al., 2017. Guidance for the Development and Application of Crash Modification Factors NCHRP Project 17-63 [Draft Final Report, LIMITED USE DOCUMENT]. Transportation Research Board of the National Academies., Washington, D.C. Claros, B., Sun, C., Edara, P., 2018. HSM calibration factor, calibration function, or jurisdiction-specific safety model - how to choose the approach? In: Transportation Research Board 97th Annual Meeting. Washington, D.C.: Transportation Research Board (TRB). Colety, M., Crowther, B., Farmen, M., Bahar, G., Srinivasan, R., 2016. ADOT StateSpecific Crash Prediction Models: An Arizona Needs Study. Arizona Department of Transportation, Phoenix. Dadvar, S., 2018a. Improving crash predictability of the highway safety manual through alternate local calibration process. In: Transportation Research Board, 97th Annual Meeting. Washington, D.C. Dadvar, S., 2018b. Improving Crash Predictability of the Highway Safety Manual Through Alternate Local Calibration Process. Morgan State University., Baltimore, Maryland. Dadvar, S., Khodaparasti, H., 2018. Roadway Safety Data Integrator (RSDI) Tool. Retrieved October 16, 2018, from Some Parts of My World. https://www. ehsandadvar.com/2018/10/roadway-safety-data-integrator-rsdi-tool.html. Dadvar, S., Shin, H.-S., Lee, Y.-J., 2016. Local calibration of crash modification factors to improve predictability of highway safety manual. In: Transportation Research Board 95th Annual Meeting. Washington, D.C. Dixon, K., Monsere, C., Xie, F., Gladhill, K., 2012. Calibrating the Future Highway Safety Manual Predictive Methods for Oregon State Highways. Department of Transportation, Oregon. Elvik, R., 2009. An exploratory analysis of models for estimating the combined effects of road safety measures. Accid. Anal. Prev. 41, 880–976. Farid, A., Abdel-Aty, M., Lee, J., 2018. Transferring and calibrating safety performance functions among multiple States. Accid. Anal. Prev. 117, 276–287. https://doi.org/ 10.1016/j.aap.2018.04.024. Federal Highway Administration (FHWA), 2016. Crash Modification Factors Clearinghouse. Retrieved January 2016, from. www.cmfclearinghouse.org. Federal Highway Administration (FHWA), 2018. Retrieved January 15, 2018, from Highway Safety Information System (HSIS): https://www.hsisinfo.org/. Geedipally, S.R., Shirazi, M., Lord, D., 2017. Exploring the need for region-specific calibration factors. Transp. Res. Record: J. Transp. Res. Board 2636, 73–79. https://doi. org/10.3141/2636-09. Gross, F., Hamidi, A., 2011. Investigation of Existing and Alternative Methods for Combining Multiple CMFs. Hagen, L., 2015. Use and misuse of crash modification factors. Local Agency Traffic Safety Academy (LATSA) Presentations. Department of Transportation, Florida February. Harwood, D.W., Council, F.M., Hauer, E., Hughes, W.E., Vogt, A., 2000. Prediction of the Expected Safety Performance of Rural Two-Lane Highways. Federal Highway

17