--- category: literaturenote citekey: hadayeghitemporaltransferabilityupdatingzonal2006 title: Temporal transferability and updating of zonal level accident prediction models authors: "Hadayeghi, Alireza; Shalaby, Amer S.; Persaud, Bhagwant N.; Cheung, Carl" year: 2006 date: 2006-00-00 2006 doi: 10.1016/j.aap.2005.12.003 publication: "Accident Analysis & Prevention" url: "https://doi.org/10.1016/j.aap.2005.12.003" zotero_key: ZUN7M7TU zotero_storage: B8H9E7F4 collections: transpordi_planeerimine folder: 001_artiklid/Road Safety Indicators firstAuthor: "Hadayeghi, Alireza" status: converted --- ![](_page_0_Picture_0.jpeg) Accident Analysis and Prevention 38 (2006) 579–589 ![](_page_0_Picture_2.jpeg) # Temporal transferability and updating of zonal level accident prediction models Alireza Hadayeghi a,∗, Amer S. Shalaby b,1, Bhagwant N. Persaud c,2, Carl Cheung c,3 a *Department of Civil Engineering, University of Toronto and Synectics Transportation Consultants Inc., 36 Hiscott Street, St. Catharines, Ont., Canada L2R 1C8* b *University of Toronto, 35 St. George Street, Toronto, Ont., Canada M5S 1A4* c *Ryerson University, 350 Victoria Street, Toronto, Ont., Canada M5B 2K3* Received 3 March 2005; received in revised form 22 September 2005; accepted 5 December 2005 #### **Abstract** This paper examines the temporal transferability of the zonal accident prediction models by using appropriate evaluation measures of predictive performance to assess whether the relationship between the dependent and independent variables holds reasonably well across time. The two temporal contexts are the years 1996 and 2001, with updated 1996 models being used to predict 2001 accidents in each traffic zone of the City of Toronto. The paper examines alternative updating methods for temporal transfer by imagining that only a sample of 2001 data is available. The sensitivity of the performance of the updated models to the 2001 sample size is explored. The updating procedures examined include the Bayesian updating approach and the application of calibration factors to the 1996 models. Models calibrated for the 2001 samples were also explored, but were found to be inadequate. The results show that the models are not transferable in a strict statistical sense. However, relative measures of transferability indicate that the transferred models yield useful information in the application context. Also, it is concluded that the updated accident models using the calibration factors produce better results for predicting the number of accidents in the year 2001 than using the Bayesian approach. © 2005 Elsevier Ltd. All rights reserved. *Keywords:* Temporal transferability; Model calibration; Bayesian approach; Safety planning models #### **1. Introduction** Accident prediction models are used to predict the number of accidents at intersections, on arterial road sections, on highways or on transportation networks. Of late, there is considerable interest in transportation planning applications, for which these models (usually developed at the planning zone level) are aimed at establishing a relationship between accident frequency and a number of explanatory variables such as traffic volume, road network characteristics and socioeconomic and demographic features. It is important for safety planning models to enable transportation planners to assess the safety implications of alternative transport planning options that are considered for medium to long-range implementation, in a way similar to how road designers make use of tools such as the Interactive Highway Safety Design Model (IHSDM)4 to assess the safety implications of design decisions. For this reason and others, it is desirable to obtain reasonably accurate safety planning models that are transferable across time and have good predictive performance. In general, the importance of accident prediction models makes it crucial that they be properly calibrated ([Persaud et al.,](#page-10-0) [2002\).](#page-10-0) The quantity and quality of the data used to calibrate accident prediction models is very important to the success or failure of these models. Models calibrated with limited data will not perform adequately in terms of predictive accuracy. Therefore, having a reasonable dataset is very important in the development of accident prediction models. Moreover, the specification of an ∗ Corresponding author. Tel.: +1 905 704 0763; fax: +1 905 682 4495. *E-mail addresses:* ahadayeghi@synectics-inc.net (A. Hadayeghi), amer@ecf.utoronto.ca (A.S. Shalaby), bpersaud@ryerson.ca (B.N. Persaud), c22cheun@ryerson.ca (C. Cheung). 1 Tel.: +1 416 978 5907; fax: +1 416 978 5054. 2 Tel.: +1 416 979 5000x6464; fax: +1 416 979 5122. 3 Tel.: +1 416 979 5000; fax: +1 416 979 5122. 4 Interactive Highway Safety Design Model (IHSDM), Federal Highway Administration, US Department of Transportation. appropriate functional model form for calibration of accident prediction models is a complex task. It gets even more complicated for safety planning models because of the paucity of these. Data for transportation planning variables are usually obtained from surveys held periodically (for example, the Transportation Tomorrow Survey is conducted once every 5 years in the Greater Toronto Area). These surveys usually cover a comprehensive area and are therefore expensive in terms of the time and cost associated with data collection and analysis. As such, once a model is well specified to capture the decision process in one context, it is cost effective to transfer such a model to other contexts that have limited calibration data as long as the basic nature of the decision-making process remains the same [\(Badoe](#page-9-0) [and Miller, 1995a\).](#page-9-0) There are two aspects to model transferability—temporal and spatial. The first, results from application of a model estimated at one point of time for prediction at another point of time for the same spatial environment. This occurs whenever estimated models are applied in a forecasting framework. The second aspect – spatial transferability – involves the application of a model estimated in a specific geographic region for prediction in a different one for the same time period. Of course, there can be a combination of temporal and spatial transfers. Several research efforts have been conducted to address the temporal and spatial transferability of transportation planning models. Most of this research was on trip generation models and disaggregate and aggregate mode choice models [\(Ben-Akiva,](#page-9-0) [1981; Koppelman and Wilmot, 1982; Atherton and Ben-Akiva,](#page-9-0) [1976; Gunn et al., 1985; Gunn, 2001; Badoe and Miller, 1995b;](#page-9-0) [Ben-Akiva and Bolduc, 1987; Parody, 1977\).](#page-9-0) However, no similar efforts have been directed at safety planning models. This is likely in reflection of the reality that few safety planning models have been calibrated to date. A literature search indicated that only few studies have been undertaken to evaluate and quantify traffic safety impacts in the planning process [\(Hadayeghi et al., 2003; Lovegrove and Sayed, 2005;](#page-9-0) [Ladron de Guevara et al., 2004; NCHRP 8-44, in press\).](#page-9-0) Of most relevance to the current effort was some work by [Oh et al. \(2003\)](#page-9-0) that validated intersection accident prediction models intended for use in the IHSDM from two perspectives—internal validity and external validity. Internal model validity examines the strength of the original models in explaining the underlying phenomenon. External validity examines the ability of the models to predict crashes over space and time. The authors recommended that the validation over time could suffer from omitted variables that influence safety, poorly measured and surrogate variables, and model functional forms. On the other hand, the authors argue that internal validity and validation over space could suffer from omitted variables, site selection and countermeasure selection bias, poorly measured and surrogate variables, and misspecification of model functional form. This paper has two basic objectives. First, it examines the temporal transferability of the accident prediction models by using appropriate evaluation measures of predictive performance to assess whether the relationship between the dependent and independent variables holds reasonably well across time. The two temporal contexts are the years 1996 and 2001, with models estimated from 1996 data being used to predict 2001 accidents. A series of accident prediction models based on 1996 database were calibrated in the previous study ([Hadayeghi et al.,](#page-9-0) [2003\)](#page-9-0) to estimate the number of accidents in planning zones in the City of Toronto as a function of planning variables. It should be stressed that the spatial transferability of accident prediction models is outside the scope of this paper. However, some insights on this issue have been provided in some other papers [\(Persaud](#page-10-0) [et al., 2002\).](#page-10-0) The second objective of the paper is to compare alternative updating methods of temporal transfer and to estimate effects of the size of the application context data on model transferability. The updating procedures examined are the Bayesian updating and updating based on recalibration factors. Models calibrated for the 2001 samples were also explored, but were found to be inadequate, so were not tested in the 2001 application context. In addition, this study investigates the transfer bias caused by sample size. This issue was not addressed in previous studies which focused mainly on using only one method and one sample size. This is important since it is believed a large portion of differences in coefficients and predictions is caused by random variation due to selection of data. Also, the goodness of fit measures resulting from updated models are compared to the models estimated using the entire dataset in the application context. A key contribution of this paper is the variety in the validation methods and measures used, tested and compared. #### **2. Data** The data used in this study are comprised of accident, socioeconomic, demographic and road network characteristics, and traffic flow for the City of Toronto's 463 traffic zones for 1996 and 481 traffic zones for 2001. The zone definitions for 1996 and 2001 are not identical. For the purpose of this study, the 2001 zone definition is adopted for both 1996 and 2001. The data for this study were obtained from three sources. The accident data for 1996 and 2001, the first dataset, was provided electronically by the Traffic Data Centre (TDC) of the City of Toronto. The accident data are geocoded, which allows the grouping of accidents by traffic zone using GIS tools. The relevant information for each accident record includes severity (fatal or non-fatal injury and property damage only (PDO)) and period of day. The latter allows for separate models to be developed for the morning peak period (6:00–9:00 am) to which traffic intensity data pertained. The second dataset, consisting of socioeconomic and demographic data, was obtained from the 1996 and 2001 Transportation Tomorrow Survey (TTS), a 1-day travel survey conducted in the Greater Toronto and surrounding areas (GTA) on approximately 5% of its population. The survey is conducted to provide detailed demographic and travel information of GTA household members to local, regional, provincial road authorities and to transit operating agencies. The traffic demand data were obtained from the results of a traffic assignment of the morning peak-period vehicle trips to the road network using the [EMME/2](#page-9-0) [\(1998\)s](#page-9-0)oftware package to estimate vehicle kilometers travelled (VKT) in each zone. This approximation is necessary because cordon counts, the only available source of traffic demand data collected consistently across the city, are made at a limited number of locations and therefore cannot be used directly and sufficiently to provide traffic flow estimates for each zone in the city. It is worth mentioning that it is a common practice in building travel demand and planning models to use traffic assignment model outputs. In addition to the above data, 1996 GIS road files were obtained from the Integrated Land Use, Transport and Environment (ILUTE) group at the University of Toronto, and the corresponding 2001 database, CanMap Street Files, was attained from University of Toronto Map Library. #### **3. Comparison of 1996 and 2001 conditions** Table 1 presents the magnitude of changes in various characteristics in the City of Toronto between 1996 and 2001. The number of households grew by 3.68%, from 908,499 in 1996 to 943,252 in 2001. The population of the City of Toronto grew by 2.7% in the 5-year period, while the total population of the Greater Toronto Area (GTA) increased by 9.3%. The difference in percent growth between the GTA and City of Toronto is indicative of the reality that more people lived in the suburbs in 2001 as a result of urban sprawl. During this period, the total employment in the City of Toronto increased by more than 7%. Total vehicle kilometers travelled (VKT) per person increased by 15%, from 0.75 in 1996 to 0.88 in 2001. However, there was little change in total major road kilometers in the same period, suggesting that traffic conditions should have worsened from 1996 to 2001. This observation is confirmed by the average volume to capacity (*V*/*C*) ratios in Table 1, which increased by almost 30% from 1996 to 2001. The percent average changes in zonal traffic volume from 1996 to 2001 in the City of Toronto, based on the 1996 zonal definitions, are presented in [Fig. 1](#page-3-0) . As seen, most of the zones experienced an increase in traffic volume in the 5-year period. Table 1 also shows that the total number of accidents increased by 27% from 1996 to 2001, an average annual increase of 5.41%. On the other hand, the number of severe (i.e., fatal plus injury) accidents increased only by a total of 5.64% in the 5-year period. [Fig. 2](#page-3-0) presents the zonal percentage change in total number of accidents for the City of Toronto. As can be seen, most of the zones outside of downtown have experienced a higher percentage change in total number of accidents than that for the downtown zones. ## **4. Model specification** A series of macro-level, zonal accident prediction models that estimate the number of zonal accidents in the City of Toronto for 1996 was developed in the work described in [Hadayeghi et al.](#page-9-0) [\(2003\). A](#page-9-0) generalized linear modelling approach was employed, in which negative binomial regression models were developed separately for total accidents and for severe (fatal and non fatal injury) accidents as a function of socioeconomic/demographic, traffic demand and network data variables. A total of 22 possible explanatory variables describing zonal characteristics were explored. The definition of variables was based on (a) previous research for similar studies and (b) an iterative process in which various definitions were tried and the ones that gave the best results were ultimately selected. The following model form was used $$E(A) = a_0 \text{ VKT}^{b_0} \exp \sum_{i=1}^{n} b_i X_i$$ (1) Table 1 Change between 1996 and 2001 data in the City of Toronto | Attribute for independent variables | 1996
2001 | | Difference | Percent change (%) | | |-----------------------------------------------------------------------------------|--------------|---------|------------|--------------------|--| | (a) Changes in urban structure, travel behavior and socioeconomic and demographic | | | | | | | Vehicle kilometer travelled (VKT) | 1728775 | 2089257 | 360482 | 17.25 | | | Average volume capacity (V/C) | 0.22 | 0.31 | 0.09 | 30.18 | | | Average posted speed | 42.08 | 42.48 | 0.40 | 0.95 | | | Total population | 2305600 | 2368700 | 63100 | 2.66 | | | Number of households | 908499 | 943252 | 34753 | 3.68 | | | Number of part-time employees | 181224 | 187890 | 6666 | 3.55 | | | Number of full-time employees | 875225 | 943373 | 68148 | 7.22 | | | Total number of employees | 1056449 | 1131263 | 74814 | 6.61 | | | Total minor road kilometer | 4172.66 | 4406.24 | 233.58 | 5.30 | | | Total major road kilometer | 859.83 | 900.73 | 4.90 | 0.54 | | | Total number of intersections | 15498 | 16649 | 1151 | 6.91 | | | Attribute for dependent variables | 1996 | 2001 | Difference | Percent change (%) | | | (b) Change in Number of Accidents | | | | | | | Total accidents | 53286 | 73043 | 19757 | 27.1 | | | Severe (fatal and injury) accidents | 15818 | 16764 | 946 | 5.64 | | | Total accidents, morning peak period | 5317 | 8691 | 3374 | 38.82 | | | Severe accidents, morning peak period | 1664 | 1971 | 307 | 15.58 | | ![](_page_3_Figure_2.jpeg) Fig. 1. Percent change in average traffic volume from 1996 to 2001 in the City of Toronto based on 1996 zonal definitions. ![](_page_3_Figure_4.jpeg) Fig. 2. Percent change in total number of accidents from 1996 to 2001 in the City of Toronto based on 1996 zonal definitions. Table 2 Results of negative binomial regression for models I and II | Parameter | Yearly total accident model (Model I) | | Yearly severe accident model (Model II) | | | |-------------------------------|---------------------------------------|------------------------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--| | | 1996 Model | 2001 Model | 1996 Model | 2001 Model | | | Intercept (log of α0) | 4.1767 (0.2957, 199.47) | 2.8065 (0.3572, 61.75) | 2.6618 (0.3239, 67.55) | 1.9593 (0.3736, 27.50) | | | Log of VKT | 0.2656 (0.0347, 58.76) | 0.2845 (0.0540, 27.81) | 0.2415 (0.039, 38.41) | 0.1739 (0.0561, 9.60)
0.2423 (0.0208, 136.31)
0.0650 (0.0162, 16.03)
−0.0096 (0.0067, 2.07)
−0.3963 (0.3193, 1.54)
0.0013 (0.0012, 1.03) | | | Total major road in kilometer | 0.1175 (0.0186, 40.05) | 0.1960 (0.0197, 93.98) | 0.1225 (0.0202, 36.77) | | | | Number of households (× 10−3) | 0.0864 (0.015, 33.27) | 0.0582 (0.0156, 13.89) | 0.0858 (0.0164, 27.38) | | | | Speed | −0.0431 (0.0071, 36.61) | −0.0124 (0.0065, 3.64) | −0.0312 (0.0078, 16.05) | | | | Volume/capacity (V/C) | −1.3678 (0.2766, 24.45) | −0.6472 (0.3112, 4.33) | −1.2232 (0.3042, 16.18) | | | | Intersection density | 0.0052 (0.0012, 19.87) | 0.0024 (0.0012, 3.98) | 0.0038 (0.0013, 9.14) | | | | Goodness of fit measures | | | | | | | Mean Pearson χ2/d.f. | 1.0313 | 1.1559 | 1.0346 | 1.0110 | | | Pearson R2 | 37.48% | 37.24% | 31.90% | 43.06%
0.2296 | | | α | 0.1941 | 0.2355 | 0.2092 | | | *Note*: Refer to Eq. [\(1\). T](#page-2-0)he dependent variable is the natural logarithm of yearly total accidents. Parameter estimate (S.E., χ2). where *E*(*A*) is the predicted accident frequency per zone per year, VKT the zonal vehicle kilometers travelled during peak hours (6:00–9:00 am), *Xi* the other explanatory variables, *a*0, *b*0 and *bi* is the model parameters The above model form is consistent with the non-linear relationship between accidents and traffic intensity (measured in this case by VKT) suggested by several other researchers. Separate models are calibrated for total and severe (fatal plus non-fatal injury) accidents per year and, for each, models were estimated for the entire day and for the morning peak period (6:00–9:00 am) to which the VKT and other traffic variable estimates from EMME/2 pertain. The final model parameters and corresponding χ2 and goodness-of-fit criteria are shown in Tables 2 and 3. This Table is taken from [Hadayeghi et al. \(2003\)](#page-9-0) which discusses the modelling approach and related issues. The models developed in this study for both 1996 and 2001 include major arterial and minor road data in each zone, but exclude freeway data since the presence of a freeway in a zone was found after exploratory analysis to be generally unrelated to characteristics of the zone. # **5. Measures of model transferability** The transferability of a model relates to the application of a model estimated in one context based on adequate data to another context with limited data. Model transferability is based on the idea that the estimated model from a previous study in a different context may provide useful information for estimating parameters appropriate for the application context. "Local" or "estimation" context refers to the context for which the adequate dataset was used in the model estimation. The "transfer" or "application" context refers to the context for which predictions are required, and to which the model from the local context is transferred for application. Transportation planning models are not perfectly transferable between contexts since the models do not capture the behavior of individuals completely and the data available for the estimation of the unknown parameters are not error free due to a variety of sources such as sampling errors ([Lerman, 1981\).](#page-9-0) As [Lerman \(1981\)](#page-9-0) and [Ben-Akiva \(1981\)](#page-9-0) discuss, it is appropriate to describe the measure of transfer effectiveness on a continuous Table 3 Results of negative binomial regression for models III and IV | Parameter | Yearly total accident model for
morning period (Model III) | | Yearly severe accident model for
morning period (Model IV) | | | |---------------------------------|---------------------------------------------------------------|-------------------------|---------------------------------------------------------------|------------------------|--| | | 1996 Model | 2001 Model | 1996 Model | 2001 Model | | | Intercept (log of α0) | 1.7851 (0.2909, 37.66) | 0.4328 (0.3127, 1.92) | 0.2079 (0.0403, 2.7) | 1.0527 (0.4562, 5.32) | | | Log of VKT | 0.2225 (0.0430, 26.73) | 0.3268 (0.0574, 32.47) | 0.2171 (0.0606, 12.85) | 0.3163 (0.0803, 15.53) | | | Total major road in kilometer | 0.1171 (0.0201, 34.07) | 0.2127 (0.0202, 111.39) | 0.1316 (0.0264, 24.8) | 0.2528 (0.0267, 89.51) | | | Number of households (× 10−3) | – | – | 0.0892 (0.0220, 16.43) | 0.0701 (0.0216, 10.51) | | | Number of employements (× 10−3) | 0.0692 (0.0162, 18.18) | 0.0533 (0.0155, 11.80) | – | – | | | Speed | −0.0324 (0.0073, 19.59) | −0.0139 (0.0063, 4.91) | −0.0213 (0.0102, 4.34) | −0.0141 (0.0087, 2.61) | | | Volume/capacity (V/C) | −1.1156 (0.3296, 11.46) | −0.7236 (0.3234, 5.01) | −0.9074 (0.4500, 4.07) | −0.8137 (0.4412, 3.40) | | | Intersection density | – | – | – | – | | | Total minor road in kilometer | 0.0108 (0.0051, 4.52) | 0.0006 (0.0049, 0.02) | – | – | | | Goodness of fit measures | | | | | | | Mean Pearson χ2/d.f. | 1.0805 | 1.0802 | 1.0791 | 0.9659 | | | Pearson R2 | 36.56% | 48.46% | 31.57% | 42.17% | | | α | 0.1586 | 0.1959 | 0.1961 | 0.2290 | | *Note*: Refer to Eq. [\(1\). T](#page-2-0)he dependent variable is the natural logarithm of yearly total accidents. Parameter estimate (S.E., χ2). scale rather to describe transferability as a dichotomous property. Transferability is a property that may exist to a greater or lesser degree depending on the formation of the model and the characteristics of the estimation and application contexts [\(Lerman,](#page-9-0) [1981; Ben-Akiva, 1981\).](#page-9-0) Measures of model transferability were used to assess the effectiveness of models when they are transferred for use in an application context (2001 in this study) that is different from their calibration context (1996). The indices and test-statistics are all based on log-likelihood measure. These measures, defined below, are largely similar to those used by [Koppelman and Rose](#page-9-0) [\(1983\)](#page-9-0) and [Badoe \(1994\).](#page-9-0) #### *5.1. Nested likelihood ratio test (LR)* This measure indicates the statistical similarity of the estimation and application model coefficients. In this test, the null hypothesis is that the application and estimation context can be presented by a single model. Basically this test examines whether the model parameters have remained stable over time and is given by $$LR = -2[LL_{i \cup j}(\hat{\theta}_{i \cup j}) - LL_{i}(\hat{\theta}_{i}) - LL_{j}(\hat{\theta}_{j})]$$ (2) The above equation is χ2 distributed with degrees of freedom equal to the number of model parameters. In the event of rejection of the null hypothesis, an asymptotic *t*-test can be performed to test which parameter(s) were responsible for the rejection. For this, the *t*-statistic is given as $$t = (\hat{\theta}_{ki} - \hat{\theta}_{kj}) / \sqrt{(\text{var}(\hat{\theta}_{ki}) + \text{var}(\hat{\theta}_{kj}))}$$ (3) where ˆ θik is the estimate of parameter *k* in context *I* and var(ˆ θik) is the variance of the estimate of parameter *k* in context *i*. #### *5.2. Transfer index (TI)* Transfer index (TI) is a relative measure that indicates how well a transferred model performs in predicting the application dataset relative to a locally estimated model in the application context and of similar specification. For this particular research, this index compares performance of the transferred and application context models relative to the performance of a 2001 "constant" model that has no independent variables. The index has an upper bound, 1.0, which is attained when the transferred model performs as well as an estimated model on the application data. It has no lower bound. Negative values imply that the transferred model is worse than the local constant model. It is expressed as $$TI = [LL_j(\hat{\theta}_i) - LL_j(\hat{c})]/[LL_j(\hat{\theta}_j) - LL_j(\hat{c})]$$ (4) where LLj(ˆ θi) is the log-likelihood value in context *j* using model parameters estimated from year *i*, LLj(ˆ θj) the loglikelihood value given by application context model *j* and LLj(ˆc) is the log-likelihood value given by constant model estimated in application context *j*. ## **6. Updating 1996 models** This section provides an approach for updating local models into an application context in a situation when only limited data are available in the application context for calibrating a meaningful model. The updating technique is helpful in applying an "old" estimated model to an application context to predict present conditions with minimum data requirements, consequently cutting the cost of data collection and time. Also, it seems desirable, when transferring a model, to update it by making adjustments based on locally available information. To illustrate such a situation, and to investigate the feasibility of this approach, random subsets of zonal information were selected from the 2001 database. To explore the impact of sample size on updating performance, 10 samples of each of 10, 20, 25, and 35 zones and their corresponding information were randomly selected from the 2001 dataset. However, before using updating procedures, a set of models were estimated using different sample sizes. These models were used to examine whether the coefficients in the estimation and application contexts are similar when limited data are available. #### *6.1. Updating using a calibration factor* The updating procedure was recommended by [Harwood et](#page-9-0) [al., 2000](#page-9-0) and a recent paper ([Oh et al., 2003\),](#page-9-0) for application in the Interactive Highway Safety Design Model (IHSDM) for transferring accident prediction models calibrated for one geographic region to apply to another region. In this procedure, [Harwood et al. \(2000\)](#page-9-0) in essence obtained a calibration factor as the total number of accidents observed in a sample from one jurisdiction divided by the sum of the predicted number of accidents in the sample using the model from another jurisdiction. The calibration factor was calculated as follows: calibration factor = $$\sum_{i=1}^{n} Y_i / \sum_{i=1}^{n} \hat{Y}_i$$ (5) where *Yi* is the number of observed accidents in zone *i* in 2001 for a given sample size and Yˆi is the predicted number of accidents in zone *i* by applying values of the 2001 independent variables (for the same sample size of 2001) to the 1996 model. Then, the calibration factors estimated for each sample size were simply multiplied by the base models (1996 models) in order to "transfer" the 1996 models for application to 2001 conditions. ### *6.2. Bayesian updating* Bayesian updating, which was introduced by [Atherton and](#page-9-0) [Ben-Akiva \(1976\),](#page-9-0) is another approach that is adopted for updating the 1996 models using the 2001 sample data. This methodology combines sample information with prior information in order to achieve more accurate updated information. [Atherton and Ben-Akiva \(1976\)](#page-9-0) assumed both prior and posterior (updated) distributions of the parameters to be normally distributed. The updated parameters are expressed as $$\beta_{\text{updated}} = [(\beta_{96}/\sigma_{96}^2) + (\beta_{01\text{-sample}}/\sigma_{01\text{-sample}}^2)]/[(1/\sigma_{96}^2) + (1/\sigma_{01\text{-sample}}^2)]$$ (6) $$\sigma_{\text{updated}} = 1/[(1/\sigma_{96}^2) + (1/\sigma_{01\text{-sample}}^2)]^{0.5}$$ (7) where βupdated is the 2001 updated coefficient, β96 the coefficient estimated from 1996 model, β01-sample the coefficient estimated from 2001 sample model, σupdated the standard deviation of 2001 updated coefficient, σ96 the standard deviation of coefficient estimated from 1996 model and σ01-sample is the standard deviation of coefficient estimated from 2001 sample model. #### *6.3. Measures of goodness of fit* Measures of goodness of fit are used to assess the performance of the updated models relative to the 1996 models. These measures are similar to those used by [Oh et al. \(2003\).](#page-9-0) #### *6.3.1. Mean prediction bias (MPB)* This measure describes the magnitude and direction of the average bias in the subject model. The smaller the absolute value of average prediction bias is, the better the model predicts the observed data. Positive values of MPB indicate that the model over-predicts the observed accident data, and vice versa: MPB = $$\sum_{i=1}^{n} (\hat{Y}_i - Y_i)/n$$ (8) where Yˆi is the predicted number of accidents at zone *i*, *Yi* the observed number of accidents at zone *i* and *n* is the data sample size. #### *6.3.2. Mean absolute deviation (MAD)* This measure provides a measure of the average misprediction of the model. A value close to zero suggests that the model on average predicts the observed data well. It is defined as $$MAD = \sum_{i=1}^{n} |(\hat{Y}_i - Y_i)|/n$$ (9) # *6.3.3. Mean squared prediction error and mean squared error* The mean squared prediction error (MSPE) is the sum of squared differences between observed and predicted accidents divided by the sample size. It is defined as MSPE = $$\sum_{i=1}^{n} (\hat{Y}_i - Y_i)^2 / n$$ (10) The mean squared error (MSE) is the sum of squared differences between observed and predicted accidents divided by the sample size minus the number of model parameters. It is defined as $$MSE = \sum_{i=1}^{n} (\hat{Y}_i - Y_i)^2 / (n - p)$$ (11) where *p* is the number of model parameters. A comparison of MSPE and MSE shows potential over or under fitting of the models to the estimation data. A MSPE that is higher than MSE may indicate that models may have been over fit to the estimation data and that some of the observed relationships may have not been correctly shown in the model. #### **7. Results and discussion** #### *7.1. Transferability results* The dependent variable of each developed model is the number of zonal accidents per year. The regression parameters were estimated based on the maximum likelihood method for negative binomial (NB) regression and using PROCGENMOD in SAS statistical software ([SAS Institute Inc., 1999\).](#page-10-0) # *7.1.1. Yearly model for total accidents* [Table 2](#page-4-0) presents the parameter estimates and corresponding χ2's for the 1996 model and the 2001 model for total yearly accidents (Model I). A comparison of the parameter estimates in [Table 2](#page-4-0) indicates that the speed becomes insignificant at the 5% level in the 2001 model. The parameter estimates of each variable have identical signs in both models. The dispersion parameters (α) are in the range of 0.19 and 0.23 for the 1996 model and the 2001 model, respectively, which indicates that the data are quite overdispersed relative to the Poisson distribution and that the negative binomial error structure is therefore justified. The goodness of fit measures for each model, also presented in [Table 2,](#page-4-0) show the mean Pearson χ2 ratios to be close to 1, an indication that the NB regression model fits the data very well. The Pearson *R*2 values for the 1996 model and the 2001 model are 37.48% and 37.24%, respectively, which indicate that the developed models explain the variation in the dependent variable to a reasonable degree. In addition, a test was done to examine whether keeping the dispersion parameter in the 2001 model the same as the one in the 1996 model would affect the 2001 model performance considerably. It was found that the model parameters and corresponding parameters' signs were almost the same as those of Model I. The total predicted number of accidents from this model for the whole study area is different by less than 1% from the total from Model I, which is quite insignificant in comparison. Therefore, no further investigation was conducted. Results of tests of parameter equality, which were done by the nested likelihood ratio test, are shown in [Table 4.](#page-7-0) These indicate that 1996 and 2001 model parameters are not statistically equal. Therefore, the null hypothesis can be rejected. The errors in the modelling calibration procedures are most likely to affect transferability of the explanatory variables and the estimated parameters. To investigate whether the rejection of the null hypothesis is because of an individual coefficient or sub-set of coefficients, the asymptotic *t*-test is used to test equality of 1996 and 2001 variable coefficient estimates. Part (b) of [Table 4](#page-7-0) presents the results of this test for the yearly model for total accidents. As it can be seen, only two explanatory variables of VKT and number of households cannot be rejected at the 5% significance level under the null hypothesis of equality in the two contexts. Theoretically, it can be concluded that the Table 4 Results from transferring 1996 Models to 2001 dataset | Transfer measures | Model I | Model II | Model III | Model IV | | |-----------------------------------------------------------|---------|----------|-----------|----------|--| | (a) Absolute and relative transfer measures | | | | | | | Nested likelihood ratio | 128.6 | 52.4 | 177.4 | 30.84 | | | a (× 105)
LL01(θ96) | 3.087 | 0.460 | 0.179 | 0.0116 | | | Transfer index | −0.966 | 0.528 | −0.881 | 0.640 | | | Parameter | Model I | Model II | Model III | Model IV | | | (b) Statistical test of parameter equality in two context | | | | | | | Log of VKT | −0.29 | 0.99 | −1.45 | −0.99 | | | Total major road in kilometer | −2.71 | −4.13 | −3.35 | −3.23 | | | Number of households (× 10−3) | 1.30 | 0.90 | – | 0.62 | | | Number of employements (× 10−3) | – | – | 0.71 | – | | | Speed | −3.19 | −2.10 | −0.06 | −0.54 | | | Volume/capacity (V/C) | −1.73 | −1.88 | −1.21 | −0.15 | | | Intersection density | 1.65 | 1.41 | – | – | | | Total minor road in kilometer | – | – | 1.44 | – | | a LL01(θ96) is the 2001 context log-likelihood using θ96 from year 1996. roadway parameters such as major road kilometers and posted speed did not remain stable over time. The value of transfer index, which describes the degree to which the log likelihood of the transferred model exceeds the reference model relative to improvement provided by a model developed in the application context, is found to be −0.966. The negative sign implies that the transferred model is worse than the local reference model. ## *7.1.2. Yearly model for severe accidents* The model parameters and corresponding χ2 values for the yearly model of severe accidents (Model II) are shown in[Table 2.](#page-4-0) All the coefficients are close in magnitude and share the same sign in the both the 1996 and 2001 models. The intersection density and *V*/*C* variables are insignificant at the 5% level in the 2001 model but are significant in the 1996 model. The dispersion parameters (α) of 0.2092, and 0.2226 for the 1996 model and the 2001 model, respectively, again confirm the appropriateness of using negative binomial as opposed to Poisson regression. The mean Pearson χ2 ratios for the two models are within the acceptable range of 0.8–1.2, another indication that the choice of the negative binomial error structure appears to be appropriate. The Pearson *R*2 values for the 1996 model and the 2001 model are 31.90% and 43.3%, respectively, which indicate that the developed models explain the variation in the dependent variable to a reasonable degree. Results of the test of parameter equality in Table 4 indicate that the 1996 and 2001 model parameters are not statistically equal. *V*/*C*, speed, and major road kilometers are the parameters most responsible for the statistical inequality, as shown in part (b) of Table 4. The value of transfer index of 0.528, suggests that the transferred model provides some portion of the information obtained from the 2001 model, irrespective of model specification. #### *7.1.3. Morning peak period accident models* For the morning peak period models, accidents between 6:00 and 9:00 am were extracted from the dataset by running a query in Microsoft Access. [Table 3](#page-4-0) shows the parameter estimates for the total accident models (Model III) and severe accident models (Model IV) for the morning peak period. For both Models III and IV, the signs of all variables and constant terms are the same for the 1996 and 2001 models. The coefficients in the 2001 models are largely different from those of the 1996 models, in particular, those with low levels of significance such as total minor road kilometers in the 2001 model. For both Models III and IV, the coefficient of the total major road kilometers variable for the 2001 models is three times as large as that for the 1996 models. The goodness of fit statistics show mean Pearson χ2 values close to one, again indicating that the models fit the data reasonably well. The dispersion parameters (α) for all the models are significantly different from zero, again confirming the appropriateness of negative binomial regression. The Pearson *R*2 values are reasonable for all the models. The statistical tests of equality of model parameters for the Models III and IV reject the null hypothesis of equality, indicating that some or all model parameters have not remained stable over time. Both Models III and IV exhibit statistically significant equality at the 5% significance level in all the parameters, except major road kilometers. The values of Transfer index are found to be −0.881 and 0.640 for Models III and IV, respectively, suggest that the transferred model provides some portion of the information obtained from the 2001 model, irrespective of model specification. However, the negative value, as explained previously, implies that the transferred model is worse than the local reference model. # *7.2. Updating results* As expected, none of the models calibrated with limited data performed reasonably in terms of the magnitude of the coefficient parameters or their level of significance in the application context regardless of the sample size. The magnitude of coefficient parameters for some of the models were the same as the 1996 models but either the parameters were insignificant at the 5% confidence level or the goodness of fit measures were not Table 5Goodness of fit measures for predicting 2001 accidents from 1996, 2001 and updated models | del
Mo
typ
es | GO
F | 20
01
del
Mo | 6
199
del
Mo | dat
ing
wi
th
le o
f
Up
sam
p
10
in
20
01
zon
es | | dat
ing
wi
th
le o
f
Up
sam
p
20
in
20
01
zon
es | | dat
ing
wi
th
le o
f
Up
sam
p
25
in
20
01
zon
es | | dat
ing
wi
th
le o
f
Up
sam
p
35
in
20
01
zon
es | | |------------------------|---------------------|-----------------------|-----------------------|--------------------------------------------------------------------------------------------|-------------------------------------------|--------------------------------------------------------------------------------------------|-------------------------------------------|--------------------------------------------------------------------------------------------|-------------------------------------------------|--------------------------------------------------------------------------------------------|-------------------------------------------| | | | | | Re
cal
ibr
d
ate
199
6 m
ode
l | Ba
ian
dat
yes
up
e | Re
cal
ibr
d
ate
199
6 m
ode
l | Ba
ian
dat
yes
up
e | Re
cal
ibr
d
ate
199
6 m
ode
l | Ba
ian
dat
yes
up
e | Re
cal
ibr
d
ate
199
6 m
ode
l | Ba
ian
dat
yes
up
e | | Mo
del
I | MP
B | 1.1
7 | −3
7.8
0 | −1
0.5
7 (
27
.39
) | −3
0.0
5 (
39
.78
) | −6
.87
(
15.
49
) | −2
8.0
4 (
18.
50
) | 1.3
1 (
17.
59
) | −3
4.1
5 (
26
.86
) | −2
.42
(
21
.94
) | −4
0.6
4 (
23
.72
) | | | MA
D
MS
E | 59
.20
67
80 | 65
.17 | 71
.62
(
4.8
6) | 61
.95
(
16.
97
) | 69
.73
(
2.6
1) | 58
.85
(
16.
14) | 71
.51
(
5.3
8) | 65
.19
(
18.
26
) | 71
.33
(
5.6
1) | 65
.77
(
16.
39) | | | MS
PE | | 92
94 | 95
169
04
(
0) | 58
89
93
(
23
) | 56
89
(
294
) | 96
85
350
6)
( | 66
68
90
(
8) | 46
83
92
(
00
) | 05
67
92
(
3) | 62
63)
87
(
44 | | del
Mo
II | MP
B | 2.3
1 | −0
.51 | −0
.66
(
8.5
9) | 7.4
9 (
20
.33
) | 0.5
4 (
3.0
5) | −2
.69
(
5.4
1) | 0.6
2 (
2.1
2) | 0.5
3 (
1.8
0) | −1
.17
(
4.7
1) | 3.2
1 (
6.0
0) | | | MA
D
MS
E | 14
.89
44
9 | 15
.36 | 17
.68
(
2.3
8) | 21
.45
(
13.
33) | 16
.97
(
0.5
6) | 15
.69
(
2.1
4) | 16
.78
(
0.4
6) | 13
.99
(
1.8
7) | 16
.85
(
0.7
3) | 16
.58
(
2.5
8) | | | MS
PE | | 41
9 | 54
5 (
123
) | 80
8 (
90
2) | 48
2 (
12) | 44
5 (
127
) | 47
6 (
4) | 51
3 (
90
) | 49
4 (
25
) | 44
1 (
180
) | | Mo
del
III | MP
B
MA
D | 2.2
8
8.1
1 | −6
.10
8.5
1 | −1
.42
(
3.5
6)
8.4
0.7 | −6
.09
(
3.5
2)
7.4
3.2 | −0
.53
(
1.7
7)
8.2
0.3 | −6
.54
(
1.7
8)
8.7
1.6 | 1.1
6 (
1.9
0)
8.6
0.6 | −8
.24
(
2.3
8)
10
.16
1.5 | 0.2
6 (
2.5
1)
8.4
0.7 | −5
.87
(
3.1
3)
8.2
2.1 | | | MS
E | 13
2 | | 7 (
7) | 9 (
1) | 4 (
6) | 4 (
3) | 2 (
3) | (
7) | 8 (
0) | 1 (
0) | | | MS
PE | | 16
8 | 15
2 (
12) | 13
8 (
155
) | 14
0 (
4) | 14
3 (
50
) | 14
2 (
9) | 27
8 (
149
) | 14
3 (
11) | 17
8 (
149
) | | Mo
del
IV | MP
B | 0.3
9 | −0
.52 | 0.2
3 (
1.1
1) | −0
.76
(
1.0
8) | −0
.04
(
0.8
5) | −0
.28
(
0.6
9) | 0.1
0 (
0.6
6) | −0
.48
(
1.0
6) | −0
.25
(
0.5
8) | −0
.29
(
0.7
9) | | | MA
D
MS
PE | 2.3
9
10
.15 | 2.3
4 | 2.6
5 (
0.1
6) | 2.6
6 (
0.6
2) | 2.5
4 (
0.1
8) | 2.3
2 (
0.7
5) | 2.5
3 (
0.1
3) | 2.5
0 (
0.3
8) | 2.4
7 (
0.1
0) | 2.2
3 (
0.3
9) | | | MS
E | | 10
.90 | 12
.58
(
1.0
9) | 12
.89
(
7.0
6) | 12
.02
(
0.6
5) | 11
.16
(
7.3
5) | 11
.72
(
0.4
2) | 10
.46
(
3.9
8) | 11
.73
(
0.4
6) | 9.2
2 (
3.7
9) | *Note*: MPB = mean prediction bias, MAD = mean absolute deviation, MSPE = mean squared prediction error, MSE = mean squared error (S.D.). satisfactory. This was expected since the sample sizes selected for this study were too small (less than 8% of the total data) to result in a meaningful conclusion. Therefore, it was concluded that the updating procedure is the best approach when limited data are available and no further investigation was conducted regarding developing models with limited data. [Table 5](#page-8-0) presents the relative goodness of fit measures for predicting the number of accidents in 2001 using the 1996 models, the 2001 models, models that apply a calibration factor to the 1996 model and the Bayesian approach. For the latter two, various sample sizes are used to estimate the "updated" models that are applied. As expected, the measures indicate that 2001 model fits the 2001 data better than other models. The real value of these results is in the comparison of the performance of the "updated" models among themselves for different sample sizes and with the 1996 model. This comparison shows that in general the recalibrated "updated" models perform better than the updated Bayesian models and the1996 models. However, no clear-cut conclusions can be drawn regarding the performance of updated models in respect to the sample size used for estimating them. #### **8. Conclusions and summary** This paper examined the temporal transferability of zonal accident prediction models for the City of Toronto. In so doing, model calibration and related transferability issues were also addressed. The coefficient estimates for the 2001 models are consistent with the 1996 models. Most of the coefficient parameters and constant terms are significant for the 1996 and the 2001 models across all four accident type models (Models I–IV). The results of nested likelihood ratio test indicate that the model parameters from two urban contexts are not statistically equal and have not remained stable over time. Therefore, from a theoretical perspective, the transfer of macro-level accident prediction models is undesirable. However, relative measures of transferability indicate that the transferred models yield useful information in the application context. Transfer index values show that transferred models capture at least 52% of the variance provided by estimating all the parameters in the 2001 data. The results of the *t*-test for all the models indicate that the VKT and socioeconomic and demographic parameters are stable over time. In the absence of a complete dataset for calibrating the 2001 model, it looks like the "updated" models, which are developed by applying calibration factors to the 1996 models, perform reasonably and better than the 1996 models and the Bayesian updated models regardless of the sample size used for estimating these factors. The Bayesian approach has not yielded good results in general. The possible problem for this method is that in this method the coefficients in model transferring are heavily affected by the estimation context data since the number of observations is larger in this context. Also, this method assumes that the prior and posterior (updated) distributions of the parameters are normally distributed, which may not be a good assumption. This research looked only at temporal transferability of zonal accident prediction models. A natural next step for further research would be the investigation of spatial transferability and the combination of temporal and spatial transferability. That research should aim to identify the conditions that make two study areas "similar enough" to justify transferring a model from one to the other. #### **Acknowledgment** The authors acknowledge the Natural Sciences and Engineering Research Council of Canada (NSERC) for its support of some of the fundamental research on which this work is based. # **References** - Atherton, T.J., Ben-Akiva, M.E., 1976. Transferability and updating of disaggregation travel demand models. Transport. Res. Rec. 610, 12–18. - Badoe, D.A., 1994. An investigation into the long range transferability of work-trip discrete mode choice models. PhD Dissertation. Department of Civil Engineering, University of Toronto, Ont., Canada. - Badoe, D.A., Miller, E.J., 1995a. Analysis of the temporal transferability of disaggregate work trip mode choice models. Transport. Res. Rec. 1493, 1–11. - Badoe, D.A., Miller, E.J., 1995b. Comparison of alternative methods for updating disaggregated logit mode choice models. Transport. Res. Rec. 1493, 90–100. - Ben-Akiva, M., 1981. Issues in transferring and updating travel-behaviour models. In: Stopher, P.R., et al. (Eds.), New Horizons in Travel Behaviour. Lexington Books, Lexington, MA, pp. 665–686. - Ben-Akiva, M., Bolduc, D., 1987. Approaches to model transferability and updating: the combined transfer estimator. Transport. Res. Rec. 1139, 1–7. - EMME/2, 1998. User's Guide Manual Release 9. INRO Consultants Inc. - Gunn, H.F., 2001. Spatial and temporal transferability of relationships between travel demand, trip cost and travel time. Transport. Res. Part E: Logist. Transport. Rev. 37 (2/3), 163–189. - Gunn, H.F., Ben-Akiva, M.E., Bradley, M.A., 1985. Tests of the scaling approach to transferring disaggregate travel demand models. Transport. Res. Rec. 1037, 21–30. - Hadayeghi, A., Shalaby, A.S., Persaud, B., 2003. Macro-level accident prediction models for evaluating safety of urban transportation systems. Transport. Res. Rec. 1840, 87–95. - Harwood, D.W., Council, M., Hauer, E., Hughes, W. E., Vogt, A., 2000. Prediction of the expected safety performance of rural two-lane highways. Report FHWA-RD-99-207, FHWA, Department of Transportation. - Koppelman, F.S., Rose, G., 1983. Geographic transfer of travel choice models: evaluation and procedure. In: Optimization and Discrete Choice in Urban Systems: Proceedings of the International Symposium on New Directions in Urban Systems Modeling, pp. 272–309. - Koppelman, F.S., Wilmot, C.G., 1982. Transferability analysis of dissagregate choice models. Transport. Res. Rec. 895, 18–24. - Ladron de Guevara, F., Washington, S.P., Oh, J., 2004. Forecasting crashes at the planning level: simultaneous negative binomial crash model applied in Tucson, Arizona. Transport. Res. Rec. 1897, 191–199. - Lerman, S.R., 1981. A comment on interspatial, intraspatial, and temporal transferability. In: Stopher, P.R., et al. (Eds.), New Horizons in Travel Behaviour. Lexington Books, Lexington, MA, pp. 628–632. - Lovegrove, G.R., Sayed, T., 2005. Macro-level collision prediction models for evaluating neighbourhood traffic safety. In: Proceedings of the Sixth Transportation Specialty Conference, Canadian Society of Civil Engineering, Toronto, Canada. - Oh, J., Lyon, C., Washington, S., Persaud, B., Bared, J., 2003. Validation of FHWA crash models for rural intersection lessons learned. Transport. Res. Rec. 1840, 41–49. - Parody, T.E., 1977. An analysis of disaggregate mode choice models in prediction. Transport. Res. Rec. 649, 23–31. - National Cooperative Highway Research Program (NCHRP), in press. Incorporating Safety into Long-range Transportation Planning, Project 8-44. National Cooperative Highway Research Program. - Persaud, B., Lord, D., Palmisano, J., 2002. Calibration and transferability of accident prediction models for urban intersections. Transport. Res. Rec. 1784, 57–64. - SAS Institute Inc., 1999. SAS/STAT® Users Guide, Version 8. SAS Institute Inc., Cary, NC.