meulenerseffectivenessblackspotprograms2008.md 33 KB
category: literaturenote citekey: meulenerseffectivenessblackspotprograms2008 title: Effectiveness of the Black Spot Programs in Western Australia authors: "Meuleners, Lynn B.; Hendrie, Delia; Lee, Andy H.; Legge, Matthew" year: 2008 date: 2008-00-00 2008 doi: 10.1016/j.aap.2008.01.011 publication: "Accident Analysis & Prevention" url: "https://doi.org/10.1016/j.aap.2008.01.011" zotero_key: 32J8SVPN zotero_storage: JSTQC7KF collections: transpordi_planeerimine folder: 001_artiklid firstAuthor: "Meuleners, Lynn B."
status: converted
Accident Analysis and Prevention 40 (2008) 1211–1216
Effectiveness of the Black Spot Programs in Western Australia
Lynn B. Meuleners a, Delia Hendrie a, Andy H. Lee a,∗, Matthew Legge b
a School of Public Health, Curtin University of Technology, Australia b Western Australian Institute for Medical Research, The University of Western Australia, Australia Received 25 January 2007; received in revised form 5 April 2007; accepted 30 January 2008
Abstract
This study evaluates the effectiveness of the Black Spot Programs in Western Australia. Reduction in crash rate at the treated locations and the economic benefits of these treatments were assessed. The results showed that the programs have been effective overall, reducing all reported crash rate by 15%. The estimated crash cost savings were 50.8 million Australian dollars, of which 89% could be attributed to the reduction in casualty crashes. This led to net savings to the community of 40.4 million Australian dollars (\$35.1 million attributable to casualty crashes) after subtracting the capital costs of treating sites, maintenance and operating costs. The benefit cost ratio across all treatment sites was 4.9. Evaluation of the treatments has identified some effective treatment types and others without any significant change in either the rate or cost of crashes. The latter could be due to insufficient number of sites that received the treatment, the post-treatment period being relatively short, or the treatments genuinely had little impact on road safety. Findings of this study provide objective information for the development of effective strategies on road safety investment. © 2008 Elsevier Ltd. All rights reserved.
Keywords: Black spot treatment; Benefit cost ratio; Cost effectiveness; Economic evaluation; Road safety
1. Introduction
The term "black spot" (or "hot spot") is used to describe locations that have a higher than average crash rate and has been a standard safety engineering methodology for over 30 years. The identification, analysis and treatment of black spots are widely regarded as an effective approach to road crash prevention (Elvik, 1997; Levine, 2006).
The National Black Spot Program was first introduced in Western Australia (WA) in 1996/97 and extended until June 2006. The objective is to reduce the frequency and severity of crashes at locations experiencing a poor crash record, thereby reducing the significant trauma and suffering by crash victims, family and friends. The State Black Spot Program, which started in 2000, aimed at further improving road safety across WA. The program consolidates the partnership between the State and local governments to provide a road system that is safe for all road users. The State Black Spot Program complements the existing National Black Spot Program. It is intended, however, that
E-mail address: Andy.Lee@curtin.edu.au (A.H. Lee).
the State Program criteria will enable application of funds to a wider range of projects at hazardous situations by using different thresholds from the federal program; for example, using data for all crashes rather than just casualty crashes (Main Roads WA, 2003). Though the Black Spot Program is fairly new to Australia, similar safety engineering programs have been implemented in other countries for a long time. In the United States, the concept of 'hot spot analysis' is part of Federal law, called the Hazard Elimination Program, and has been in effect in every State since 1973.
This study aims to evaluate the effectiveness of the Black Spot Programs in WA. In view of the experience from other safety programs (Goldman et al., 2006), it is anticipated that the results will provide government and relevant agencies with reliable and objective information for future investments in road safety measures and strategies.
2. Methods
2.1. Program and study design
The State Black Spot Program provided funding of \$13 million for road safety related works on State and local government roads during 2000–2002. All road classifications were eligible
∗ Corresponding author at: School of Public Health, Curtin University of Technology, GPO Box U 1987, Perth, WA 6845, Australia. Tel.: +61 8 92664180; fax: +61 8 92662958.
for funding. The program targeted existing black spots (spots or sections of roads) and also potential hazardous locations. Black spots could be at an intersection, mid block spots, a short section of road, or long sections of road with a proven history of high crash density. Potentially hazardous locations were selected on the basis of formal road safety audits, although this practise may not identify the causal sequence in crashes at a location. Half of the funding was spent on roads within the Perth metropolitan region.
The study adopted a before-and-after comparison of all reported crashes, including fatal, hospitalisation, injury and property damage only (PDO) crashes, at treated sites for the years 2000–2002. Analysis was also undertaken on the economic worth of the State Black Spot Program.
2.2. Data collection
2.2.1. Western Australian Road Injury Database
The Road Safety Council of WA defines a crash as "any apparently unpremeditated collision reported to the police which resulted from the movement of at least one road vehicle on a road open to and used by the public and involving death or injury to any person, or property damages" (Legge et al., 2005). It is mandatory in WA to report a crash to the police if a person is injured or if property damage exceeds \$1000.
Police reported data was obtained from the WA Road Injury Database. The database contains detailed information on characteristics of people and vehicles involved in crashes, crash circumstances and police reported injury from January 1987 to the present. Crash data up to and including 2003 for Black Spot Program evaluation were retrieved on: crash date; crash severity; local government area of crash; and specific crash location.
2.2.2. Black spot site data
Main Roads WA provided information on each black spot treatment, including location and municipality, treatment description, and treatment start and finish dates (to within 1 week). Black spot sites, which had less than 6 months post treatment were excluded from the analysis. Moreover, some black spots had undergone a combination of treatments, reducing the number of available sites for separate analysis. Consequently, the "dominant" treatment type was chosen for clustering sites in order to improve the statistical power.
It is possible that high crash rates at certain sites might reduce in subsequent years even if no treatment was applied, primarily due to random variations or the common 'regression to the mean' effect. In order to smooth out random fluctuations as well as providing sufficient evidence of any temporal trend in crashes, at least 3 years, and preferably 5 years of data should be collected pre- and post- intervention (Nicholson, 1986). Consequently, crash data for the 5 years prior to treatment were retrieved at each location, but the post-treatment time span ranged from 6 months to 3 years, which limited the data available for analysis.
2.2.3. Costing data
Two types of costing data were used for economic evaluation, namely, the program costs and the cost savings from reduction in road crashes after implementation of the program. The costs of treating black spots included the initial capital outlay as well as operating and maintenance costs. For the latter, an average cost per treatment was provided as more site-specific data were unavailable.
The cost savings from fewer road crashes at treated sites were calculated based on the road crash severity costs for Australia (Bureau of Transport Economics, 2000), adjusting for price increases and state variations (Willett, 2004). These included the human costs of treating injuries plus any associated productivity losses and loss of functioning, vehicle repair and related costs, and general crash costs. However, road user costs such as vehicle operating costs and travel time were excluded because of the extensive details required (Bureau of Transport Economics, 2001). The unit of costing was expressed in 2003 Australian dollars according to crash severity, with \$2,226,100 for fatal, \$494,800 for hospital admission, \$29,900 for medical treatment, and \$11,900 for PDO crashes. The use of crash costs based on crash severity rather than type of crash has the disadvantage that a single serious crash at a site can potentially have a considerable impact on the calculation of the cost-effectiveness of a site. However, if the number of treatment sites being assessed is sufficient, this effect should cancel out. Recent Australian studies evaluating the cost-effectiveness of Black Spot Programs have used crash costs based on severity rather than crash type (Bureau of Transport Economics, 2001; Newstead and Corben, 2001).
2.3. Statistical analysis
Poisson regression based on generalized estimating equations (GEE) was undertaken to evaluate the effectiveness of the black spot treatments. The number of crashes per year is a discrete count variable and may be assumed to follow a Poisson distribution. However, the longitudinal nature of the observations render the application of standard Poisson regression inappropriate, and methods such as the GEE should be used to model the temporal trends while accommodating the inherent correlation of the panel data. Details on the GEE methodology can be found in Dupont (2002) and Twisk (2003).
Since the duration over which the crash data were collected varied between black spot sites, an appropriate weighting scheme should be incorporated to reflect the extent of individual exposure. While 5 years of crash data were collected before the treatment to account for regression to the mean, the after periods were substantially shorter and different between sites. Therefore, crash rates were analysed instead of crash frequencies.
The GEE approach provides valid statistical inferences by generating robust standard error estimates for the regression coefficients. It is thus capable of correctly estimating the overall treatment effect, broad category treatment and specific treatment effects. Similar treatment types were combined to attain a higher statistical power. For example, all treatments involved in the provision of a roundabout were grouped together regardless of the size of the roundabout installed. The Poisson GEE analyses of the longitudinal crash rates were performed using the STATA (version 9) statistical package.
2.4. Economic analysis
Benefit cost analysis was used for the economic assessment of the program. Benefit cost ratios (BCR) were calculated as the ratio of (i) the present value of the time stream of cost savings from a reduction in road crashes from the black spot treatments to (ii) the present value of the time stream of costs incurred to achieve these savings. A BCR much greater than 1 indicates a worthwhile project. The BCR can also be used as an index to rank comparable projects.
BCRs were calculated using the following sources of data: (i) the black spot site data relating to the capital costs of treating sites and the maintenance and operating costs; (ii) data on the effectiveness of treatments in reducing the number of road crashes and (iii) the unit road crash cost data. The capital costs of installing treatment were adjusted to 2003 Australian dollars based on the road and bridge construction price index (Australian Bureau of Statistics, 2004). The treatment life of projects ranged from 2–5 years for edge lines to 20 years for grade separation (Main Roads WA, 2003). On advice from traffic engineers at Main Roads WA, a treatment life of 10 years was adopted as a likely average across all treatment types. This assumption was further tested with benchmarks of 5 years and 15 years in the sensitivity analysis. Maintenance and operating costs were assumed to remain the same throughout the 10-year period. Likewise, savings from a reduction in road crash costs achieved since installing the treatments were assumed to be constant over the full 10-year period. Future costs and cost savings were discounted by 5% in the base case, and rates of 3% and 8% in the sensitivity analysis. BCRs were calculated for the whole Black Spot Program, components of the program, and specific treatment types. However, the sensitivity analysis was only conducted for the whole Black Spot Program.
3. Results
3.1. Crash rate evaluation
A total of 154 black spot sites were nominated for treatment. Figs. 1 and 2 show two of these sites where a seagull island
Fig. 1. A seagull island installed to separate turning vehicles at a black spot site.
Fig. 2. A pedestrian nib (kerb extension) installed to improve sight line for pedestrians and vehicles and reduce crossing distance for pedestrians at a black spot site.
and a nib were installed, respectively. Four sites were excluded from analysis because of missing data at these sites. The final sample of 150 treated sites consisted of 134 intersections and 16 road sections and non-intersection sites. The average number of months observed post-treatment was 23.7 with a range of 6–33.1 months.
Table 1 presents the results of the Poisson GEE analysis for all reported crashes at the 150 locations. A significant reduction of 15% (p < 0.001) in all crash rates was found. Similar reductions of 16% (p < 0.001) among the 114 metropolitan treatment sites and 14% (p < 0.001) among the 36 rural sites were also evident. With respect to specific treatment types, the respective reductions were: 29% for roundabouts (p < 0.001), 19% for traffic island on approach (p < 0.01), 50% for indented right island (p < 0.001), 59% for nibs (p < 0.001), and 52% for ban right turns (p < 0.001). However, the corresponding changes in crash rate were not significant for non-skid treatment, improve/reinforce priority signs, traffic control signals, left turn slip, seagull islands, and improved route lighting.
3.2. Economic evaluation
Table 2 presents the results of the economic evaluation of the Black Spot Programs for all reported crashes. The corresponding results in terms of casualty crashes are not shown for brevity. Assuming a treatment life of 10 years for all projects, the estimated crash cost savings were 50.8 million Australian dollars, of which 89% were attributable to a reduction in casualty crashes. Given the present value of treatment and operating/maintenance costs of \$10.4 million, this led to a BCR across all treatment sites of 4.9. A BCR of 4.9 implies a net saving of \$4.90 to the community for each dollar invested in the program. Sites treated in rural areas had a slightly better rate of return (BCR = 6.3) than their metropolitan counterparts (BCR = 4.3).
Treatment types varied in their estimated rates of return. Treatment types that were found to be good public investments in terms of their BCR were (in decreasing order of costeffectiveness) ban right turn treatment, indented right island,
Table 1 Black spot treatment effect on all crash rates in WA, 2000–2002, based on Poisson GEE analyses
| Estimate | Standard error | p-value | All crash reduction (%)a | |
|---|---|---|---|---|
| Whole program | −0.157 | 0.017 | 0.001 | 14.6 |
| Metropolitan sites | −0.170 | 0.021 | 0.001 | 15.7 |
| Rural sites | −0.146 | 0.031 | 0.001 | 13.6 |
| Broad categories | ||||
| Intersection treatments | −0.177 | 0.020 | 0.001 | 16.3 |
| Metro | −0.173 | 0.022 | 0.001 | 15.9 |
| Rural | −0.200 | 0.051 | 0.001 | 18.2 |
| Road section and non-intersection treatment | −0.104 | 0.038 | 0.006 | 9.9 |
| Metro | −0.105 | 0.114 | 0.356 | 10.0 |
| Rural | −0.117 | 0.036 | 0.001 | 11.1 |
| Treatment types | ||||
| All roundabouts | −0.344 | 0.042 | 0.001 | 29.2 |
| Metro | −0.239 | 0.087 | 0.006 | 21.3 |
| Rural | −0.920 | 0.274 | 0.001 | 60.2 |
| Traffic control signals | −0.239 | 0.156 | 0.127 | 21.2 |
| Non-skid treatment | −0.069 | 0.047 | 0.145 | 6.7 |
| Traffic island on approach | −0.207 | 0.085 | 0.015 | 18.7 |
| Seagull island | 0.056 | 0.048 | 0.243 | −5.7b |
| Left turn slip | −0.117 | 0.073 | 0.108 | 11.1 |
| Median on existing road | −0.085 | 0.108 | 0.430 | 8.2 |
| Nibs (pedestrian facilities) | −0.882 | 0.246 | 0.001 | 58.7 |
| Ban right turns | −0.732 | 0.208 | 0.001 | 52.0 |
| Indented right island | −0.684 | 0.176 | 0.001 | 49.6 |
| Improve/reinforce priority signs | 0.149 | 0.427 | 0.727 | 16.0 |
| All State roads | −0.035 | 0.451 | 0.432 | 3.5 |
a Includes all crashes (fatal, hospitalisation, injury and property damage only crashes).
roundabouts in rural areas, traffic island on approach and roundabouts in the metropolitan area. Although non-skid treatment, left turn slip and median on existing road have positive BCRs exceeding 1, their reductions in crash rate were not significant. All other treatment types were ineffective in terms of their economic benefits to the community.
Table 3 shows the effect of varying the assumptions relating to the discount rate and treatment life of projects on the esti-
Table 2 Economic evaluation of the Black Spot Program for all crash reductiona
| Present value of treatment and operating/maintenance costs (A\$)b |
Present value of crash cost savings (A\$)b |
Benefit cost ratio | |
|---|---|---|---|
| Whole program | 10,391,988 | 50,778,278 | 4.9 |
| Metropolitan sites | 7,825,760 | 33,830,852 | 4.3 |
| Rural Sites | 2,566,228 | 16,202,844 | 6.3 |
| Treatment Types | |||
| All roundabouts | 4,529,071 | 30,107,190 | 6.6 |
| Metro | 3,436,891 | 15,239,880 | 4.4 |
| Rural | 1,092,179 | 10,739,977 | 9.8 |
| Traffic control signals | 1,000,269 | −4779,484 | −4.8 |
| Non-skid treatment | 679,169 | 7,515,224 | 11.1 |
| Traffic island on approach | 657,191 | 4,061,624 | 6.2 |
| Seagull island | 628,974 | −896,126 | −1.4 |
| Left turn slip | 506,763 | 4,996,963 | 9.9 |
| Median on existing road | 353,458 | 498,977 | 1.4 |
| Nibs (pedestrian facilities) | 136,697 | −2,143,128 | −15.7 |
| Ban right turns | 28,670 | 5,685,393 | 198.3 |
| Indented right island | 183,194 | 2,786,441 | 15.2 |
| Improve/reinforce priority signs | 18,942 | −2,335,800 | −123.3 |
| All State roads | 381,239 | −5,374,343 | −14.1 |
a Calculations based on a treatment life of 10 years and a discount rate of 5%.
b Negative crash reductions indicates an increase.
b A\$ 1 = 0.61 D = US\$ 0.82 (April 2007).
Table 3 Sensitivity analysis of the economic evaluation of the Black Spot Program
| Present value of treatment and operating/maintenance costs (A\$)a |
Present value of crash cost savings (A\$)a |
Benefit cost ratio | |
|---|---|---|---|
| Base case | |||
| Discount rate, 5%; treatment life, 10 years | 10,391,988 | 50,778,278 | 4.9 |
| Discount rate | |||
| 3% (10 years) | 10,450,941 | 55,027,076 | 5.3 |
| 8% (10 years) | 10,317,176 | 45,386,487 | 4.4 |
| Treatment life | |||
| 5 years (5%) | 10,082,468 | 28,470,833 | 2.8 |
| 15 years (5%) | 10,634,509 | 68,256,973 | 6.4 |
a A\$ 1 = 0.61 D = US\$ 0.82 (April 2007).
mated rate of return. The Black Spot Program appeared to be cost-effective across all variations in assumptions, with lower discount rates and longer treatment lives of projects improving rates of return and vice versa. A discount rate of 3% would increase the BCR of the Black Spot Program to 5.3. Similarly, an expected treatment life of 15 years would increase the BCR to 6.4.
4. Discussion
The analysis confirmed that the Black Spot Programs implemented in WA were effective in reducing the overall crash rate and cost of the crashes at the treated sites. By assessing all severity types of crashes (including PDO), all possible benefits as well as potential detrimental effects of treatments were taken into consideration (Newstead and Corben, 2001).
The evaluation has identified certain treatment types such as roundabouts that were highly successful in crash reduction in both metropolitan and rural areas. Other effective treatments were traffic island on approach and indented right island. The lack of significant reductions in half of the treatment types may be due to several reasons. Firstly, the treatment may genuinely have little effect on road safety. Secondly, traffic flow may have changed at some of the treated sites. Information on traffic volumes over time would be useful to control for possible changes in traffic flow pre- and post-treatment. However, such data were not available for all treated sites so that the effect of traffic flow cannot be quantified. Thirdly, the post-treatment period was too short in some sites (the minimum being 6 months) for sufficient crash data to be collected. Ideally, 3–5 years of data are required for adequate comparison across treatment types (Nicholson, 1986). Finally, the small sample size of some treatments such as traffic control signals and improve/reinforce priority signs may be unable to produce statistically significant effects in the time available.
Treatment types also varied in their cost-effectiveness, with ban right turn and indented right-island projects showing the highest rates of return. Other treatment types with positive economic benefits were roundabouts and traffic islands on approach. Again, a limitation of the economic evaluation was the relatively short duration of follow-up after treatment. The average follow-up period was 24 months, much lower than the assumed treatment life of 10 years. Random fluctuations in crash experience at treated sites may lead to inaccurate estimates, which bias the long-term economic evaluation. Compared with the most recent national evaluation of the economic returns from black spot treatments, the BCRs of the Black Spot Program in WA are relatively low. An evaluation of the first 3 years of the Federal Road Safety Black Spot Program (1996/1997–1999/2000) provided a BCR of 18.6 across a sample of 608 projects, with corresponding BCRs of 19.5 and 13.2 for projects in capital cities and regional areas, respectively (Bureau of Transport Economics, 2001). The higher BCRs in the national evaluation are explained by the greater crash reductions found in the Federal program. However, an earlier evaluation of the Federal Government's Black Spot Program that operated from 1990/1991 to 1992/1993 found that the program delivered benefits of \$4 for each dollar of expenditure (Bureau of Transport Economics, 2001), a very similar result to that obtained in this study.
A major shortcoming of our evaluation was the lack of a control group of sites matched to the experimental group of black spot locations, thus limiting the conclusion made about the impact of the treatments on crash reductions over time (Hauer, 2002). Moreover, some of the black spot sites had undergone a combination of treatments, which substantially reduced the number of available sites for separate analysis. Therefore, the dominant treatment type was chosen for classifying sites. Although it might lead to a loss of accuracy concerning the apparent effectiveness of specific treatment types, the increase in sample size improved the power of the analysis.
Several recommendations are suggested based on the findings of this study:
- 1. Maintain accurate and timely recording of treatment details, including location, treatment type, costs, start and completion dates for future evaluations.
- 2. Local government agencies should supply more detailed information about the treatment at the nominated black spot.
- 3. Collect traffic volume at individual black spot sites to ascertain its effect on crash.
- 4. Investigation of simultaneous treatment effects for sites treated with more than one crash countermeasure.
- 5. Continue the monitoring of black spot sites and longer exposure period post-treatment is needed for accurate analysis.
- 6. Control sites should be included in any future evaluations.
In conclusion, the assessment of Black Spot Program reported here have important implications on resource allocation and strategies for future road safety investment. As traffic patterns and road usage change over time, new black spots will emerge. Government authorities often treat the worst sites as a priority due to resource and budgetary constraints, consequently neglecting the benefits derived from treating the remaining sites. Therefore, ongoing surveillance and evaluations of black spots are necessary in order to determine the most effective treatment and whether the prescribed treatment(s) are effective in terms of crash reduction and economic worth in the scenarios under consideration.
Acknowledgements
The authors would like to acknowledge the contributions of Brian Kidd, Daniel Sui, Maurice Cammack and Bob Peters from Main Roads WA. Particular thanks are due to Tony Radalj, Adrian Bird and Max Bulsara for their time, support and feedback on various aspects of this project. The authors are also indebted to two anonymous reviewers for their constructive comments and suggestions.
References
Australian Bureau of Statistics, 2004. Producer Price Indexes, September Quarter 2004, Catalogue no. 6427.0. Australian Bureau of Statistics, Canberra.
- Bureau of Transport Economics, 2000. Road Crash Costs in Australia, Report 102, Commonwealth of Australia, Canberra.
- Bureau of Transport Economics, 2001. The Black Spot Program 1996–2002: An Evaluation of the First Three Years, Report no. 104, Commonwealth of Australia, Canberra.
- Dupont, W., 2002. Statistical Modelling for Biomedical Researchers. Cambridge University Press, Cambridge.
- Elvik, R., 1997. Evaluations of road accident blackspot treatment: a case of the iron law of evaluation studies? Accid. Anal. Prev. 29, 191–199.
- Goldman, L.M., Herbel, S.B., Suhrbier, J.H., Davies, G., Papayannoullis, V., 2006. Safety-conscious planning in practice: development of regional safety planning and policy priorities. Transport. Res. Rec. 1969, 83– 91.
- Hauer, E., 2002. Observational Before–After Studies in Road Safety. Pergamon Press.
- Legge, M., Gavin, A., Cercarelli, R., 2005. Reported Road Crashes in Western Australia 2002, Road Safety Council of Western Australia, Perth.
- Levine, N., 2006. Houston, Texas, metropolitan traffic safety planning program. Transport. Res. Rec. 1969, 92–100.
- Main Roads Western Australia, 2003. State Black Spot Program Development and Management Guidelines, Document no. 2, March 2003.
- Newstead, S., Corben, B., 2001. Evaluation of the 1992–1996 Transport Accident Commission Funded Accident Blackspot Treatment Program in Victoria, Report no. 182, Accident Research Centre, Monash University, Victoria.
- Nicholson, A.J., 1986. The randomness of accident counts. Accid. Anal. Prev. 18, 193–198.
- Twisk, J., 2003. Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. Cambridge University Press, Cambridge.
- Willett, P., 2004. Network Average Crash Costs. Report commissioned by Main Roads Western Australia.