Evaluation of the crash and casualty effects of average speed enforcement, carried out at 14 sites in Norway, found a 12-22% reduction of the number of injury crashes – and a “statistically significant” KSI reduction of between 49-54% (ITS International).
The study was carried out by the Institute of Transport Economics (TOI) at the Norwegian Center for Transport Research and funded by the Norwegian Public Roads Administration.
The study takes account of trend, volumes, speed limit changes at some of the sites, speed cameras at some of the sites in the before period, and regression to the mean (RTM). Regression to the mean is controlled for by using the empirical Bayes method which takes into account that exceptionally high crash numbers in the before period usually are associated with a reduction of the number of crashes in the after period, even without any effective safety measure. Most of the sites have an 80km/h speed limit.
Eight of the sites are in tunnels and the results indicate that the crash reduction in tunnels is at least of the same magnitude as on open roads.
The effects of average speed enforcement on stretches of road three kilometres downstream of the section control sites were evaluated using the same methods as at the section control sites, and injury crashes were found to be reduced by 46%.
It seems to be widely accepted that collision data can be (although not always) prone to regression-to-mean effects, and there are simple diagnostic tests around (e.g. the turning point test) to advise on the likelihood of this being the case in a particular dataset (although these tests are very rarely applied in practice). I agree with Mike’s earlier post that EB analysis is not as complicated as some would make out, and can be applied in a simple way using standard packages such as Excel. There are also ‘black box’ solutions (for example for Full Bayes analysis) for those who want to avoid the statistics as much as possible.
My understanding of the FTP approach is that it assumes that ALL of the increase in collisions in the selection period is due to RTM – is it really as ‘all or nothing’ as this? Some of the increase may be RTM and some of it not – in one extreme none of it may be due to RTM. The advantage of a Bayesian approach is that it uses all the available information and is able to identify how much of the increase is due to RTM and how much to other effects (e.g. a site truly becoming less safe). Thus, is there the danger that the FTP method may actually, in some cases for example where RTM is not present at all, significantly under-estimate the effectiveness of a road safety measure by assuming a ‘worst case scenario’ with respect to RTM? A Bayesian analysis also allows other effects such as ‘trend’ to be estimated.
Neil Thorpe, Newcastle
0
I thank Dave Finney for his most recent comment, but I must still take issue with some of his remarks.
Firstly, I could direct him to any number of road safety professionals who understand the EBM. Indeed, any competent statistician would be amused to see the degree of confusion that is made in this debate of the relatively simple topic of Bayesian inference!
Dave is correct in saying that a predictive model is developed using data from a set of sites that are similar to those where the cameras are placed (eg rural single carriageways): this model consists of a simple equation relating the expected number of accidents to variables such as the time period, section length, traffic volume etc. This model is then applied to each camera site to give the “prior mean” for the expected number of accidents at such a site (ie carrying that level of traffic etc).
What the EBM then does is to combine this “prior mean” value with the observed number of accidents at the camera site in a weighted average with the weights reflecting the precision or accuracy of the predictive model and the observation respectively. This weighted average is then the “posterior mean” which is then the best estimate of the true mean accident rate at the camera site in the before period.
Dave is incorrect though in saying that this “introduces some of the RTM effect back into the result”. The method uses ALL the available evidence relating to the site.
The Bayesian approach can be thought of as like a doctor steadily adjusting his assessment of the likelihood of a patient having a given disease as he gathers evidence on the patient’s symptoms: age, temperature, blood pressure, presence / absence of spots or rashes, blood test results etc. He too uses all the available evidence.
The two methods (EBM and FTP) are alternative approaches, and use different forms of data. Both attempt to avoid or allow for RTM. They each have their advantages and disadvantages. As Dave knows well, the application of the FTP method is not entirely straightforward: its accuracy relies heavily on the assumed site selection period (SSP) used for each site. A wrong assumption for the SSP can affect the estimate of the camera effect significantly by introducing some of the RTM effect.
Mike Maher, North Berwick
0
While the EB method may not be that complex, I have yet to meet any RSP (road safety professional) who has claimed to understand it. As I understand it, a model (set of equations) is devised and adjusted using control sites (sites thought to be similar to speed camera sites). Data from speed camera sites is then fed into the model and the model produces estimates of the expected crash frequencies had speed cameras not been deployed. If the model estimates were then compared to the crash frequencies with speed cameras, this could give results that don’t contain any of the RTM that was due to the speed camera site-selection process (although it could include many other assumptions and biases).
The main problem with the EB method is that the model estimate is then altered (usually increased) by factoring in some of the crash data at sites before speed cameras were deployed. What this does is to introduce some of the RTM effect back into the result and this is what could make speed cameras that cause an increase in crashes appear to cause a reduction. The funny thing is, this flaw in the EB method is exactly the opposite of what the FTP method uses to produce such accurate results!
The EB and FTP methods are attempting to do the same thing (remove the RTM effect) but approach from different angles. The FTP method has several advantages over the EB method but the most important is accuracy. The FTP method is the only method that can completely remove the RTM by selection effect from the results and is therefore the most accurate available (short of scientific trials).
Dave Finney, Slough
0
Idris Francis is quite right that the two methods (Empirical Bayes and four time periods method) are quite different approaches, and use different forms of data. They are alternative approaches to achieving the same objective – the avoidance of the effect of regression to the mean and the estimation of the treatment effect. Both have their advantages and disadvantages.
But there is nothing “subjective” about the EBM and it is nothing like as complex as some like to make out. It should not be necessary to make false criticisms of the EBM in order to press the claims of the FTP method.
Mike Maher, North Berwick
0
My understanding of the Empirical Bayes Method as applied to road accidents is that it compiles databases of roads by accident numbers, road type, traffic volume, speed limits etc. as a reference for what accident rates would have been expected at cameras sites had cameras not been installed.
Dave Finney, in contrast, compares post-installation rates with normal levels at those same sites before the (usually higher than normal) levels which led to their selection.
My difficulty with EB (using which Prof. Maher and Prof. Mountain estimated (App.H of 4th Report) that 75% of the reduction attributed to cameras was in reality due to regression to mean) is not that it has built-in or even deliberate bias, but that the many subjective choices inherent in applying it, risk not only genuine error and unwitting bias but also suspicion (present company excepted of course) of deliberate bias, especially as so few commentators understand the method. For all these reasons, and as even the 4th Report stated when ignoring it) EB results seem less than reliable.
Meanwhile my own analysis, essentially Dave Finney’s but extended to half the country over 25 years, using only simple arithmetic, confirms that camera benefit is as close to nil as makes no difference.
Idris Francis Fight Back With Facts Petersfield
0
Dave Finney’s comments about the supposed inbuilt error in the Empirical Bayes method are quite wrong and result from a fundamental lack of understanding of the method.
Mike Maher, North Berwick
0
Even if we believed these figures, how about all the people that did not crash? Is personal choice and freedom now so de-valued, that it can be so easily overridden in the name of questionable safety statistics? We no longer seem to have the right to be treated as an individual, as more ‘one-size-fits-all’ legislation is imposed.
Terry Hudson, Kent
0
Eric is absolutely right, it is utterly inconceivable that % reductions of anything like that order could be achieved by modest reductions in speeding unless the numbers were so small that percentages become huge.
Idris Francis Fight Back With Facts Petersfield
0
In response to Eric’s question: this is a recent item amongst other foreign based stories that we have covered. Although we are a GB based and focussed organisation, we also keep a watch on the wider world as research, developments and experiences in other countries can help inform our work in the UK. As with any other profession, we learn from those in other countries and they learn from us. I would be deeply concerned if we did not look further than our own shores.
Honor Byford, Chair, Road Safety GB
0
I would have been interested to know the before and after speeds as well i.e the reductions achieved.
Hugh Jones, Cheshire
0
Notwithstanding previous comments on this “story” (and other items), on what basis is a report from Norway about Norwegian roads/drivers relevant to RSGB? There must be countless road safety reports from across the world every week yet it’s a “benefits of speed camera” item that becomes a top RSGB story. How come?
Eric Bridgstock, Independent Road Safety Research, St Albans
0
Wouldn’t it be fantastic if an intervention could produce a 50% reduction in KSI and, perhaps because we want to believe it, many do. Read the article carefully, the KSI reduction occurred, but it doesn’t claim that the speed cameras caused that reduction.
The ITS article claimed “Regression to the mean is controlled for by using the empirical Bayes method” but this is only an estimate (not a measurement). Furthermore, the empirical Bayes method has an error built in to it that can ensure a benefit is found. Even if the speed cameras are causing more KSI, the empirical Bayes method would still find a KSI reduction after their deployment. See the 3 paragraphs above “4.8 Achieving the full potential of the FTP method”:
http://speedcamerareport.co.uk/04_rtm.htm
Rather than make estimates and engage in endless debate over which is the most accurate (least in error), there is a straightforward solution. Just run speed cameras within simple scientific trials.
Dave Finney, Slough
0
It is inconceivable that ANY measure that does no more than encourages law-abiding drivers to travel at nominally no faster than a number on a sign could have such dramatic effects (around 50% reduction in KSI), espcially when the Highways Agency report in 2010 found that average speed cameras introduce or exacerbate distraction, bunching, sudden braking and sudden lane changing. An independent review of similar claims for the A14 ASC deployment exposed a number of “tricks”, leaving negligible road safety benefit from the cameras.
Eric Bridgstock, Independent Road Safety Research, St Albans
0
This is a very good illustration of why the KSI figure is quite useless at determining the effectiveness of any intervention. If it gave us the before and after collision ‘rate’ rather than the collision result then we would be much wiser. As it stands however the figure of a 50% reduction in KSI probably means that only one person was killed or injured compared to two in the previous period. That variation could well be down to the number of passengers in the vehicles when they collided not the actual collisions themselves.
Duncan MacKillop, Stratford on Avon
0