While joint road safety schemes implemented by the police and fire service are “well received”, they are threatened by challenges around proving their effectiveness, according to an officer from West Midlands Police (Police Oracle).
Sergeant Russell Webb made his comments about joint initiatives by West Midlands Police and West Midlands Fire Service, including a film called The Ripple Effect (above) which was produced to deliver safety messages to new drivers.
The two forces have also developed an intervention to combat speeding and anti-social use of vehicles in the South Yardley district of Birmingham.
Speaking to Police Oracle, Sergeant Webb said: “Giving drivers who previously had a good driving history the chance to avoid prosecution by attending an educational session at the fire station, run by officers from both services, has been well received.
"People who have previously attended other speed awareness courses said they didn’t take much from those – whereas they said this was fantastic.
“A key part of this approach was that we wanted it to be educational rather than punitive. We didn’t want to talk down to people or make them feel like they were getting told off – and they responded to that.”
As well as classroom based teaching the intervention included a practical demonstration of casualty extraction from a vehicle.
Sergeant Webb concluded: “What we have done so far has been looked at as best practice, but the biggest challenge is how to evaluate it. How do we prove it has a statistical impact?”
His concerns were echoed by Colin Heyes from Cheshire Fire and Rescue Service, who said: “We had a very similar initiative which ran for 10 years in Cheshire and which worked very well, but last year the police pulled away from it.
“It was a senior management decision within the police because of a lack of evaluating it – we couldn’t measure the impact it was having. It’s hard to measure the effect these things have, but that doesn’t mean it’s not having one.”
Surely the evaluations will be carried out against a published set of objectives? It would be a brave Road Safety practitioner who had immediate and sustained behaviour change in all participants as an objective, wouldn’t it? Do we not need, nationally, to agree what we hope to achieve, realisticaly, with these short interventions, and then evaluate against those?
Mandy Rigault. Oxfordshire.
0
There are so many aspects that you could cover in road safety education. In our experience real life stories have a good effect.
I’d also add to the warning about evaluation… No matter how good an evaluation is, we’ve got to remember that they very often can only really capture REPORTED behaviour or attitude, etc. Just because someone ticks a certain box doesn’t mean that actually believe it or actually do it.
Rhiannon, Lancashire
0
These before and after evaluations can be meaningless for more than the aforementioned reasons of lack of independent evaluation. Road safety experts and the Police (one and the same?) are data based in their evaluations, and while data is the only real link to their available results, they cannot possibly account for different drivers in different locations driving for different reasons either before or after. Drivers are not data based elements, they drive on emotion, and those emotions are in a state of constant flux dependent on their task in hand, the time they have to execute that task, as well as weather conditions and whether they have time to pop into the newsagents on the way etc. Where do all these elements appear in the databased evaluation set?
Derek Reynolds, Salop.
0
Nick:
Yes I have read that and again if you look at it and talk to Fiona the same applies. Very low numbers and self-reporting evidence of behavioural change. I am sure if you read the scope of the report it criticises previous evaluations for using self -reporting methods then goes on to use the same methodology due to cost restraints.
I would maintain the programmes probably do work but people keep citing poor evaluations for their efficacy. Perhaps why some police have made such comments.
There was also the one that ANDISP commissioned many years ago. As far as I can remember when I was at the meeting ANDISP rejected it because it did not agree with what ANDISP thought.
Keith
0
Thanks Keith:
It’s probably also worth looking at another study into the effectiveness of speed awareness courses, carried out by Brainbox Research (Dr Fiona Fylan) on behalf of ACPO – you can find the study via this link in the Road Safety Knowledge Centre:
http://www.roadsafetyknowledgecentre.org.uk/sections/enforcement/knowledge/1071.html
Nick Rawlings, editor, Road Safety News
0
Nick:
I am in favour of any initiative that helps contribute towards casualty reduction on a global level. However, the police are data driven as are road safety units. Self-reporting evaluation is a cheap alternative to robust prolonged evaluations of the effectiveness of a programme. Generally adopted as a method of evaluation due to funding restraints.
Your own report on the study: http://www.roadsafetygb.org.uk/news/2629.html suggests this.
“Professor Martin said: “The results clearly show that the speed awareness course led to reliable improvements in client’s attitude to speeding and importantly their intention not to break the speed limit. (only intentions – no long term post course study evidence)
“The benefit of the course occurred immediately and persisted several weeks after course delivery. The speed awareness course led to very reliable improvements in clients’ attitude towards not speeding.” (several weeks after again no long term post course evidence).
The US undertook a robust evaluation (using control group approach with very high numbers in excess of 50,000 (compared to this study of only 1,300) and established different results for the efficacy of such courses.
Speed Awareness Analysis Prof. Robin Martin Parts 1,2,and 3
Part 1 https://www.youtube.com/watch?v=R7Rls6l–T0
Part 2 https://www.youtube.com/watch?v=szG-A5dxznU
In clip 2 Professor Martin states that they were not allowed to undertake control group evaluation and therefore accepted the next best way of evaluation. Professor Martin also talks about long term measurement being essential, how can a total of 7 weeks 1 week before 6 weeks after be considered a long term study of the programme? He also mentions the inability to measure actual behaviour change due to constraints of the evaluation programme.
I don’t believe he mentions anything about the efficacy of modifying infrequent and frequent behaviour which is an important factor. Although the study may have been in depth there is no evidence of evaluation of behaviour change after the course only intention to change behaviour.
Is it really surprising that there is strong evidence not to speed after a month of attending the course? Where is the study relating to three years post test?
Keith
0
Keith:
Could you provide a link pls to the NDORS evaluation you refer to in your second para below which you say is ‘not robust’.
Nick Rawlings, editor, Road Safety News
0
Ian:
Could you provide the evidence for change in attitude that stands up to robust scrutiny?
As you must be aware the last evaluation of the NDORS Speed Awarensss Scheme commissioned I believe by TTC was not robust. The author of the report placed 3 clips on youtube, in one he states the effect last about 3 months with no real long term change in behaviour. He admits that without a control group the evaluation is not robust and that they were not allowed to use a control group.
Behaviour change in infrequent behaviour is fairly easy to modify (getting someone to give blood). Behaviour change in frequesnt behaviour such as driving is far more difficult to obtain.
Could you provide a link for your evaluation of the courses?
Incidently what do you mean about Public Money? Speed courses and the like are self funded by the attendees and not the public purse and would not come under tha same scrutiny. Most people involved in road safety would jump at the opportunity of providing courses to the public that are funded only by the attendee.
Keith
0
Simply measuring the before and after effects of those who attend will go give no measure of success. What about factors outside the course that will have an influence?
The only effective measure is two groups, those that attend and those that are not allowed to attend. However, such evaualtion in the UK will not happen. How would you feel if your partner is given a course and you are not simply becuase they want to put you in a control group?
Keith
0
Do I detect a power struggle between these authorities? “Our intervention is better than yours”.
Like Iain, I was surprised by Sgt Webb’s comments. I’ve sat in on speed awareness courses, driver improvement courses and fire and rescue presentations/demos but because they vary in how and what was presented, comparisons are not really fair, however none of them were preaching or punitive. Whichever intervention of this nature you push, it’s down to the recipient whether they benefit long term or not.
Hugh Jones, Cheshire
0
The ‘Ripple Effect’ video is very emotive and one cannot fail to be moved. However these tragedies are I suspect typical of most ‘above speed limit’ cases in that they involved reckless speeds well above the safe speed for the conditions, let alone the limit and the first clearly involved many other factors, not least going the wrong way. Suspect it could even have been stolen? We have around 6% of KSIs occuring above the speed limit but how many are actually caused by drivers exceeding limits by small margins? Well, the fact that such cases are never presented by ‘road safety campaigners’ (I’ve yet to see one and I have asked many times) indicates they must be few. All the cases used invariably also involve drink/drugs/stolen/unlicensed or reckless ‘joyrider’ speeds which nobody condones. Yet the prime target by far of the prosecutors is the sober, otherwise legal driver doing a few MPH over a limit. I have yet to see any convicing evidence that such drivers are killing and maiming anybody despite the hype. And no, a buch of 6 year olds telling us it’s true doesn’t make it fact. Sadly anybody watching ‘police,camera,action’ etc will be only too aware of the lenient sentences given to joyriders.
Yes, you can evaluate speed awareness courses. You compare the accident record of the driver before the course with after. Measuring their compliance with rules is no measure of success.
Chris Wilson,Stourbridge
0
We will be contacting our members in the West Midlands who have the expertise and experience to assist our police and fire colleagues in setting appropriate evaluation in place for these interventions and within the initial design of any future programmes.
Honor Byford, Chair, Road Safety GB
0
Given the advances in telematics then maybe it is time for a change to the motor vehicle licensing to mandate that anyone who is recorded driving outside that license (ie not complying with the law) would be limited to the driving of vehicles with telematics installed.
Such telematics could then be used to monitor behaviour subsequent to any training or further non-compliance. What a rich source of data that would provide for measuring effectiveness of interventions.
Rod King, 20’s Plenty for Us
0
Over the years NDORS has failed to provide any qualitative data with regard to the effectiveness of thier alternative schemes. Simply regurgitating comments of former attendees does not carry the weight. Asking attendees to complete self-evaluating questionnaires is not a robust method of evaluating such schemes.
The USA have undertaken robust evaluatiuon of their alternative to prosecution schemes, by using two groups to evaluate. Those that are given the chance to undertake the course and those that are prosecuted. Each group had similar demographice with numbers running in to the high 50,000.
As long as the UK evaluates such programmes through self-evaluation reporting there will be no real measure of the success of such programmes.
Keith
0
Some amazing quotes in that story. I would suggest that the relevant constabulary takes a good look at it’s speed awareness delivery. If the comments are that negative it is clearly being delivered badly. Our experience is the opposite, where most clients feel that the course has been beneficial.
As for the idea that you can’t evaluate this type of scheme, yes you can. What you can’t prove is a direct correlation between the intervention and casualty rates, but you can evidence a change in attitudes over the short or long term…..if you are willing to make the effort. As a budget holder, any member of my team proposing a scheme with no hint or effort guaging the effectiveness would be sent away to think again. That’s public money you’re spending, you have a duty and responsibility to make the best possible effort to ensure effective spend.
Iain Temperton, Norfolk
0