While students enjoy using VR and find it engaging, limited or one-off exposure to road safety education is unlikely to bring about ‘meaningful changes in attitudes’, according to evaluation of a Scottish road safety programme.
driVR is a 50-minute classroom intervention, aimed at 16-18 year olds, which utilises virtual reality (VR) to encourage participants to consider their attitudes towards road safety.
Funded by Transport Scotland, the project is delivered by Safety Cameras Scotland, Police Scotland and Glasgow City Council. Earlier this year, it won Education and Training Initiative of the Year at the 2019 Young Driver Road Safety Awards.
During driVR, students experience two VR films, take part in group discussions and complete a workbook – to encourage them to consider changing their behaviour in order to make them safer on the roads.
The first film shows a female pedestrian in her late teens who is on her phone constantly – while the second, filmed from the front passenger seat of a Ford KA, follows a young male driver, with a similar aged male in the back, who displays a number of bad and illegal behaviours.
In the final minute the viewer realises that these narratives are set in the same timeline and the car and pedestrian will soon collide. The experience ends with a fade to black rather than showing any gore.
At the end of the lesson, students are asked to set themselves goals and are provided with promotional items, including VR glasses, so that they can discuss and share their experience with friends and family.
driVR was recognised at the 2019 Young Driver Road Safety Awards
Evaluation of the scheme, carried out by TRL’s Dr Neale Kinnear, concluded that while students found driVR engaging, the limited exposure was unlikely to deliver ‘meaningful change in attitudes’.
The evaluation will be used to modify the initiative moving forward – including more exposure to road safety education.
Simon McLaughlin, Safety Cameras Scotland, said: “TRL’s evaluation found that first and foremost, driVR does no measurable harm. It also showed students enjoyed using VR and found it engaging.
“However, the research also showed that driVR did not demonstrate any meaningful change in attitudes towards road safety. While this result is disappointing, it is no surprise given the lesson’s limited dosage.
“Based on this feedback a series of recommendations have been made by both TRL and the driVR creators.
“Primarily, it is recommended that students’ exposure to road safety education is increased and that we continue to evaluate our efforts to ensure we are causing no harm – and eventually making a positive impact on road safety behaviours.”
Downloads:
driVR TRL evaluation report (pdf)
driVR pilot project overview (pdf)
driVR deliberately set out to put itself under microscope and used one of the most robust evaluation methods possible, something desperately lacking for road safety interventions. This was a key aim of the project and a success in itself as there is learning in the results, which is an important point in evaluation.
Whilst it is disappointing that we didn’t see a big shift in positive changes, we never expected this from driVR. A 50 minute lesson was never going to be a silver bullet. What is disappointing too though is to see cynicism regarding the awards. No young driver road safety intervention, to my knowledge, has been evaluated to this degree and been shown to make the changes suggested we should expect. What should be recognised however is that appropriate and measured steps are taken to work towards achieving those kind of changes. We believe driVR attempted to do that and, gratefully, the awards panel seemed to agree.
I urge anyone to read not only the evaluation results but also the pilot project overview to see what we learned from this and to see what recommendations have been made that would benefit not only this project but others like it. I hope that this might show that the driVR project was worthwhile and that the award was warranted. Hopefully this is just a starting point towards driVR, used alongside other resources, ultimately having the impact we all want to see.
Simon McLaughlin
+1
I agree with Matt and Rob in that it’s great to see new schemes being robustly piloted and evaluated. I think it’s important to set realistic expectations when it comes to interventions with a relatively short touchpoint.
Evaluation is essential to ensure public money is being spent effectively and the public are protected from potentially harmful education. However, when small to medium size interventions are properly planned by those experienced in behavioural change and the relevant subject matter, the risk of causing harm is surely reduced.
In respect of the Awards – I’m not personally involved in the judging process, but I do know the panel comprised some very experienced and well-respected road safety professionals and academics with a robust scoring criteria. If schemes were pending evaluation, then a spotlight would have been directed towards the research, planning, behavioural change methodologies and other indicators of best practice.
James Evans
+11
I am with Matt Staton. A national award for a programme which makes no difference? What’s the point of the programme? How credible are the awards?
Rob Tillier
+2
Really good to see this kind of external evaluation taking place. Just a shame the awards scheme didn’t consider waiting until after the results were found. Can you imagine if it had come back to say it had been harmful and had already been announced as Education and Training Initiative of the Year?
Matt Staton
+8