Fear of judgement impacting road safety evaluation

Big challenges remain in ensuring universal, high-standard evaluation of behaviour change interventions aimed at making people safer on the roads.

That’s the headline finding of a new report, authored for the RAC Foundation by Tanya Fosdick, head of research at Agilysis.

The report says many practitioners feel that they lack a professional identity, possibly caused by the absence of a professional qualification, “with participants suggesting that it can feel daunting to have one’s work judged; this in turn means that only positive evaluation results are published”.

Financial constraints also limit the amount of evaluation being carried out.

As part of the research, presented at the 2019 National Road Safety Conference on 13 November, practitioners were asked to complete a survey – while three focus groups were held with participants including road safety officers, road safety managers and fire and rescue officers.

The report also says financial constraints limit the amount of evaluation being carried out.

It identifies five groups when it comes to evaluation, with each having clear characteristics:

• ‘Absolute beginners’ – who used no behaviour change theories in intervention design, and evaluated none of their schemes
• ‘Believe in yourself ‘– who used behaviour change theories for less than 50% of their interventions and evaluated less than 50% of them
• ‘A little knowledge’ – those who used behaviour change theories for less than 50% of their interventions but evaluated more than 50% of them
• ‘Measure twice, cut once’ – those who used behaviour change theories for more than 50% of their interventions but evaluated less than 50% of them
• ‘Walking the talk (mostly)’ – those whose used behaviour change theories for more than 50% and evaluated more than 50% of their interventions

Tanya Fosdick said: “What struck me was the desire for a professional identity, that comparisons were made with engineering and public health, with those comparisons giving those in road safety education a sense of inadequacy and feelings of not being taken seriously.

“Many of the themes from the focus groups echoed those which emerged from the surveys. Limited qualifications can lead to a lack of professional identity in a diverse sector.

“In the survey, it was found that not all respondents were evaluating their interventions, a finding that was explained in the focus groups, with participants suggesting that it can feel daunting to have one’s work judged; this in turn means that only positive evaluation results are published.

“There was also an acknowledgement that good-quality evaluations can be expensive, and that this expense can lead to compromises in approach.”

---