MAVCON24, the UK’s energy measurement and verification conference, took place in Birmingham on 23 October, 2024. As part of the proceedings I ran an exercise on how we can measure staff engagement, particularly in sustainability, the aim being to quantify where a workforce is on the spectrum from disaffected to proactive.

We need numerical indicators because we need to measure, and more importantly track, progress. We want to establish whether interventions to improve engagement work in the first place, whether they continue to work thereafter, and where we stand relative to any improvement targets.

MAVCON has traditionally been focused on the objective evaluation of energy savings, and although that has typically related to technical energy-saving measures, staff behaviour-change now features as a significant component of most corporate energy programmes. The scope is also widening to include environmental impacts. When it comes to measuring staff engagement, therefore, purely quantitative monitoring of energy performance alone is not the answer because that only looks at energy efficiency and it is too easily confounded by coincidental impacts of other measures. Hence the need for metrics indicating the degree of engagement in its own right.

It is widely recognized that awareness and engagement on energy and sustainability reflect prevailing organisational culture. A workforce that doesn’t care about anything much will behave accordingly across the board, whereas those who have willingly engaged in improvement projects related to any aspect of their working environment are more likely to take a positive attitude in other contexts. That is useful because it means that we can observe behaviours maybe only loosely related to our primary focus, and then take what we see as a proxy for overall engagement. The first time I encountered this thinking in practice was in a factory where they periodically counted the number of safety near-miss reports: any fall-off in the number implied slipping engagement levels in general.

The recommended methodology involves identifying a set of ‘index behaviours’ (like the near-miss reports for example), which can be periodically evaluated by counting or measurement, or by scoring on a subjective scale. The scores are then combined in a weighted total. High weightings would be given to important behaviours which provide a lot of data points and can be scored objectively, while low weightings are given to infrequent or less significant phenomena or that can only be evaluated subjectively.

For the MAVCON exercise we divided the audience into small groups and asked each group to think of five observable behaviours related to employee engagement (not necessarily related to energy performance). We asked them to think how they would you assign a value to each, and what weighting it should have in the overall score. To give a bit of variety, the groups worked on one of three scenarios:

1. a factory making products from round steel bar that involves cutting, bending, heat treatment, degreasing and powder-coating and using hydraulic, pneumatic and electrical equipment;
2. a rural district council with: a central office housing administrative and management staff, with meeting and training rooms, customer service desks and working space for elected members; three leisure centres operated by a contractor; and significant number of staff working away from base; and
3. a wholesale food service company with ambient, chilled and frozen warehouse storage receiving bulk deliveries from suppliers and mixed loads of product then despatched daily in refrigerated lorries to customers within a radius of 60 miles.

With eleven groups to debrief there was not time to feed back every suggestion but we harvested some headlines as follows. Firstly the factory groups came up with:
• number of energy defect reports;
• scored energy checklist; and
• amount of waste

The groups thinking about the district council proposed these:
• attendance at briefings;
• training requests;
• amount of paper used;
• number of suggestions;
• level of volunteering; and
• incidence of out-of-hours running

Finally the top ideas from the group looking at the food distribution company were:
• quality of maintenance;
• vehicle telematics data (see my comments below);
• freezer alarm frequency;
• staff absences; and
• lost product

The group identifying vehicle telematics data were thinking of it in terms of fuel performance but I suggested they could think instead in terms of purely behavioral data such as harsh braking, speeding and so on. The ‘training requests’ idea is a good one, but it was rather narrowly focussed on energy training whereas it could be broadened to encompass all requests for discretionary training of any sort. The main thing the groups tended to get wrong (in my view) was to choose behaviours that would not be easy to score or measure. ‘Quality of maintenance’ might fall into this category: I would like to see this expressed as a countable quantity even if it means focussing on some specific thing like tyre pressures or poor burner tuning where a threshold can be set. Some groups also defaulted back to energy performance, which we were trying to get away from for the reasons stated earlier.

I’d like to thank the groups’ representatives for graciously accepting some robust criticism during the debriefing and hopefully my ‘grilling’ (in the words of one of them) helped everyone understand the principle of the behavioural approach to measuring staff engagement.

The behavioural approach has a number of advantages. Firstly, it avoids the pitfalls of self-reporting where you just ask people about their attitudes. Secondly, unlike staff surveys it does not rely on achieving a high participation rate, because it inherently samples the entire enterprise. Thirdly it measures outcomes rather than intentions and fourthly it isolates engagement from other energy-saving activities. Moreover, publishing the calculation method and criteria gives cues to the workforce about what is expected of them. In that respect it works very like many corporate bonus schemes and indeed it is quite likely that there would be overlap with such schemes.

In conclusion, adopting a behavioural scorecard methodology will enable your organisation, when it embarks on an employee-engagement project, to set a target for the campaign, to assess it effect, to measure continual improvement, and to detect loss of engagement which signals a need for refresher training or other booster activity.

I would like to thank Dr Hilary Wood and the team at EEVS firstly for taking MAVCON over from me so that it can continue, and secondly for the quality of the event that they put on. This was the first face-to-face MAVCON since 2018 and it was great to see so many old friends and also to meet in person a number of people who have attended my online events over the last four years.