In this piece, INvolve’s Head of Advisory Ryan Vincent details some of the ways your organization can create a strong framework for DEI impact measurement.
We’ve all heard the age-old adage, ‘what gets measured gets done’, and for the most part, it’s true. For one, it is far easier to hold individuals and institutions to account for their actions when the need for accountability is supported by metrics. Whether these are project evaluations or analyses conducted over a certain time frame, assessing performance based on reporting provides clarity on actions that are working, and where refinement is needed.
DEI, and broader culture change programs, have long eluded consistent, accurate, and, meaningful measurement. Even with the advent of ESG reporting and its continued impact on investment, business strategy and accountability, DEI’s place in it remains inconsistent.
Measuring DEI can be challenging and poses questions that businesses need to be able navigate.
When it comes to reporting, we often ask nebulous questions like:
How does one measure how anti-racist your employees are being?
How do we keep track of the number of microaggressions middle managers commit before and after an inclusive management workshop?
How does the level of understanding and empathy a cis-man has for trans issues in the workplace change after you invited a speaker on trans-rights for a Pride event?
How authentic is our C-suite’s commitment to DEI initiatives?
The challenge is, while DEI work touches every part of the employee lifecycle, it’s also extremely personal, often heavily influenced by experiences outside of the workplace as well as media, and the socio-political climate. There are so many factors creating both tailwinds and headwinds for your average DEI initiative, that it can be challenging to pinpoint exactly where culture change has come from, what is accelerating or decelerating it, and importantly, if it’s having a net-positive impact on your business and the people within it.
The questions we ask ourselves concerning DEI initiatives, individual workshops or learning programs and DEI strategy more widely, tend to be ephemeral and too easily swayed by subjective opinions. As DEI professionals, our motivation often sits in the minds we want to change, the small cultural-wins that over the years add up to paradigm shifts and the lives we hope to impact for the better. There’s sometimes a mismatch between what we want to do and what pragmatic, money-minded businesses we operate within want to see.
So how do we consolidate all of this into an operational and effective framework for measuring the impact of DEI initiatives?
It’s simpler than it might seem:
Get Granular
Where local data collection regulations allow, be as granular as you can with your data collection efforts. Tie demographic data to employee engagement scores and surveys, attrition, recruitment, performance metrics and your talent pipelining. Make sure the demographic categories you are using are also specific enough to be able to unpick nuances.
This means no more hiding behind broad categories like ‘BAME’ as building a true understanding of your organization requires more specificity. In jurisdictions where this isn’t possible, get creative with how you illicit this feedback and understanding. Across most of the European Union for example, if you are able to justify a ‘legitimate interest’ in collecting the data (for which many DEI initiatives, such as ensuring equity in your recruitment practices, qualify) or can ensure no one individual is identifiable through their sharing of such data (such as via anonymised surveys not tied to employee data) and allow people to ‘opt-in’ (giving explicit consent for their personal data to be collected) you are in the clear. If you still find resistance, setting up qualitative feedback mechanisms (such as focus groups, shadow boards, or DEI councils) is your best bet.
Tell a human story
When we’re asked for metrics, our minds typically go straight for numbers – ROI, net promoter scores, engagement rates, etc. But DEI is inherently an emotive and deeply personal subject matter. Numbers alone don’t always do it justice or catalyze meaningful action.
Always collect both quantitative and qualitative data concerning DEI. If your annual engagement, DEI or culture survey doesn’t include an open-ended question concerning DEI at your organization, add one. From here, you can investigate themes and trends in qualitative responses which you might not even have included in your design of the main survey. Here’s another quote for you: “you can’t improve what you don’t measure”. It’s very likely there are many blind spots in your DEI strategy, open ended questions on surveys and regular listening sessions can be an excellent mitigation tactic to prevent these blind spots from hindering your DEI progress. And if you really, just can’t resist the allure of hard numbers, you can perform sentiment analysis on qualitative data too.
Be a scientist
Balance the need to ‘be human’ with the necessity to design and test your interventions and initiatives.
Every DEI initiative or programme inherently has a hypothesis, a statement that says, ‘if we do x, then y will happen’. Lily Zheng, DEI strategist and consultant recommends thinking of these as DEI ‘Theories of Change’. You may not need to report on metrics for every single event or initiative you run, but every one of these things should have metrics tied to them. How will you know that Black employees feel more psychologically safe in your organization following the rollout of sensitivity and inclusive management training for your middle-management population? Design specific feedback mechanisms and integrate useful metrics you already collect (such as a psychological safety index in your employee engagement survey, or number of incident reports to HR) every time you run an initiative which can help test your hypotheses concerning it.
Frame your metrics
Aligning your DEI reporting to the reporting which you already know gets your leadership, board or management springing into action is a great place to start for inspiration on how to conduct your DEI reporting.
Examine your organisation’s business strategy and design methodologies for reporting DEI metrics against each strategic pillar your organisation has in its core business strategy. The more clearly you can tie DEI metrics to the performance of the business, the more legitimacy your operations carry. Framing your DEI metrics around risk for example can be helpful in highlighting the importance of an issue or initiative. A recent study conducted by Gallup for instance revealed that 1 in 5 Gen Z adults identify as LGBT+. If your business isn’t set up to recruit, support and develop this talent, not to mention service this growing market, that’s inherently a risk to the sustainability of your business, and an impetus to do more DEI work around LGBT+ inclusion.
Create a solid foundation
One of the most fundamental challenges with DEI reporting and its lack of consistency is the inability to track changes year on year. Metrics are always changing, and because DEI evolves so quickly (remember D&I? LGBT? BAME?) tracking impact and change for culture related initiatives which, by their nature, can take a long time, is made even more difficult.
In this way, balancing the need for being granular with the ability to go broad and point to some stable and consistent metrics is vital. Create a solid foundation of core metrics you report on every single year, alongside your more granular and nuanced findings. What does attrition look like for the culturally dominant group vs. non-culturally dominant groups? Engagement? While this may seem to contradict our first point, it is also important to remember that DEI initiatives are, at their core, for EVERYONE in a business. Overall engagement scores, attrition, employee satisfaction, progression rates and recruitment metrics are critical to DEI reporting. Just make sure you always take a closer look and resist the temptation to use these ‘whole sample’ statistics to slap some rose-tinted glasses over your eyes.
You can spend just as much time thinking about how to measure your impact you do designing and implementing the initiatives which create it. But in a market operating with tighter and tighter DEI budgets, increased scrutiny on the ROI of DEI initiatives and outright attacks on the legitimacy of the practice, taking the time to carefully design a reporting methodology around DEI at your organisation is crucial to it remaining a priority for your business in the long term. Another harsh truth is that some DEI initiatives simply aren’t as effective as the good intentions behind them. Strengthening the legitimacy of our practices will not only mean DEI remains a priority, but that the practice as a whole becomes more effective and impactful.
To learn more about how INvolve can support your organization to create a strong framework for DEI impact measurement contact us at info@involvepeople.org
