Blogs
What does ‘good’ impact measurement for climate/nature programmes look like? Jan 2024
‘Impact’ is a contested term, and attempts at measuring it can be challenging. My approach is to start with the question Impact for Who? Obviously climate change and biodiversity loss affect everyone, but when we’re making grants, we’re working with specific groups, people or projects. Having worked in many different types of grant-seeking and grant-making organisations, my starting point has always been that those who are supposed to benefit from the grant should determine what the anticipated impact should be. Donors, staff and others involved in the grantmaking side can be part of these discussions, ideally co-creating visions, strategies and impact measurements, but ultimately the outcomes of climate/nature programmes have to be felt as successful by the people who are living day-to-day with those outcomes.
In previous roles at grantmaking organisations, I’ve used different approaches to achieve this, for example:
At the Edge Fund, we held Radical Sharing Forums to understand our impact, by bringing grantees together to share updates on their campaigns with one another, which we then documented for broader learning. This resulted in building solidarity and peer-support, as well offering us an insight and overview into the successes (such as campaign tactics) and challenges (such as online fundraising) they faced.
At FundAction we undertook a values-based evaluation, where we determined what our core values would look like in practice, then interviewed grantees and others affected to understand whether these were manifesting. We compared this information with other data to build a picture of how we were performing. For example, one simple way to measure our value of ‘diversity’ was to visualise where our grants had gone (geography, theme etc) to highlight under-served areas.
At the Open Society Foundations, we used quantitative and qualitative measures to understand the impact of the programmes under review. I proposed the ‘Most Significant Change’ model (where grantees are asked what they perceive to be their greatest achievement with our funding, rather than asking them to report solely on activities they had originally proposed in their proposal), and combined this with research on policy changes or other clear ‘wins’ linked to our funding.
In some circles, ‘impact measurement’ is taboo terminology, associated with corporatisation. Activists, community groups and systems thinkers argue that we can’t track and monitor everything that matters, and that attempting to do so is reductive and distracting. I understand why this feeling prevails. In the international development sector, I was part of many meetings where top-down consultants seeking ‘value-for-money’ asserted that it was necessary for feminists working in the most complex circumstances to provide evidence of increased empowerment in the next year, or donors demanding that climate advocates specify the results of their policy-influencing work at the next COP, as though they were working in a factory: auto parts go in > cars come out. Grantees know that their funding depends on giving the right answers, and therefore this leads to people (at best) tying themselves in knots to twist their outcomes to fit the language of the funder, or (at worst) ‘juking the stats’. This approach works for nobody. But that doesn’t mean that impact measurement isn’t possible.
My suggestion would be: centre what those who are doing the work and living with the results (e.g. activists, indigenous communities, people at risk of natural disasters) believe is the needed impact, and then develop ways to measure this. For example, if they say they want to reduce deforestation, hold corporations to account, and raise awareness of the damage caused, then there may be different ways to achieve this. Targets such as legislation (such as effective implementation of the recent EU law on deforestation) could be combined with communications (such as screenings of the film The Territory) and legal work (such as strategic litigation on behalf of affected communities). These tactics and aims then all have their own impact metrics - meetings with senior officials held, news coverage, numbers of signatories to petitions - but priority should be given to building a good relationship with the grantee or people ‘on the ground’. Do they think this is working? Are they feeling the impact? In summary, in my experience, regular, honest and open communication is often the best form of impact measurement.
