Measuring impact: who counts?

When one starts thinking about measuring impact, one seems inevitably to be pulled in two opposite directions. On the one hand, there’s the demand for ‘metrics’. This demand often comes from those working with donors, who are said to be looking for measures analogous to those available to people thinking to invest in the mainstream stock market. The claim made is that if we had better metrics, a functioning philanthropy marketplace, donors would be prepared to ‘invest’ more – the sort of claim that is often made by those promoting transparency and accountability. The image here is releasing a floodgate and funding for good causes flowing out.

On the other hand, you have the intractable phenomenon of social change, of the many and complex factors that are involved in transforming society, changing attitudes, tackling the underlying causes of poverty and inequity. This phenomenon seems obstinately to resist the sort of metrics that investors are said to want. Even if you can measure change over a period of time, it is near to impossible to say exactly what caused it – the much-cited problem of attribution.

The interview with Percy Barnevik reminds us how much can be achieved that is readily measurable – children passing exams, women taught to read and write, enterprises started, jobs created. Who is going to say this is not worth doing? Elsewhere in this issue, Paul Shoemaker cautions funders against seeking risk for the sake of it. ‘The clients served don’t care if a programme is cutting edge, revolutionary or “high risk”. They just want a better life.’

This leads straight into the question that is part of the title of the special feature: who counts? This question – coined by UK NGO Mango, which helps NGOs strengthen their financial management – is nicely ambiguous, both its meanings equally relevant to the issue of measuring impact. First, who counts, who matters? Paul Shoemaker reminds us that it is the clients, the beneficiaries, the people ultimately affected by programmes, who matter.

Second, who does the counting? Mango maintains that financial reporting to beneficiaries improves NGOs’ impact, enabling beneficiaries to make sure that funds are spent on their real priorities. David Bonbright and Perla Ni, writing in this issue of Alliance, similarly insist on the importance of involving those affected by programmes. ‘The quality of an organization’s relationships with its beneficiaries and other constituents is highly predictive of its effectiveness and impact,’ says David Bonbright. Meanwhile Paul Brest insists that none of this is ‘a substitute for hard data’, and the search for ever better ways to measure impact continues, as highlighted in this issue of Alliance.