The interplay between the articles in the March issue was particularly interesting. I think, for example, that Matthew Bishop genuinely misunderstands Bruce Sievers’ main point. I strongly suspect that Sievers did not mean to imply that we are not interested in seeing social impacts from our social investments.

Bishop’s misunderstanding points to a conundrum at the heart of social impact measurement. Bishop uses the principal-agent problem from agency theory to unpick accountability and measurement. In fact, for a number of reasons – not least being the inherent complexities in measuring social value (eg multiple causality, time delays, diverse contexts) – we need to push beyond agency theory to get a satisfactory analytical framework. I prefer what is sometimes called a stakeholder approach or model, which leads directly to an argument for an inclusive and collaborative approach to organizational governance.

When it comes to adopting frameworks for accountability (including measurement and governance) for social change making organizations, it turns out that the best approach is one that brings the entire ecosystem into view, and in which relationships among those effecting and affected by change are an explicit part of the metrics used.

To give one simple illustration, last year Keystone undertook a study for British Overseas NGOs in Development (BOND) to explore how to promote quality for their members. When we asked their members what most determined the quality of their work, one answer trumped all others: the quality of relationships with intended beneficiaries. I believe that measuring the quality of relationships with beneficiaries is precisely the kind of process measure that Bruce Sievers would approve of. I also believe that it will correlate more directly to actual social impact than most other measures. We need more and better data to test these propositions, but that is another story.

I concur with Bishop’s main conclusion that we need a ‘step change in quality and quantity of information available to donors’. I would emphasize, however, that the quality of our measures – including impact indicators – depends fundamentally on who generates and validates them. In the domain of social change, our most effective approach to metrics is one that utilizes interim outcome indicators like stepping stones on pathways of logical connection to our ultimate social impact goals. And to know whether we are on the right path, we make plans and assess progress with those in the best position to know – those who are constituent to the changes we are trying to bring about.

David Bonbright
Chief Executive, Keystone