Viewpoint: The Next Frontier in Social Impact Measurement Isn’t Measurement at All
Wednesday, May 11, 2016
Accepted wisdom is that we can solve the comparison problem with better impact measures (methods, definitions, and standards). This works on a small scale; many grantmaking foundations and impact-investing firms solve their comparison problems by mandating common measures across their portfolios. But on a larger scale—when initiatives and enterprises differ in mission; theory of change; or socio-economic, cultural or geographic context—common measures don’t work as well. Those closest to the impact sound a familiar refrain: Common measures ask the wrong questions, measure the wrong things, and miss thereal impact. Context affects how we ought to measure impact. The definition of “job,” for example, might specify a living wage and full-time hours in some contexts, but allow entrepreneurial self-employment in others. The more contexts vary, the more likely it is that a rigid approach displaces a more insightful one. In other words, the more we rely on common measures to solve the comparison problem, the more we end up compromising the meaningfulness of social impact measures themselves. This is why measurement alone cannot solve the comparison problem.
We can, however, achieve comparability by focusing on the analytical skills needed to compare social impacts without mandating a rigid set of required metrics. The premise is that efficient capital markets demand analysts who are capable of interpreting and comparing apples and oranges. Why? Because they understand fruit. The market is best served when each organization can measure its social impact in the way that is most meaningful and insightful to its aim and operations, as long as it follows common principles for good measurement. Drawing insights from financial accounting, good analysts focus on measures that are flexible and adaptable to different contexts (within limits), applied consistently (organizations pick an approach and stick to it), and well disclosed (bring on the fine print!). We achieve comparability not at the moment of measurement, but after the analysts adjust, aggregate, and interpret the measures that get reported.
It’s better to manage variation than to eliminate it.
The most die-hard proponents of standardized social impact measures argue: “Sure, we lose something when all organizations use the same measures, but what we lose is so much smaller than what we gain. We all just need to commit to one method and stop wasting time on tiny details.” Financial accounting once gave that a try, and found it untenable.
- Impact Assessment