Josh Cleveland

SOCAP10: Learning for Social Impact

We’d hope that everyone at SOCAP10 is about learning. Discovering and discussing what works and what doesn’t in order to make a greater social impact is the main reason we’re all here this week in San Francisco. But few organizations are really good at measuring the social impact we’re all trying to create. Fewer still have institutionalized the structures, tools, cultural changes, and feedback loops necessary to learn from assessments of social impact. Laura Callanan is out to change that with her work at McKinsey.

Laura knows a thing or two about social impact. Her impressive background includes serving as Senior Adviser at the United Nations Development Programme and Executive Director of The Prospect Hill Foundation. As Associate Director at The Rockefeller Foundation, Laura had general management responsibility for the $3 billion endowment, and investment responsibility for the Foundation’s venture capital and private equity portfolio. Laura also co-managed the Program Venture Experiment, a precursor to Rockefeller’s current Impact Investing activities. She now leads McKinsey & Co’s research on social impact assessment and social investing.

Before or after reading this interview, I’d encourage you to check out McKinsey’s recent publication, Learning for Social Impact: What Foundations Can Do (.pdf), which goes into detail on the learning approach Laura champions.

In between discussions about business models, investing strategies, and everything else social enterprise-related, Laura took a minute to sit down with me at SoCap to explain how organizations can better measure the change they’re creating and better institutionalize what they learn from their past attempts. But first we discussed why measuring social impact is so hard in the first place.

Josh Cleveland, No one would argue that impact assessment isn’t important. But yet many organizations in the social sector struggle with the topic. Why is social impact assessment so hard?

Laura Callanan, McKinsey: Social problems are complex, dynamic problems happening in uncontrolled environments. We are used to measuring financial results on a quarterly basis in the corporate world. That just isn’t realistic when it comes to social impact. It’s a long-term endeavor. There are many factors to consider. The variety of detractors and drivers of progress makes it is very hard to know the exact role your work plays. Yet we still believe that it is possible of course.

Laura Callanan: It is possible. Set goals, measure progress, and assess what’s working. But be reasonable and realistic about what data you can collect. Clearly define what you want to learn. Think upfront about the information that will be most credible and relevant to your audience. Most nonprofit organizations have many different stakeholders– community constituents, investors, board members, the public. Each group cares about different things and has different standards for the information that matters to them. More than anything, approach social impact in a spirit of humility and open mindedness because you may learn surprising things. As noted on the McKinsey website, this field has been around for over 50 years. What are some of the major trends – good or bad – that you see in the field of social impact assessment today? Are things moving in the right direction? Why or why not?

Laura Callanan, McKinsey: Three trends give me hope that we are headed in the right direction:

First, the social sector is attracting new talent from diverse backgrounds who naturally want to measure results. People with experience in venture capital, technology, investing, industry, science, medicine are all getting involved in social entrepreneurship and the social sector writ large. They’re bringing their resources, skills, and energy with them, and thinking about new ways to measure impact.

That is not to say that the business sector can easily teach the social sector how to measure social impact applying only private sector frameworks. But if you have experience in a sector with clear and agreed upon measures of success, it gives you something to extrapolate from, a model to build off of. These social sector newcomers are applying their experience at measuring progress and results in other disciplines to the complex, dynamic social sector – and it’s great! It’s moving the social sector forward.

Second, the huge promise of social investing is motivating folks to figure out how to measure social impact now. There is a lot of discussion about the large amount of pent up capital interested in social investment. The assumption is that these investors are waiting to be able to quantify social returns before they start writing the really big checks. If they are being asked to accept a below market rate of return on their financial investment, they want to be able to assess the value of the social return they are generating, and conclude for themselves that the trade-off is fair. This is driving resources and brain power to social impact assessment. PULSE, IRIS, GIIRS, GIIN – these are examples of the efforts the social investing sector has launched to answer the question of “how much social impact?’ And from my perspective, if you can measure social impact in the context of an investment, you can measure it period.

Third, government interest in innovation and evidence is on the rise. The Obama administration is explicit about sourcing innovative ideas from social entrepreneurs. At the same time, they have set the bar high — demanding evidence that social interventions work before scaling innovative solutions. The holy grail of social innovation is getting a new solution scaled by government. This is another big motivator to crack the code on social impact assessment. The Social Innovation Fund accelerates this trend, — bringing financial resources and its convening power to discussions about standards and norms for social impact assessment. Let’s focus now on the specific work you’re doing at McKinsey. Tell us a bit about how Learning Driven Assessment fits into those trends and addresses some of the challenges we have been discussing.

Laura Callanan: Learning Driven Assessment is focused on finding what works, what doesn’t work, and why. And most importantly, feeding back what you find to inform decisions about strategy, program design, and implementation. In our white paper on Learning Driven Assessment, we’ve summarized five best practices for social impact assessment. Best practice number five is “creating a learning culture.” If you get this right, all the other steps will fall into line.

Work in the social sector is not easy. Learning requires experimentation and failure. Innovation requires risk-taking. Creating a learning culture requires leadership from the top of the organization. There needs to be a common understanding around what types of risk are acceptable, and how to manage and mitigate risk appropriately. People within the organization must alignment around what’s good risk and what’s bad risk. A true learning culture is not afraid of measuring results even when – especially when – you don’t know what the results will be.

My hat’s off to those social sector leaders who role model learning from expectation failures – the Hewlett Foundation with their Worst Grant Award, the Annie E Casey Foundation and their brown bag lunches talking about things that haven’t worked, the Robert Wood Johnson Foundation,which dedicated 100 pages in their latest Anthology to the topic of learning from failure. How is McKinsey in particular working to increase uptake around the Learning methodology? Are there partnerships with other assessment groups etc., which have helped in disseminating these techniques to the field?

Laura Callanan: One of the things I’m proudest about with our Learning for Social Impact initiative is that it has been highly collaborative. We sought to build off the knowledge and progress the social sector has made in understanding how to assess social impact. We worked closely with 13 leading foundations and reached out to more than 100 thought leaders, academics, social entrepreneurs, social investors and everyone in between.

One way we tried to add value while building off the work that has gone before was by creating a go-to repository of social impact assessment tools. We designed and created the Tools and Resources for Assessing Social Impact (TRASI) database containing 150 tools, methods, and best practices that can help practitioners assess social impact for programs across all sectors and geographies. The Foundation Center developed a great user interface and now manages TRASI. New submissions to the database are always welcome and I’d encourage NextBillion readers to go to to share their tools.

Our pledge to our partners was that we would make all the results publicly available for free in their entirety. We’re sharing our work through our web portal, which includes a variety of resources. Later this Fall, we’ll launch an interactive guide and workbook to help practitioners design an assessment plan. Stay tuned!