NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

South Africa has arguably the most mature impact ecosystem in Africa. It has a large community of both seasoned and new social enterprises, incubators, accelerators, pitch competitions, awards and public and private sector funders. But though many parts of this rapidly growing ecosystem are well-established, others are still nascent in many ways.

One area that is playing catch-up is the impact measurement and management field. In a recent study, Landscape Mapping for Impact Measurement and Management (IMM) in South Africa, conducted by Impact Amplifier and Genesis Analytics, we explored the state of the sector in South Africa. We spoke to impact investors, intermediaries and social enterprises about their assessment practices to determine:

•  Why and how organizations are measuring their impact;
•  The benefits and challenges of impact measurement;
•  How impact data is used and reported; and
•  The future of impact measurement and management in South Africa

This study’s core intention was to benchmark the current status of IMM approaches in the country and to share experiences and lessons as a means of propelling the sector toward best practices.

Seven Takeaways from the Research

Some of the key findings included:

• 72 percent of study participants reported having an impact measurement and management practice;
• 66 percent believe that measuring impact is central to achieving company growth and organizational objectives;
• Those who are not measuring impact reported that limited resources, both financial and human, as well as insufficient knowledge and skills constrained their impact measurement capabilities;
• 32 percent reported alignment with formal impact measurement techniques such as Environmental, Social and Governance (ESG), Sustainable Development Goals (SDG) and IRIS;
• 61 percent indicated that they had been measuring impact for less than two years;
• 92 percent indicated that the top benefit of measuring and managing impact was to better demonstrate how their organizations make a difference; and
• Across stakeholder groups, investors were cited as the most important audience for impact results drawn from the data that is being measured, used and reported.

Impact Amplifier has been conducting investment readiness acceleration for social enterprises and raising funding from impact investors for the last eight years in South Africa. This landscaping study was an important exercise in providing a baseline for IMM, raising awareness and creating momentum for growing the measurement ecosystem in the country. But as the study’s scope and intentions were limited, its results did not reflect some of Impact Amplifier’s experiences, or some trends we have observed. It also did not provide a full understanding of the motivations behind some of the behaviors reported.

Questions About the Findings

The most glaring difference between our experience and what study participants reported relates to impact measurement and management by social enterprises. As mentioned above, 72 percent of the overall study participants reported having an impact measurement and management practice in place. But when isolating social enterprises from all study participants, that figure dropped to around 50 percent for enterprises who completed the online survey (the majority of study participants) and increased to 78 percent for those who participated in an interview. Why was there such a striking difference between social enterprise interviewees and survey respondents? Were interview participants too embarrassed to acknowledge their impact measurement inadequacies? Did survey respondents have a different understanding of what was being asked than interviewees? We cannot be certain about the answers to these questions.

What is more important than this 28 percent discrepancy, however, is that an affirmative answer to “Is your organization measuring impact?” did not qualify what was meant by “impact.” There is no common accepted definition of what impact means, and no standard methodology to measure it in South Africa. Even if there was, the study was not designed to verify the answers provided by participants. So although the percentage of social enterprises measuring “impact” is about 50 percent as per the study, what they are actually measuring is unknown.

This finding contrasts significantly with our work training and accelerating over 400 social enterprises in the last eight years: We’ve noted that the vast majority of these businesses are measuring outputs and outcomes, but less that 10 percent are actually measuring impact. Take for example a social enterprise which focuses on improving the lives of small farmers by providing technical support and low-cost working capital loans, with the intention of increasing yields to provide better food security and/or increasing income. In our experience, this enterprise would typically measure how many hours of training were delivered, how many loans were provided, and the repayment rates for those loans (which are all outputs) – and sometimes the increases in yields and income (which are both outcomes). What isn’t measured is how those increases in yields affected the farmer’s daily nutrition or food availability, or how that excess income changed the farmer’s access to education, health care, housing or clean water – not to mention how these changes relate to farmers who didn’t get the technical support or the working capital loans.

In another example, consider a social enterprise attempting to improve the quality of STEM education for high school students by enhancing teachers’ skills, and providing new teaching tools and other infrastructure. This enterprise would typically measure how many teachers were trained, and how much new infrastructure was provided (both outputs) – but it wouldn’t measure how the teachers’ skills have been enhanced (an outcome). And more importantly, it wouldn’t measure any improvements in students’ grades in STEM subjects, how those improved grades increased the number of students that pursued STEM subjects at tertiary institutions, or if those students graduated and pursued careers in STEM fields (all impacts). It also wouldn’t measure how much of that success can be attributed to the increased teacher skills and new infrastructure.

Why Aren’t More Social Enterprises Measuring Impact? 

In our experience, the primary reasons why so few social enterprises are measuring impact fall into one of the following key categories:

• Lack of Information: Most social entrepreneurs do not know the difference between an output, an outcome and impact. This is because impact measurement isn’t a subject they have studied at University, or been exposed to through their peer networks or mainstream incubators and accelerators, as most of these are focused on traditional entrepreneurship, not social entrepreneurship.
• Resource Scarcity: Even if entrepreneurs have an understanding of how to measure impact, it still requires time and resources that are often scarce – particularly for social enterprises in the earlier stages of maturity.
• Longitudinal Requirements: One issue that prevents many enterprises and organizations from measuring actual impact is that their impact only occurs many years after an intervention. This is particularly true for organizations working with younger people, whose work is focused on education, nutrition, healthcare and life skills interventions, as the consequences of these programs often may not be observable for 5-10 years. For an emergent industry with few social enterprises over five years old, measuring this kind of impact is not possible yet.
• Emotional Knowledge: People doing social justice work know they are making an impact, because they can feel it. Whether they are the entrepreneurs driven by a mission to support humanity or employees who signed on later, they interact with the beneficiaries of their work and witness a wide variety of positive changes. Formalizing and measuring this can appear unnecessary – time and effort poorly spent when the needs being addressed are so large.
• Fear: Impact measurement can evoke both conscious and unconscious fears within social enterprises that what they hold to be true may be proven false. Quantifiable facts about impact can disrupt long-held beliefs, require substantial cost to repair, and put their funding security at risk. So for some, it is better not to know.

These motivations are not unique to South African social enterprises – they apply to entrepreneurs and organizations working all over the continent and around the world, to varying degrees. Building an ecosystem that understands these obstacles and provides access to best practices, tools and other resources is the only way we can continue to move the industry toward evidence-based impact measurement. Social enterprises and impact investors in South Africa and beyond are creating solutions to problems many believed were either impossible to address, financially unsustainable or exclusively the domain of the public sector and NGOs. To continue and expand this momentum, it is vital that the sector have the resources and capacity to prove the impact it is genuinely making.

Tanner Methvin is a partner in Impact Amplifier and Earth Capital.

Image provided by author.