NexThought Monday – Collecting Data That Matters: Nine tips for making sure your organization extracts the most value from measurement
There’s been much debate recently about whether or not businesses should measure impact. But that’s the wrong place to focus; instead, the focus should be on measuring responsibly – that is, collecting data that is right for an organization.
Measurement is an iterative process. Metrics can and should shift as a company evolves from design to scale. Organizations should focus on collecting data that helps them improve their operations, such as data that can lead to them creating more value for their customers, producers and/or the communities where they operate. This could include data that informs more efficient use of resources, adaption of products and services to user’s needs, improvement of business practices, and achievement of greater impact both financially and socially.
Creating impact and improving performance is thus tied to creating value for customers and the business. At the second annual BoP Global Network Summit July 16-17, 2015, in Burlington, Vt., fellow panelists and I explored tools and methodologies businesses are using to understand how they are really doing and to improve mutual value. The session also examined how such metrics can evolve as a business progresses through different stages.
The session kicked off with a high-level review of past and recent debates related to impact assessment, and the current state of assessment borrowed from an article my colleague, Yaquta Fatehi, and I wrote for a special issue of Thinkers magazine, released at the summit.
During the session, Mayke Harding, senior expert at the BoP Innovation Center, spoke about how a company adding a new product to its portfolio used social return on investment (SROI) to forecast impact. Harding demonstrated that forecasting SROI during the design stage allows businesses to better understand how and when value to the BoP is created, which leads to additional activities to increase the SROI ratio.
As program manager at the William Davidson Institute (the parent organization of NextBillion), I then spoke about using multidimensional poverty impact data to predict sales and retention of a business. In addition to assessing changes in multidimensional metrics such as self-esteem, empowerment, etc., to enhance positive impacts and mitigate negative ones, using impact data for prediction allows a company to learn valuable information to inform operations. This could include assessing the traits of employees who stay versus leave and who perform well versus poorly, in order to create tailored development plans for employees as well as inform recruitment.
The next presentation focused on a larger business that has scaled in many locations. Rob Michalak, global director of social mission at Ben & Jerry’s, shared how they are using metrics to understand their progress on sourcing based on key company values, and to inform resource allocation. Emily Shipman, program director at the Sustainable Food Lab, then provided an overview of a community of practice that is collaborating on shared learning questions to assess smallholder supply chains in order to help strengthen and scale the development impact of these supply chains.
Highlights/takeaways from the session included:
• Metrics are essential to inform decision-making, understand impact, monitor progress toward targets and identify areas of opportunity.
• Many professionals now accept all assessment types (impact, process, performance, etc.) and methods (qualitative and quantitative) and use a range of these to incorporate multiple perspectives and types of indicators depending on their organizational needs and budget.
• Using assessment to create value works best when it is driven by the business or organization being assessed, as it ensures reporting is relative and in line with organizational goals.
• Evolve with your metrics – as you grow and your metrics grow, they can become more robust and complex. Think big, start small. Focus on collecting data to answer priority questions and build your data collection from there.
• Be clear in the goals of data collection and the value you expect to extract from that data so that focus and clarity is maintained as you acquire data and use it.
• Assess only what you will use and what will bring value to the organization. This means you should have the capabilities to analyze the data, and be able to share the findings in a way that resonates with different audiences. This is a critical component to reaping the most value from assessment.
• Measure what you can actually affect – don’t measure for the sake of measuring, but ensure you’re measuring with good reason and in an area in which you can intervene. Having a theory of change can guide you in what to collect and what you don’t need to collect.
• Collected information can be valuable to internal and external organizations (data collected for a particular purpose can also be used to create value in other ways).
• Pay special attention to opportunities to deliver value to the research subjects through data collection. Do not merely extract data.
Some thoughts about current trends in assessment and where it seems to be heading:
• There’s movement toward data collection approaches that allow for quicker and less resource-intensive assessment such as rapid feedback loops, lean data, social performance and impact proxies, which can complement deeper dives.
• You should report back to those you collected data from so they can see the value of their contribution. With increased demand for surveys and data gathering, we need to be careful not to inundate people with too many questions, as this can result in the collection of less accurate data over time.
• Collaborate for better results. Many organizations have similar questions, and we can work together to create learning agendas, collect data, and interpret and act on the findings.
• There’s a need for more context-based reporting and cost-effective solutions for field reporting.
• We need to increase our ability to systematically review preexisting data.
• Assessment findings should be mapped to existing macroeconomic data.
Given the many roles measurement plays, deciding what to assess and how to assess it can be difficult. It is often easier to follow another organization’s lead and measure what they measure, than to think hard about whether it makes sense for your organization to measure impact yet. And, as was clear at the BoP Global Network Summit, the ways in which we measure our organizations are evolving, too. It’s a complex field, to be sure, but organizations and those they serve can reap dramatic benefits when we get it right. Getting it right relies on businesses leading assessment activities in order to ensure that any data collected creates value for them and their stakeholders.
Note: The author would like to recognize the panelists’ contributions in making the session a success, as well as Brittany Nunnink, a sustainable entrepreneurship MBA student at the University of Vermont, who helped capture key takeaways from the session.
Heather Esper is the program manager of impact assessment at the William Davidson Institute at the University of Michigan.
- Impact Assessment