Tuesday
July 9
2013

Jeremy Nicholls

Measure for Measure Series: Data, data everywhere? Creating bottom up approach to harnessing information

Editor’s note: Measure for Measure is a NextBillion series that focuses on trends, tools and viewpoints in impact measurement. Check out The Big Idea page on NextBillion for other posts in the series.

Two years ago a familiar question for anyone justifying the value of his or her work was: “But how do you measure it?”

Since then there has been an explosion in the ways of measuring; Google any outcome you want to measure and you’re likely to find a ready-made solution. There is more information being collected and more being made available, as freedom of information and Big Data take hold. There are now many more ways of measuring more subjective changes using scales, for example, from fairly general scales for gauging self-esteem to very specific scales like the care giver burden scale.

These days, it’s not just about measuring how much change has happened. Increasingly, the focus is on the value of that change in anything from, say, the value of amenity trees to the value of low-level dementia. The challenge now is not so much how to measure, but which measurement approach should I use for a particular outcome?

At the same time, as methods of measuring different outcomes expand, so too has the interest in reporting on the outcomes of an organisation’s work, whether this it’s social or environmental value, negative or beneficial impact. There has been growing recognition of the importance of non-financial value. Compared to financial value, we barely measure non-financial impact. Therefore, we don’t use it to influence decisions and direct money to activities that are not creating as much value as they might otherwise.

There is, at least, one major risk with all this activity and one major challenge.

The risk arises from the driver for this information. Unfortunately, the driver often is not coming from organisations and their governance, seeking to create as much value, impact or benefit as they can given the resources at their disposal. Rather, it is coming from investors and funders. The organisation’s involvement in measurement is still primarily to prove its effectiveness, and therefore, to raise more money.

Whether it’s a case study on a single person or a way of measuring a specific change experienced by an entire population, information is being summarised so it can be used. In many cases, investors and funders need more summary data than boards of directors. If an organisation is generating information for business decisions it will be able to summarise the data in different ways for different users, including investors and funders. If it isn’t, then it won’t be so easy. The organisation will implement an information system for one type of summary data, which may not be useful for other users and may not be based on underlying data, which could be used for business decisions. This reinforces the problem. As information is produced that is not seen as especially useful (other than to maintain funding) this further delays a time when organisations’ governance demands information from management on creating effective, non-financial value.

In financial accounting, the information is based on transactions, many pieces of data, being captured regularly, which can then be summarised at one level of detail for management and at another for external bodies, investors, and regulatory bodies. The discussion on how things can change to create more value can always move from summary data to the underlying transactions wherever necessary. The system doesn’t lose information.

By contrast, information on social or environmental value is not based on transactional information, or information reflecting the value for different stakeholders as they transact. Therefore it is not so easy to consider how changes in products or services can create more value. The system was not designed for that purpose.

The main reason for this is that the beneficiaries of non-financial value cannot hold the organisation to account. This brings us to the main challenge, which is that information is not being shared across organisations. What is good and useful is not emerging from practice. Instead it is emerging from the needs of one particular, often internal, audience.

Standardization is one solution. That is, standardizing the way in which outcomes are measured, so that in sharing we can also understand. This is where IRIS is playing a major role. But we also need to find new ways to create a bottom-up approach to creating information, information that reflects the experience and value of those affected by an organisation’s work, those social or environmental outcomes, impacts or benefits. This way, organisations can start to debate how to can create more value, but on an even playing field. We need to legitimise that experience and its value, and then share it so we can explore where commonalities are emerging and so standardise as appropriate.

This is why we started wikiVOIs, a database that is open to anyone to add to, comment on and edit, while adding in their views of outcomes, or indicators or the values of those outcomes. It is based on two ideas; the first being there can be many, many outcomes and potentially many, many different ways of measuring them. The second is that we need some construct to understand the information and use it to make better decisions. In wikiVOIS, the ordering comes from the structure that people experience outcomes, those outcomes can be measured and a value from the perspective of those affected. So VOIS stands for values, outcomes, and indicators (for) stakeholders. It’s a way of giving a voice to those affected, of legitimising their experience.

It’s a public good, owned by a social enterprise, which is owned by its members. We work closely with others, for example IRIS, so that data can be shared as appropriate. We are actively searching for organisations that want to share information and we encourage you to add data and to suggest improvements.

Jeremy Nicholls is the chief executive of the Social Return on Investment (SROI) Network, which is a membership organisation for individuals, organisations and companies supporting principles and standards in accounting for social and environmental value.

Categories
Impact Assessment
Tags
Impact Assessment, measurement, metrics