NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

It sounds so simple: Everyone gets access to financial services and – presto – the foundations for the SGDs will be laid. We were intrigued by such claims made by World Bank economist Leora Klapper on NextBillion recently: “…to eradicate poverty, achieve gender equality, provide quality education, or meet any of the United Nation’s Sustainable Development Goals (SDGs), we must begin by creating a financially inclusive world.” Klapper’s piece also suggests that the World Bank’s Findex database on financial services usage somehow demonstrates this.

Sadly, both claims are incorrect. We were particularly incensed by the overstatements because, as the authors of a recent systematic review of reviews – probably the most exhaustive overview study ever done on financial inclusion’s impacts – we know just how messy and ambiguous the evidence is. What we see in Klapper’s article is not evidence-based policy-making, but policy-based evidence-making.

Overselling the (Mixed) Evidence on Financial Inclusion and the SDGs

Supposedly, as Klapper claims, “the evidence is clear: Financial services enable social and economic development.” In fact, our systematic review of reviews, which took stock of all existing high-level evidence on financial inclusion, finds the exact opposite: The evidence is mixed, inconsistent and often not of sufficient quality for drawing such conclusions. We’re sorry if that sounds too boring, but the inconvenient truth is that financial inclusion is not changing the world in the sense of accelerating progress toward the SDGs, let alone being fundamental for attaining them.

Why? First because, although the impacts from financial access are, on average, more likely to be positive than negative, they are not transformative in scope or scale. The review evidence we assembled mostly finds small changes in immediate outcomes – for instance, family enterprise growth – but not in ultimate poverty indicators, such as families’ incomes or net assets. The evidence also suggests that social impacts, such as women’s empowerment, often result from things like training, rights-education or group-building, which are wrapped up with financial inclusion interventions – but not from the financial service itself. This strongly cautions against having faith in the digitalist, fintech-based inclusion agenda that many of Klapper’s examples point to.

Second, regarding financial inclusion’s impact: Don’t take her word for it, Klapper says, as “repeatedly, rigorous academic studies” have shown financial services to be driving development and poverty alleviation. Her article then offers hand-picked examples. It was precisely to put a halt to such cherry-picking of evidence (which lacks external validity) that we undertook our own, systematic review. What we find is that positive impacts for some people in some places often don’t materialise for others in other places. Sometimes even the opposite happens (ie: negative impacts). What is true for one place or person isn’t necessarily true for another – this is what “mixed evidence” refers to. To use hand-picked, single-project, single-study data, and generalise from it, is widely understood to be unscientific – or, plainly speaking, “poor economics.”

Third, we found that many academic studies are methodologically too weak to support strong conclusions. This applies to systematic reviews, the majority of which we had to downgrade in our meta-review due to quality concerns – and even more to the primary studies underlying these (which are exactly the type of studies Klapper cites). To wit: Using a range of low-quality studies to try to paint a broader, exaggeratedly positive picture – especially one as impressionistic as the one Klapper presents – is risky. It’s reminiscent of the pre-2008 tactics used by banks that pooled dubious individual assets into “triple-A” structured financial products to mask their questionable quality.

Questioning Faith in the ‘Benevolent Power of Finance’

Moving beyond our own systematic review of reviews, we find that many economists at the World Bank still have an unshakable faith in the benevolent power of finance. They seem to assume that having more financial services will promote poverty alleviation, and they want the world of possible evidence to fit their assumption. To be clear: We are not saying the exact opposite – that financial services are inherently useless or harmful to poor people. Rather, we’re arguing that we need more convincing evidence for their supposed developmental beneficence. We think that, if one were to ask poor households to place financial services somewhere in a Maslow-type hierarchy of needs, they would place them near the top – nice to have, but not as essential as things like schooling, health, safety, water, nutrition, sanitation, and so on. Greater access to financial services is a logical by-product of the economic and social development that’s essential to the SDGs – it’s not the foundation for that development. That is why our review of reviews also looked at alternative approaches.

That brings us to Klapper’s other claim: that the Findex somehow shows the importance of financial inclusion. It doesn’t. It only measures financial service access and usage, not impacts. Don’t take our word for it: Take a look for yourself. Data-gathering is very important – and indeed we were hugely dismayed when, three years ago, the World Bank locked away its other major financial inclusion dataset, MIX, behind a steep paywall – but bean-counting should not be conflated with impact evaluation. Notably, despite the World Bank’s long-standing enthusiasm for financial inclusion (since the 1980s), our meta-review found no research on the social impact of financial inclusion funded or published by the World Bank that was sufficiently rigorous to be included in our study .

Perhaps we can end on a conciliatory note, highlighting where we do agree with Klapper’s article. Firstly, our review also finds that savings services matter. They aren’t transformative, but they have quite clearly positive effects, are rarely harmful and can be provided in a range of different ways. Secondly, we agree that financial inclusion needs proper regulation and robust protection for customers, especially (but not limited to) debtors, who often lack meaningful rights in practice. And thirdly, we agree that accelerated progress toward the 2030 SDGs is paramount. But for that to happen – and for financial inclusion to play a role – we will need better data (including the Findex database), better evidence, and better use of that evidence. Overstating positive findings, overlooking negative ones, and overselling one sector’s impact does a disservice to the global efforts to achieve the SDGs – and to the movement toward evidence-based development.

Phil Mader is a Research Fellow at the Institute of Development Studies.

Maren Duvendack is a Senior Lecturer in Development Economics at the University of East Anglia.