V. McIntyre

NexThought Monday – The (Financial) Wisdom of the Crowd: How a group of non-experts outperformed traditional credit ratings at identifying good and bad borrowers

Determining risk has always been a hurdle for financial inclusion practitioners, as well as a barrier for small and micro-businesses looking to expand. But if non-traditional measures prove dependable, they may fill the information gap and make lending to poor entrepreneurs and small businesses more feasible.

A new study shows that individual peer-to peer lenders using the information given them through online loan applications can predict borrower default much better than traditional credit scoring. This is encouraging news, not only to the P2P industry, but also to poor borrowers in developing countries and those who lend to them. The researchers will be discussing the implication of their findings for financial inclusion at an upcoming executive education program at Harvard Kennedy School.

Working with the American P2P website Prosper.com, Asim Khwaja, co-director of Harvard-based Evidence for Policy Design, and coauthors Rajkamal Iyer, Erzo F.P. Luttmer and Kelly Shue analyzed 194,033 borrower listings from the site, 17,212 of which were funded.

The website functions as a market where individuals can offer loans at different interest rates to borrowers, giving higher rates to those perceived as higher risk. So the interest rate a borrower receives can be viewed as a proxy for the borrower’s credit worthiness – one based on the collective perception of non-experts using a somewhat different set of tools. These peer lenders had access to some of the standard “hard” financial information commonly used by banks and credit rating agencies, such as the borrowers’ debt-to-income ratio and number of past delinquencies. But they also could view nonstandard or “soft” information, some of it unverified and voluntarily provided, including the maximum interest rate borrowers were willing to pay, their photograph, and their own description of why they need the loan and what made them good credit risks.

Khwaja and his co-authors then compared how well the collectively determined interest rate did in predicting default compared to the borrowers’ exact credit score, which was unobservable by the peer lenders and, as a traditional credit rating, based on more and finer-grained financial data.

The results were surprising: online lenders predicted an individual’s likelihood of defaulting on a loan with 45 percent greater accuracy than the credit score.

Next, the researchers developed an even more challenging benchmark. They made a new score based on what an econometrician would say if he had access to all standard financial information and included all the coded information available in the P2P website. They gave the econometrician a further advantage, providing him in-sample data on default realizations, something that is unavailable to any bank in any loan application.

They found that the peers predicted with a remarkable 87 percent of the accuracy of the econometrician.

While the borrowers in this market were generally those with lower credit scores, the online lenders were better at spotting good borrowers than bad. They achieved 92 percent of the predictive power of an econometrician for high-quality borrowers, and 77 percent for low-quality borrowers. Compared to the traditional credit rating, they were twice as good at identifying high-quality borrowers, and 52 percent better at spotting low-quality ones. This suggests that P2P markets could provide effective financial screening for all borrower types – a finding that holds promise for both developed and emerging economies.

Non-traditional measures for non-traditional borrowers

The Prosper.com study was not Khwaja’s first attempt to answer the question of how to predict default in borrowers with little or no credit history or collateral. His project Entrepreneurial Finance Lab (EFL) incorporated psychometric testing into credit scoring for such borrowers. Starting as a Harvard project in 2010, EFL developed an automated psychometric application that took a half hour to an hour to complete and measured the borrower’s attitude and outlook, ability, business acumen and character. They tested it in many contexts, including with an organization in Peru lending to micro, small and medium-sized businesses. The resulting study showed that applicants rejected by the new test had a probability of defaulting up to four times greater than those accepted. The test is now used by at least 36 financial institutions in 27 countries.

Khwaja notes that, “There is a lot of excitement in the fintech space these days. The lesson from the financial crises is not that we shouldn’t innovate, but rather that we need to come up with well-thought-out and innovative designs that can leverage new technology such as crowdsourcing and nontraditional data like psychometrics.”

In May of this year, Khwaja and other financial inclusion researchers will explain their latest findings at Rethinking Financial Inclusion: Smart Design for Policy and Practice, a weeklong program offered by Harvard Kennedy School Executive Education and Evidence for Policy design. In afternoon group sessions, Khwaja will teach “smart policy design,” a framework for policymakers and organizations to test and refine their projects. The appetite for such training bodes well for the growth of evidence-based approaches to financial inclusion innovation: Last year’s program attracted participants from banks, microfinance institutions, NGOs and government ministries in 29 countries.

V. McIntyre is a freelance writer for the Harvard Kennedy School.

credit scoring, lending