NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

The COVID-19 pandemic is pushing research projects around the world to move from face-to-face data collection to shorter, phone-based methods using SMS messages or voice calls. As research continues in this modified form, a critical challenge will be to keep studying how the pandemic affects poor and vulnerable households under these new constraints.

Measuring poverty in a lean, remote way has been the foundation of IPA’s Poverty Probability Index (PPI) since its inception. Here, we share a few of the strategies we’ve learned along the way, and discuss how we can support researchers and practitioners as they adapt to measurement in a pandemic.

Finding and Understanding Poor Households

In the coming months, many organizations will be working to focus their response efforts on the people who are most likely to suffer from COVID-19’s health and economic consequences. The PPI is a short set of questions designed to estimate the probability that respondents are below national or international poverty lines. It’s been in use for years: It was originally commissioned in 2005 by the Grameen Foundation (with the help of CGAP and the Ford Foundation) specifically to make it faster and easier to assess the likelihood that the group of people an NGO program or research project focuses on is truly below the poverty line.

The PPI is typically a set of 10 easily answered questions like “What is your roof made out of?”, “How many of your kids are in school?” and “Did you eat any rice this week?”. This makes it easier and faster to administer than typical consumption surveys, which might ask dozens or hundreds of questions with answers that are hard for respondents to recall (e.g.: “Do you know how many grams of bread you’ve eaten this week?”). Instead, the PPI uses machine learning models with nationally-representative survey data to come up with a quick series of easy-to-answer questions. The question sets are tailored to more than 40 different countries, because what’s predictive of poverty status in (for instance) China may be very different than in South Africa or Egypt.

While a useful tool in many contexts, the PPI can also be customized for organizations’ specific contexts or platforms. For example, IPA’s Poverty Measurement team recently worked with researchers at i2i to adapt the standard 10-question Rwanda PPI to a shorter and cheaper six-question version optimized for SMS text responses.

Adapting Ongoing Research to Phone-Based Measurement

In addition to its work on the PPI, the Poverty Measurement team builds phone-based poverty tracking surveys that take advantage of existing data, letting researchers monitor changes in welfare during a crisis. COVID-19 is likely to have far-reaching economic effects as people get sick, consumers stay inside and businesses shut down. Existing studies with baseline poverty estimates, along with tracking surveys that can measure short-term changes in welfare, will be a vital resource in understanding these impacts.

One example of this approach in practice is a project that PPI director Elliott Collins worked on as part of the BRAC-CEGA Learning Collaborative, evaluating cash transfers and the Graduation model in South Sudan. In the last phase of the study, the researchers were forced to cancel follow-up rounds of in-person surveys as the armed conflict there expanded. Instead, they ended up pivoting to faster and simpler phone-based surveys. To refine the surveys, the team returned to the first three rounds of data and estimated a flexible structural demand model to reveal which household and consumption questions were primarily driving the results so far.

More recently, IPA has helped our partner 60 Decibels implement their lean-data framework as they track the welfare of farmers across seasons. In lieu of the PPI’s usual focus on chronic poverty, IPA designed consumption-focused phone-based PPI’s that helped 60 Decibels see which farmers were falling into temporary poverty during the lean season, while waiting for the harvest to come in. Using these poverty tracking surveys, they found that 42% of the farmers in their sample were living below the $3.20/day poverty line in the resource-constrained pre-harvest season, but that this figure fell to 23% after harvest. Combining baseline studies with tracking surveys can help uncover these nuances, improving development programs’ ability to address the shifting challenges of poverty.

Measuring Resilience and Financial Health in the Face of Economic Uncertainty

With the prospect of a looming global recession, household financial resilience and vulnerability have become a centerpiece of programmatic and policy planning. Many are now asking: Who might be in a brittle financial position during the coming downturn, at risk of significant hardship if things go wrong?

A project with IPA’s Financial Inclusion Program, supported by the Bill and Melinda Gates Foundation, aims to answer that question. It’s tracking 9,000 people and developing a multifaceted approach to understanding who might be financially vulnerable when difficult times arise, such as during a global economic crisis. IPA’s research suggests that measuring financial health can give researchers and non-profits an idea of who is most vulnerable during a crisis or sudden economic downturn. Understanding the prevalence and causes of this kind of vulnerability can help governments decide where social safety nets need to be expanded, help NGOs target emergency cash and in-kind transfers, and enable the financial sector to find the most supportive way to provide credit and liquidity.

The Poverty Measurement and Financial Inclusion teams are now looking for opportunities to partner with organizations that work with vulnerable populations and could use the global financial health framework to quickly assess how susceptible these households are to economic hardship.

Switching to Phone Surveys isn’t All Bad

Phone surveys will be a dramatic logistical change for many research teams, and the shorter modules that work best with them mean it will be hard to replicate existing in-person survey modules. However, while COVID-19 is throwing a wrench in many best-laid research plans, moving to phone-based surveys has some advantages. Lower cost and lack of travel time means phone surveys can be run with high frequency, even in large samples. This can make results more precise and reliable, even when fewer variables are being measured. More numerous and frequent rounds of data and larger samples to accurately measure poverty and resilience will be key as the research community attempts to track the short-term economic impacts of the COVID-19 crisis.

If you are interested in collaborating on a data collection project to assess and track the welfare of vulnerable households, from long-term poverty tracking to quick, one-time financial health assessments, please reach out to Elliott Collins at ecollins@poverty-action.org.

Elliott Collins is Director of Poverty Measurement and John Branch is Policy & Communications Associate at Innovations for Poverty Action.

Photo provided by organization.