NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

Thursday, October 2, 2014

Randomized controlled trials, an impact evaluation method, have been a contentious development issue of late, with as many advocates as critics.

A Devex survey of senior development executives across six continents and varied organizations revealed that nearly half aren’t familiar with the debate or have no opinion at all. This indicates that while RCTs remain a hot topic among academics and practitioners, the issue is not yet relevant to many senior managers. Another sizable group — 37 percent of executives surveyed — responded that RCTs are “somewhat overhyped.”

As billions of dollars are channeled each year toward meeting development goals like poverty reduction and basic education for all, tools to measure impact like RCTs are increasingly being used to inform policymaking.

The World Bank currently has at least 150 RCTs across its regions and sectors. The nonprofit Innovations for Poverty Action has done 400 RCTs in 51 countries.

A randomized trial IPA conducted in Kenya found that school-based deworming reduces absenteeism by 25 percent. With total cost per child amounting to less than 50 cents per year, deworming was deemed one of the most cost-effective solutions to improve school attendance. As a result of this research and the deworming efforts of IPA’s partner Evidence Action, more than 35 million children are dewormed every year.

“Deworming turned out to be very effective in bringing children to school. It is an example of how RCTs can come up with a result nobody expected,” said Rachel Glennerster, executive director of the Abdul Latif Jameel Poverty Action Lab, or J-PAL, a research center at the Economics Department at the Massachusetts Institute of Technology that uses randomized evaluations to answer critical policy questions in the fight against poverty.

But while RCTs may prove successful in measuring the impact of specific interventions, critics point out the tool has limited value when it comes to broader issues.