Prachi Windlass

Assessing ROI in Educational Development: In India, assessment tools are still lacking, but change is coming

At the Michael & Susan Dell Foundation, we’re focused on measurable outcomes. Key areas include children’s learning and school performance, as well as teacher effectiveness – and how to quantify improvements and then replicate them. Though others in the philanthropic world share our belief in the need for robust education assessment tools and approaches in India, the current situation with practical yet scientific assessment solutions is disappointing. There are multiple factors contributing to the issue:

  1. There is a severe lack of comprehensive and reliable assessment tools in India to measure student learning levels. For example, student performance data segmented by specific subject competencies and school systems does not exist. The few existing tools used for such analyses are poorly designed and unreliable.
  2. Teachers lack actionable and student-specific data that can be used inside classrooms to improve their teaching and benefit students. Therefore, they are forced to take a one-size-fits-all approach in classes, which typically consist of children at varying learning levels, sometimes including students in different grades even if they are in the same age group.
  3. Most education organizations have never analyzed their intervention outcomes in a scientific and rigorous fashion. Even when such data can be made available, significant coaching and support is required to help the organizations understand the results and subsequently use it to guide program design.

With a constant eye on the goal of delivering tangible benefits to Indian students, the foundation has leveraged objective, data-driven decisions and a feedback loop to ensure our programs achieve continuous learning improvements. In the absence of any ready-made solutions, it became critical for the foundation to create rigorous assessments not just to measure our own portfolio’s effectiveness, but help other programs improve and scale. Our commitment was reflected in investments in assessment vendors, tools, research studies and nonprofit organizations.

What are the returns on our investment in assessments thus far?

  1. Several education organizations such as Naandi Foundation have been able to significantly improve their learning level outcomes as a result of using the feedback from such third-party assessments.
  2. Objective measurement data has been used as proof for extending successful pilots into the government school system. For example, Bodh Shiksha Samiti conducted a 60-school classroom assessment pilot in Rajasthan in partnership with the state government. Objective, third-party assessments that demonstrated large improvements in student performance were instrumental in the state’s decision to integrate these methodologies in all of Rajasthan’s government schools.
  3. Multi-stakeholder programs such as the Mumbai School Excellence Program have heavily relied on assessment data to drive program design decisions, as well as partner selection.

Today, nearly 200,000 children in approximately 15 foundation-funded education programs are assessed biannually, through third-party assessments. Through our efforts, we have created a robust framework and methodology that can be implemented by other like-minded funders and education programs:

  1. The consolidation of our learnings in creating high-quality assessments and how we have used them in our work resulted in an assessments white paper.
  2. Again, leveraging our measurement framework and methodologies, the foundation sponsored a study to provide insights on establishing benchmarks for students’ learning levels. This research (conducted by Educational Initiatives) was the first of its kind to provide a comprehensive view of students’s grade-specific learning proficiencies across a variety of school systems.

With the growing realization that quality of school education has to significantly improve in India, particularly for underprivileged students, our hope is that this is just the beginning of a wider dialogue on applying a robust assessment framework to education in India. At the foundation, we know the return on our investments in education assessment ecosystem will only continue to grow.

Prachi Windlass is the Lead, India Education at the Michael & Susan Dell Foundation

Categories
Education, Impact Assessment
Tags
impact investing, impact measurement