Schooling Ain't Learning, and Learning Assessments (Alone) Ain't Reform

A new report examining independent learning assessments in developing countries shows that while they produce robust measures to date they have done little to improve the quality of learning. Growing awareness of the sorry state of education is necessary, but it is far from sufficient to spark change.

In Tanzania in 2008, more than 80 percent of surveyed adults suggested that the government was meeting the country’s educational needs fairly well or very well.  Two years later, nationwide tests by the civil society group Uwezo suggested that less than a third of kids in Standard Four passed Standard Two-level tests on of literacy and numeracy.  The disconnect between low quality and high satisfaction might be one important reason that broken school systems are allowed to fester not just in Tanzania but across the developing world, churning out millions of students each year who have learned a fraction of the syllabus — if that.

Efforts like Uwezo were designed to help end the disconnect — creating pressure for reform by measuring and publicizing the schooling-learning gap. The new report, commissioned by the Hewlett Foundation and carried out by Results for Development (R4D), studies the experience of learning assessments in nine countries, including the very first large-scale effort, India’s Annual Status of Education Report (ASER) and Uwezo, (which covers Kenya and Uganda in addition to Tanzania). These initiatives conduct assessments of literacy and numeracy in a nationally (and often subnationally) representative set of households. The number of kids involved varies from 15,000 in Senegal to more than 500,000 in India. And ASER and Uwezo’s efforts both involve over 20,000 volunteer testers. R4D suggests that the assessment efforts are technically strong (if noting that more attention should be paid to the comparability of results over time). That suggests their picture of poor learning outcomes is depressingly accurate. In India, for example, nearly one-half of grade 5 students cannot read a grade 2 text and one-fifth cannot follow a grade 1 text.

The report suggests that the older initiatives like ASER and Uwezo have helped spark national debate and led to the prioritization of learning results in education policy documents but, suggests R4D, “generating concrete action to improve learning outcomes on the part of key stakeholders has proven much more challenging for both initiatives.” One piece of evidence: ASER survey data between 2006 to 2012 show that more states are declining in student performance than are improving. 

An explanation for the assessment-reform gap is suggested by some of the quotes in the report. Policymakers in ministries knew that few kids were learning in schools but didn’t care or couldn’t do anything about it. A senior Ministry of Human Resource Development official from India told the report authors: “Government always knew that learning levels are poor in public schools. We did not need ASER to tell us this fact which to us has always been self-evident.”

Given that, the report argues that the assessment efforts might have bigger impact if they were able to reach out beyond national capitals. R4D authors suggest assessment groups could extend the role of the volunteers who carry out the learning tests alongside partner organizations involved in education to act as champions in disseminating results and fomenting change at the local level, not least through creating a dialogue between parents, teachers and headmasters. Certainly someone should be doing that, it might depend on the particular circumstances of the country and the organizations involved if it should be the group that runs the assessments. Research and advocacy on policy are two different things — sometimes it still works to have them under one roof; other times it makes sense to have the research independent from the advocacy effort. 

Nonetheless, the R4D report bolsters the conclusions of CGD’s study group report Schooling Is Not Education! Using Assessment to Change the Politics of Non-Learning. The study group emphasized that assessments were no silver bullet and worked only as part of a larger effort. Experience from countries including Brazil and Chile suggest schooling reform needs to be a national priority and parents, teachers, and headmasters have to be empowered with the knowledge and freedom to act on assessment results. And given ASER's Rukmini Banerjee and Uwezo's Rakesh Rajani (now at the Ford Foundation) were both members of the study group, doubtless both organizations will agree with the thrust of R4D’s analysis.

The learning crisis needs systemic reform and our knowledge base of what works is limited (that is the subject of Justin Sandefur and Lant Pritchett’s Research on Improving Systems of Education effort). Assessments are the vital first step in learning the scale of the problem and finding out what works to help fix it. And the national and international attention that the efforts of groups like ASER and Uwezo have brought to the learning crisis has been invaluable. If they — or others — can help empower the legions of volunteers involved to act as agents of change, they might help foment an education revolution.