Note: This blog post is part of a series in which CGD experts present arguments from “Schooling for All: Feasible Strategies to Achieve Universal Education” and invite (sometimes dissenting) commentary from experts within the global education community.
In the past 50 years, the developing world has witnessed dramatic improvement in school enrollment and learning outcomes. In chapter 1 of our new CGD report on “Schooling for All: Feasible Strategies to Achieve Universal Education”, we argue that this historical success should inform countries' forward-looking education policy agenda—especially in places where some of the core policies that did so much good in the 1990s and 2000s, like abolishing fees and building schools in hard to reach areas, have not been fully implemented to date.
We argue for prioritizing not just “what works”, but “what scales”
The historically proven route to increase learning has been to increase schooling. While countries have struggled to improve school quality, the massive increase in educational access has also delivered a globally unprecedented increase in learning—with literacy rates in low- and lower-middle-income countries now higher than a half century ago by double-digit percentage points.
Randomized trials have shown how to do this: subsidize access, take care of kids' health and nutrition while in school, and don't expect huge gains from reducing class sizes or buying new books. A 2013 summary of this literature published in Science by recent Nobel-laureate Michael Kremer, Conner Brannen, and Rachel Glennerster has held up quite well. The abstract is worth quoting in full:
"Across many different contexts, randomized evaluations find that school participation is sensitive to costs: Reducing out-of-pocket costs, merit scholarships, and conditional cash transfers all increase schooling. Addressing child health and providing information on how earnings rise with education can increase schooling even more cost-effectively. However, among those in school, test scores are remarkably low and unresponsive to more-of-the-same inputs, such as hiring additional teachers, buying more textbooks, or providing flexible grants. In contrast, pedagogical reforms that match teaching to students’ learning levels are highly cost effective at increasing learning, as are reforms that improve accountability and incentives, such as local hiring of teachers on short-term contracts. Technology could potentially improve pedagogy and accountability. Improving pre- and post-primary education are major future challenges."
Taking this list on board, we propose applying one additional filter to this list, in addition to effectiveness and affordability: scalability.
Some items on this menu have proven technically and politically difficult for developing countries to scale up in public school systems. In the decade since the end of the trials summarized by Kremer et al (2013), development economists have increasingly recognized the difficulty of translating pilot projects into successful national policies. For instance, early grade reading programs or home visits, while successful at trial stage, have often been abandoned or seen their impacts wane as they were taken to scale. These are labeled “Type A” policies in the figure below. But not everything falls apart when scaled up.
Figure 1. Which interventions scale?
The somewhat intuitive policy upshot of our argument is that new public spending will do the most good where money is the binding constraint. Meta-analysis of impact evaluations shows that several categories of policies—which we characterize broadly as less easier-to-implement, more logistical interventions, labeled “Type B” in the figure—show impacts that are more robust in large government-run programs. This includes things like school meals and extending the length of the school day. The core argument of chapter 1 is that if countries are able to spend more on education in the short to medium term, these things should be the first priority.
Rukmini Banerji offers the perspective of someone who has tried to tackle the hardest problems in education reform head on
In her commentary, Rukmini Banerji (CEO of Pratham Education Foundation) insists that a change in mindset, not just money, is required across various actors at different levels of the system. Unless people buy into the idea of adjusting the curriculum to the level of the pupil and defining success in terms of learning outcomes, programmatic solutions are doomed to fail.
She points out that the success her organization, Pratham, has had in improving learning has come from an iterative and long-term collaboration with researchers, of trial and error, adjustment and adaptation rather than a single RCT. And she suggests that generating evidence that meets the rigorous standards in our chapter is a costly and often western-led endeavor and that this may limit the number of interventions that have such evidence at scale.
Read Banerji’s full comment.
Comment: The pivot from every child in school to every child learning requires a new mind-set
In the last two decades or more, low-income countries, especially in Asia and Africa, have made tremendous progress in bringing children to school. In almost all countries of the Global South, over time, at least until COVID-19, the average number of years of schooling for a student has been rising. The real challenge now is how to translate years of schooling into years of effective learning.
In the efforts to universalize schooling and ensure learning for all, there have been a wide spectrum of reform initiatives. Looking across these innovations and experiences, what have we learned about what can be scaled up? Why do some scale-ups succeed and why do some scale-ups fail?
But before grappling with the question of what works at scale, it is worth reminding ourselves that education systems are in the middle of a significant shift in priorities. Although many countries know what to do to bring every child into school, every child learning well has been much harder to do. Broadly speaking, expenditure on inputs like school buildings, other infrastructure, teachers, and textbooks works well for increasing access. However, increased learning outcomes are not as straightforward to implement.
The question of children’s learning can be roughly thought of in two categories: first, what is to be done in the early years of schooling (for example, grades 1 and 2) so that children can acquire basic learning skills like reading and basic arithmetic early in their educational career? Second, how do we deal with the fact that a majority of children in grades 3 and above are well below expected levels? For those children, foundational skills have to be built before moving to what is expected of them in each grade as per curricular expectations.
Let us take the case of India as an illustration. Only in 2020, with the launch of the New Education Policy, was it acknowledged that many children enter formal schooling with a disadvantage. If a child has not been exposed to any preschool or early childhood education, then she or he arrives in grade 1 without adequate readiness for school. Often, the curriculum for grades 1 and 2 focuses on academic competencies without taking into consideration a breadth of skills that are needed for getting a child ready for school and ready to learn. NIPUN Bharat, or the new Foundational Literacy and Numeracy (FLN) mission, is focusing on how to rework teaching/learning activities in these early grades so that children can have a good start to learning. Moves are also afoot to better link early childhood programs to school initiatives in the early years. In the next few years, as each cohort moves through the school system, we will be able to assess whether these efforts are bearing fruit.
What about the “learning deficit” for children who have been in school for several years but have not acquired the basics? This issue is much harder to tackle. Until the Annual Status of Education Report (ASER) started in 2005, there were no simple assessments or learning outcome data for primary school children that focused on foundational skills. ASER data pointed to several features of the education system in India—children’s learning levels were low (a majority were several years behind where they should be), there was wide variation in learning levels in the same grade (whether in the same class or across states), and over time learning trajectories were flat. Acknowledging that there indeed was a “learning crisis” was tantamount to accepting that “business as usual” had not worked effectively for most children until now.
Why were children not able to reach grade-level expectations? Several explanations are possible—one set of explanations included possibilities such as teachers not equipped to teach effectively, children not ready for learning, and families not able to support children’s learning. But another explanation would point to the curriculum. Can the situation that we see today be interpreted as a “negative consequence of overambitious curriculum” or a result of “teaching to the top of the class”? If so, any effort to improve learning outcomes must focus not on an age-grade syllabus but on Teaching at the Right Level (TaRL). In fact, it could be the one-size-fits-all curriculum often prescribed for the entire country that is the real problem.
Coming back to the difficulties of embedding TaRL into government systems in India, it could be the case that it has been a failure of changing mind-sets (away from syllabus). Even after a large-scale TaRL effort is undertaken, the system returns quickly to its age-grade curriculum as an anchor or as a security blanket. Undoubtedly, implementation could be and can be better on many counts, but if the actual problem has not been understood properly and accepted, initiatives such as TaRL cannot be successful.
Moving away from the specific case of TaRL or the context of India, there are several other points to think about in the relationship between research and evidence on the one hand and policy and practice on the other. Context and conditions matter for any implementation, but so do the starting point and who fuels the change. Did the original experiment start small (and was it designed for that scale)? Later, after a successful demonstration, was there an effort to scale it up? What would happen if the original scale were large and with each round of implementation, improvements and adjustments were made? In low-income countries and within centralized education systems, is there internal experimentation capability for iteration? Over time are such countries investing in adaptive capability? Or will the evidence and innovations continue to come from researchers and consultants and donor agencies?
A final comment: The appropriate methodology for meeting acceptable standards of rigorous evidence is costly, and the skills for carrying out such studies often lie with researchers in western countries. Do both of those conditions constrain the scale at which evidence can be generated? Are alternative methods being developed that can study a system at scale and provide key inputs for improvement? For Pratham and TaRL in India, we have benefited hugely from a series of randomized controlled trials done in partnership with J-PAL over two decades. These studies were layered onto a long-run effort for improving basic learning outcomes that continuously tweaked and adapted different features of the approach and implemented it in many different conditions and contexts. Instead of being seen as a one-off research study that may provide input for a large-scale implementation, the ongoing efforts—research and implementation—together represent an ongoing endeavor to improve children’s futures. Perhaps that is the missing element for strengthening scaling up initiatives.
Moses Oketch argues not to give up on the hard stuff
The crucial task of improving pedagogical practices is less about money. In his commentary, Moses Oketch (Professor of International Education Policy and Development at University College London’s Institute of Education) makes the case not to abandon the more difficult reforms, as governments pivot from a focus on schooling to school quality. By definition, that new direction will require new tools. But both Oketch and Rukmini Banerji note that while spending money has worked well to improve access, it’s less obvious that money can fix the pedagogical failures that hold back learning for many kids who are already in school.
In sum, Oketch notes correctly (in our view) that a focus on what has been feasible to scale in low-income settings pushes the agenda away from new initiatives to improve test scores for kids already in school while they’re in primary grades, toward expanding access and time in school. Oketch makes a plea to retain both on the agenda.
Perhaps our only quibble there would to emphasize again that only for the easy, scalable stuff like school meals or abolishing user fees, is money really the binding constraint to progress, which makes these items a good place to start for a short-term budget request.
Read Oketch's full comment.
Comment: Scalability cannot be the sole criterion for policy decision making
Crawfurd, Hares, and Sandefur raise several important issues related to what works at scale in terms of education policy and interventions. My comments here focus on two key issues, of relevance to low- and lower-middle-income countries: (1) the need for renewed attention to the unfinished business of access and (2) the complexity of scaling pedagogic reforms shown to work in the pilot phase.
First, concerning the need for renewed attention to the unfinished business of access. The focus on learning instead of schooling is central to realizing Sustainable Development Goal 4. Without learning, there is little point in going to school; without children going to school, it is nearly impossible to organize formal learning at scale. Parents expect schooling to lead to learning. However, the “learning crisis” changes the terms of the debate since it is largely presented and interpreted as an argument and movement that is against continued expansion in enrollment and attainment. As Pritchett and Sandefur (2020) have noted, it will require both universal schooling and a dramatic improvement in learning profiles to achieve the SDG targets. However, dramatic improvement in learning is not going to happen easily and analysis comparing countries’ learning profiles does not show decisively what works, but the clear message in the learning crisis movement is that schooling itself without evidence that it also generates sufficient learning is not good enough. In order to dramatically improve learning, Crouch, Kaffenberger, and Savage (2021) have argued that there is a need to focus on systems improvement, and to use foundational learning as the guiding principle to ratchet up learning. However, Crawfurd, Hares, and Sandefur challenge that view, at least in low- and lower-middle-income countries with space to expand access. They note that pedagogy reforms can be difficult to implement and there is not enough evidence of their scalability in government school systems. Expecting imperfect government systems to dramatically improve learning has proven to be a tall order. So, they argue that expanding access, providing school meals, and extending school time are policies that have worked at scale, and focusing on these is an actionable agenda that can raise learning outcomes in the near time. Their main message is that education pays even where schooling does not generate stellar learning outcomes; therefore, from an economic standpoint, even rudimentary schooling may be a very good investment. The main theme of their argument is that scalability of a program has to be critical when interventions are presented to a minister of finance. While this is a reasonable argument, it is also very narrow. I would argue that scalability cannot be the sole criterion for policy decision making, although it may be one of the considerations.
Second, concerning the complexity of pedagogic reforms and scalability, there is general agreement that learning needs to improve dramatically in low- and lower-middle-income countries, and even Crawfurd, Hares, and Sandefur do not argue against this. Most commonly for low- and lower-middle-income countries, there is agreement that something must be done about the learning crisis. The success in schooling should not be allowed to go to waste. Thus, I would argue that the choice of Type A policies—those that are more effective when well implemented but require highly skilled staff (these policies include structured pedagogy, teacher coaching, or home visits for early child development)—versus Type B policies—those that might be less effective when implemented in terms of improving learning but are more robust to weak implementation (these include school-building, lengthening the school day, or providing school meals)—is a false choice. Instead, both Type A and Type B policies are needed in a country, as addressing the learning crisis requires education system improvement. I have argued elsewhere (Oketch 2019), as Crawfurd, Hares, and Sandefur do in their chapter, that measures of performance, efficiency, and effectiveness often embedded and dominant in randomized control trial (RCT) studies do not provide explanations of how and why an education system “is where it is” or of “what works” to improve it. But, I would also argue that RCT-based studies may offer some insight at small scale on potential mechanisms of change, which can help to identify where the “blockage” lies at the macro level, even when these pilots have not proven scalable. So, while scalability may be a useful criterion for policies and indeed critical, there are still many systems-improving lessons that can be learnt from pilots and projects that haven’t proven scalable. What I would argue against, and where I agree with Crawfurd, Hares, and Sandefur’s argument, is that RCTs and pilot projects that have demonstrated success should not crowd out those programs that have already shown success when rolled out at scale, and a gradualist approach rather than a big bang rollout of pedagogy reforms might be a wise approach to present to a minister of finance. It is not obvious that systems that have improved learning have done so through relying on pilots, but they have certainly learnt gradually or in an evolutionary manner.
References
Crouch, L., M. Kaffenberger, and L. Savage. 2021. “Using Learning Profiles to Inform Education Priorities: An Editors’ Overview of the Special Issue.” International Journal of Educational Development 86, 102477.
Oketch, M. 2019. “Randomized Controlled Trials: Limitations for Explaining and Improving Learning Outcomes.” In World Yearbook of Education 2019 Comparative Methodology in the Era of Big Data and Global Networks, edited by R. Gorur, S. Sellar, and G. Steiner-Khamsi. London: Routledge.
Pritchett, L., and J. Sandefur. 2020. “Girls’ Schooling and Women’s Literacy: Schooling Targets Alone Won’t Reach Learning Goals.” International Journal of Educational Development 78, 102242.
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
