30+ New Findings from the RISE Conference 2022 on Global Education: The Politics of Education Reform and the Interventions You Want Reforms to Put in Place

Last week was the annual conference for the Research on Improving Systems of Education (or RISE) program, which I always find to be an energizing gathering of people working in global education. Participants learned from research on teachers, student learning outcomes, the political economy of education reforms, the impact of COVID, and more.

Here’s a round-up of what I learned. As always, I’ve highlighted one or two of my favorite findings from each paper, which isn’t necessarily the primary result of the paper. If you want more, you can read the whole program, and you can also watch videos of the whole conference.

Teachers

Can we better predict who will be an effective teacher? Admission criteria (like undergraduate grades, admission test scores, and interview scores) for a teacher training program in Indonesia did predict performance in the program, but there was “no evidence that the selection criteria predicted [subsequent] student learning in a meaningful way.” (Berkhout, with Yusrina, Bima, and Suryadama*; paper; key slides)

“Many teachers [in India and Bangladesh] underestimate the share of low performers in their classrooms, and that they believe that those students will perform better than they actually do. These results are not driven by less educated, trained, or experienced teachers or explained by biases against female, low-income, or lower caste students.” (Djaker, with Ganimian and Sabarwal; paper; key slides)

What drives changes in teachers’ classroom practice? In Zambia, researchers asked teachers what had driven those changes and where those catalysts took place. Primary drivers of change included “sharing and discussing challenges” with colleagues (which often takes place at school) and “acquisition of new skills and teaching methods” (which often doesn’t), potentially suggesting value for teacher professional development that combines in-school and centralized training (Mathenge and de Barros, with Henry; paper; key slides)

What determines what teachers do? Activities at the intersection of (a) what teachers themselves value, (b) what is seen as feasible in a given situation, and (c) what teachers believe those who are “in charge” expect. Changing teacher norms means pushing on one or more of these levers. (Yue-Yi Hwa; key slides; video of presentation)

Learning

Has equity in education improved over time? This is more complicated to measure than you might think: is greater equity reflected by more children having access to education? Or by more students reading by age 10? Or by an increased focus on disadvantaged children? In Ethiopia over the last decade, equity of access has improved and students have access to better teachers across several measures, but learning outcomes have dropped for most groups. (Rolleston, with Oketch, Rossiter, and Tiruneh; key slide)

In Vietnam, parents are able to accurately identify which teachers are most effective in teaching their students. (Guha, with Carneiro, Giannola, and Krutikova; paper; key slides)

Measures of schooling, cognitive ability, and socio-emotional skills in Pakistan don’t explain much of people’s earnings from work (just 13 percent for men and 26 percent for women). But still, the returns to cognitive skills are high: one standard deviation higher cognitive skills are associated with $800 higher discounted lifetime earnings. (But be careful: this is just a correlation.) (Danon, with Das, Filmer, and de Barros; video of presentation)

Free after school tutoring in rural Bangladesh boosted English and math scores for beneficiary kids but it also led to big positive impacts on untutored classmates. Targeting tutoring to students who are more central to social networks leads to more than double the size of the impacts on non-tutored peers. (Vlassopoulos, with Islam, Zenou, and Zhang; paper; key slides)

Positive and negative deviance

With programs to improve reading in the early grades, are most of the effects driven by a few schools with big improvements? Across programs, between 14 percent and 42 percent of school explain 80 percent of the programs’ impact on learning. Interviews in Nepal suggest that there are some early implementers of the program who resist negative social norms to achieve high impact, while many teachers “applied the program content using ineffective pedagogies,” even as observers gave those teachers “passing grades for observable compliance measures, to the detriment of quality instruction focused on student learning.” (King, with Betts, Rodd, and Neupane; paper)

Despite myriad challenges in secondary schools in Sierra Leone, some schools still deliver positive student performance. “In every single better-performing school we visited, the main driving factor contributing to quality learning was inevitably a strong leader (i.e., principal) who manages and leads the … school.” (De, with Ofori-Owusu, Hudda, and Olisenekwu; paper)

What characteristics of teachers predict their effectiveness? Machine learning analysis in Tanzania suggests that “teacher beliefs that they can help disadvantaged and struggling students learn (for Math) …, teacher practice of providing written feedback and reviewing key concepts at the end of class (for Math), spending extra time with struggling students (for Kiswahili), [and] teacher preparation on how to teach foundational topics (for both Math and Kiswahili) … are highly predictive of teacher effectiveness.” (Nahata, with Sabarwal and Filmer; paper)

Analysis from three states in India indicates that “there is a positive and significant premium of attending a preschool before starting primary school.” But “the entire effect is driven by children who attend private preschools… A descriptive study of the preschool quality by management type showed that private preschools have lower student-teacher ratios, longer hours of operation and a focus on formal instructional style of teaching.” (Gupta**; paper) Interestingly, this contrasts with other work – e.g., Wolf and others in Ghana – that training kindergarten teachers in more play-based instruction can boost school readiness. But the findings from Wolf’s paper that parents don’t like the play-based instruction suggests that parents would love these private Indian kindergartens.

What do good and bad practice look like at the level above the school (i.e., the district educational leadership)? In Ghana, a high-performing district had clear targets and a monitoring system for low-performing schools, whereas a poorly performing district lacked any specific strategy for improvement. Both districts benefit from strong international donor support. (Asim and Bell; video of presentation)

Long-term trajectories of change

A free primary education program in western Nigeria in the 1950s increased education—especially for women—and some of these education benefits spilled over into other parts of Nigeria, but not to neighboring Benin. (Okoye, with Wantchekon; paper)

Over the course of half a century around the world, many more people are in school, meaning that many more people are able to read. But education quality has taken a hit: “literacy conditional on completing five years of schooling, stagnated or declined across the developing world over half a century, with pronounced drops in both South Asia and sub-Saharan Africa.” (Sandefur, with Le Nestour and Moscoviz; paper; key slide)

What will the Gates Foundation’s global education program be focusing on going forward? (1) “Increased focus on improving learning outcomes, with a particular emphasis on responding to learning loss in the short term.” (2) “Evidence on what works to improve outcomes…: structured pedagogy, teaching at the right level, remediation.” (3) “Research focus…: measurement and comparability of [SDG indicators] 4.1.1a and 4.1.1b remains important, focus on research that can help respond to implementation challenges, consider scale and systems, [with an] ongoing eye on the typical teacher in a rural school.” (Piper; video of presentation; key slide)

What drives educational system performance? A new conceptual framework suggests that “commitment to the purpose of learning is a critical missing link to addressing the learning crisis.” Many education systems have other purposes, whether an ongoing commitment to educating elites or even infighting about purposes that disallows a focus. (Kaffenberger, with Pritchett; video of presentation)

Political economy

Over 25 years, Peru has undertaken a series of major educational reforms (while passing through 20 different ministers of education). Learning-oriented forms gained ground in the 1990s, lost it in the early 2000s, gained major ground again between 2006 and 2016, and 2016 onward has seen “extreme instability and progressing dismantling of reforms.” Technocrats need to do enough politics to get their policies to stick around. (Balarin; paper; key slides)

Between 2000 and 2018, Peru gained major ground in the international student assessment called PISA (as well as increases in enrollment and completion). This required a series of major teacher reforms, with a focus on learning and additional financial resources. The government achieved these reforms through a central bargain (big salary increases in exchange for more selective hiring and less job stability), together with constant outreach to the media, a careful focus on implementation, highly visible investments and improvements—like school infrastructure and a longer school day, and more. (Bruns, with Schneider and Saavedra; key slides)

Why do communities in northern Nigeria support Islamic and Quranic schools? State schools fail to deliver quality education, they promote out-of-reach aspirations (the expectation of formal sector employment), and they promote liberal values around gender, sexuality, authority, etc.; values which directly clash with communities’ religious or moral preferences. (Bano; key slides)

What does it take for countries to reach the “sweet spot” that enables learning-focused education reform? When their education systems are aligned around learning and when different elements of the system are coherent with each other. Outside of that sweet spot, many technical initiatives will be ineffective. But it’s still possible to improve learning through sharing ideas and norms! (Levy; key slides)

Alternative modes of provision

“Why has private education expanded so rapidly yet so unevenly” in low- and middle-income countries? Politicians in places with few public resources to expand public education may support the expansion of private education in the face of electoral competition as a way of providing visible development progress to voters. Three decades of data from India supports the hypothesis. (Read**; paper)

Schools run through public-private partnerships in Sindh, Pakistan have higher accountability, monitoring, and evaluation, but no better access, overall quality, or equity. (Rind, with Shah; paper)

Next were three papers on India’s 2010 Right to Education Act, which “imposed a quota of 25 percent of incoming enrollment in private schools for students from disadvantaged backgrounds” (quote from Singh’s paper below).

“How do children attending private schools within the context of [India’s Right to Education] act characterize their everyday experiences?” Understanding home environments is crucial to understanding how schooling fits into children’s lives. Children’s school experiences weren’t “explicitly experienced as gender or caste-based,” but with “pervasive and wide-spread labelling of children as ‘naughty’ or academically ‘weak.’” (Srivastava, with Lafleur; video of presentation)

India’s Right to Education Act has been implemented in only 11 of 28 states. Most states use an online portal for admissions, but they vary dramatically in how it works. (Wad, with Sarin; video of presentation)

In Chhattisgarh, India, “within eligible caste groups, quota applicants are drawn disproportionately from more-educated and economically better-off households and over three-quarters of the applicants who were not allotted a quota seat also attended a private school as fee-paying students.” (Singh, with Romero; paper)

COVID

After two years of school shutdowns, 40 studies across 27 countries show that learning loss has been much worse for the poorest children—both in low- and high-income countries; and some countries—like Kenya, South Africa, Brazil, and Malawi—have seen big increases in adolescent school dropout rates. (Evans, with Moscoviz; paper; summary blog; twitter thread)

In Tamil Nadu, India, students learned much less than they would have while schools were closed, but after schools opened, two-thirds of the gap closed in just four months. There was a massive remedial education campaign after schools re-opened, although we don’t know how much of the learning recovery was due to the campaign versus how much would have happened regardless. (Singh; video of presentation; key slides)

Long school closures in Uganda translated to greater parental involvement in education: a survey of parents before the pandemic indicated little parental involvement, but a survey in mid-2021 suggested a large minority of parents attending radio lessons with their children or supervising their home study. But some bad news: after schools re-opened, "24 percent of households reported at least one child did not return to school." (Atuhurra, with Winter and Nishimura; paper; key slides)

In Côte d’Ivoire, a “lack of motivation was the most frequently cited reason why children weren’t spending more time on education while schools were closed,” especially among the poorest kids. One in five parents “reported that they engaged a private tutor…to support children’s education while schools were closed.” Hiring private tutors may not be a COVID phenomenon; this is potentially very prevalent when schools are open as well. (Kabay; video of presentation)

Accountability

“Teacher absenteeism [in India] decreases substantially in the year before an election and higher in the year after.” The policy question is: “How to extend these incentives over the entire electoral cycle?” (Davies; paper)

The “Inspect and Improve” pilot in Ugandan secondary schools, intended to improve leadership and management, puts school leaders in charge of school improvement but with continuous support. Initial evaluation evidence suggests that “improvements to leadership and management practice can positively affect other areas of school operations (attendance, teaching practices, student safety and well-being).” (Kotonya and Kyasanga, with Horn; brief; video of presentation)

A social accountability intervention in Indonesian schools included consensus building among village-level stakeholders and—in some communities, “teacher pay based on camera-monitored teacher attendance.” It delivered big impacts, and additional analysis shows that the cameras made it easier for community members to hold teachers accountable, both by increasing the social legitimacy of community members holding teachers accountable and in increase community members’ technical ability to do it. (Susanti, with Hwa, Lumbanraja, and Riyanto; paper; key slides)

How can we evaluate education systems’ coherence around learning as a goal? Here are three ways: (a) use learning trajectories (i.e., how much kids learn from year to year), (b) see how well systems meet four principles (such as having clear goals for children’s learning progress, aligning instruction to be coherent with current learning levels and the learning goals, and providing effective support to teachers), and (c) examine key relationships in education systems (like politics, management, and the link between the highest executive authorities and those who provide education services) as well as features of each (like the financial resources and data they have available). (Silberstein, with Kaffenberger and Spivack; paper; key slides)

The Panels

While I can’t do justice to the panels and discussions I saw, I will share a few takeaways that struck me.

• What’s the most productive role for external researchers to support educational work in other countries? “They must be willing to give advice with whatever knowledge exists at that moment.” (Jaime Saavedra, Global Director for Education at the World Bank)
• What do we learn from positive deviance studies? “In very complex situations where you don’t necessarily know what to do, looking at positive deviance can for instance tell you what you ought to be doing a randomized control trial [of]. It’s really about discovery of potential ideas in very complex situations.” (Alec Gershberg, Penn GSE)
• How do you encourage action based on research? “Nobody is ever going to do anything because this paper said to do it or because RISE recommended it. They’re going to do it because everybody knows it’s the right thing to do. You have to really diffuse the understanding of the rationale very well.” (Lant Pritchett, Research Director, RISE program)
• Which teachers should interventions target? “The teacher that matters in my point of view is the typical teacher in the rural school, who is unhappy about being a teacher… That teacher who is moderately engaged, moderately interested. That’s the teacher to target. As researchers… we have typically targeted [a high-performing] teacher when we should be targeting [a typical] teacher. So if you target this [high-performing teacher], your program is too complicated and too complex, and far too few teachers do it… Fundamentally you want this moderately engaged rural teacher who’s at the margin, you want to target the intervention at them. That’s the teacher that matters.” (Benjamin Piper, Director of Global Education, Gates Foundation)
• “My view of the average teachers…is that they’re well-meaning, and if they were better at their job, they might enjoy their job more. So I think giving them structure that’s research based, that’s highly effective—not alone but in a package of other things—will make them better at their job. Kids will learn more and in my experience…that makes them like their job. They’re more motivated.” (Benjamin Piper, Director of Global Education, Gates Foundation)

Notes:
* I’ve listed the presenter(s) at the conference, “with” any other authors listed on the program. Thus, the author order here will often not be the author order on a final publication.
** I summarize a couple of papers that were listed on the program but didn’t end up getting presented (e.g., because authors couldn’t travel or participate remotely).