Strange Football: Education Programs Rarely Keep Score

“Is learning the only result worth financing in education?”

That was the question posed to me at a recent World Bank debate about results-based financing in education. The question is germane because the World Bank has a large program of results-based financing in health and a new modality of Program for Results lending operations, and it is negotiating a new trust fund for performance programs in education. 

Nevertheless, the question is a little bizarre since no agency is contemplating programs that would finance only learning. What I posed instead as a truly relevant and challenging question was the opposite extreme: “Why don’t we ever finance learning in education?” As strange as it sounds, aid agencies rarely, if ever, “finance learning.” I likened education aid programs to a strange football team that tracks players hired, time spent training, and time controlling the ball … but fails to pay attention to goals and wins.

The first thing to keep in mind is that World Bank programs (and those of most aid agencies and development banks) set up systems to transfer money to governments on the basis of something — typically for planned activities or inputs. So this part of the debate is not whether teachers or schools should be paid in relation to student learning but whether funding for governments should vary with improvements in learning.

The second thing to consider is how frequently debates over national education policies fail to engage on learning. When I was in Malawi in 2009, we heard a glowing presentation from the government and funders about a recently completed education sector review. It concluded the country was implementing its education strategy successfully. However, no one openly discussed the decline in enrollment over the previous two years or asked whether student learning had improved or worsened. Malawi is not unique. Education policy discussions regularly emphasize the importance of learning but rarely engage over trends in basic measures like student literacy and numeracy.

Of the World Bank projects that were presented at last week’s debate, one of them monitors whether the government’s Annual Review “reports comprehensively on … outcomes.” But there is nothing in the implementation performance report that would tell a finance minister or World Bank manager whether children who leave the program are better prepared for the next level of schooling than before. Another project tracked whether or not student achievement tests were administered but, again, had no high-level reporting on scores.

The only project presented at the debate that linked disbursements to learning measures was a recently approved project in Tanzania. The implementation status on that project tracks whether the country “has demonstrated an increase in student learning outcomes” — defined in terms of literacy and numeracy measures. The scores haven’t been reported yet. That awaits validation of the test and its application later in the program. But, as far as I know, this is the first education program at the World Bank that will draw attention of high-level policymakers to learning as a critical outcome of the education system. It should become the norm.

The World Bank’s experimentation with Program for Results modalities and linking disbursements to indicators is a welcome one. It fits with an incipient trend among aid agencies to improve management and implementation of jointly programmed operations by clarifying and focusing on milestones. But unless either (1) some of those indicators include public dissemination of test results or (2) a portion of funding is tied to changes in learning outcomes themselves — as done in the Tanzania program — aid agencies and banks will continue to finance education procedures rather than student learning.

In other words, they’ll continue to play the education program game … without knowing the score.