Many learning assessments only evaluate children of a given age (e.g. PISA for age 15) or grade. This approach gives a snapshot that can be compared across countries and produces differentials in learning across 15 year olds (e.g. by household or school characteristics). However, it only implicitly reveals the dynamics of learning as children move from grade to grade. There are still few assessments that test all children in a given age range and hence are able to directly produce a “learning profile” that shows the association between competencies and grade completed.
In their new CGD Working Paper Asadullah and Chaudhury (2013) show the results of giving an assessment of oral and written mathematics to a large sample of children aged 10 to 18 in rural Bangladesh. The use of oral questions allows them to test for understanding of simple arithmetic independent of reading ability. Each of the oral and written assessments had just four questions, which were simple and related to everyday life. For instance, two of the oral questions were:
“Suppose you save Taka 20 each month. How much will you have saved after six months?”
“Suppose you have Taka 250 in total and a chicken costs Taka 60. How many chickens can you buy? How much money will be left after the purchase?”
In line with earlier assessments in Bangladesh, (Greaney et al 1998) they define “competence” in mathematics as answering three or four of the questions correctly. They estimate the association between competence on either the oral or written mathematics assessment and the year of schooling each child had completed--while controlling statistically for a child’s gender, age (and square), parent’s schooling, household income, and a child’s Raven’s score (a measure of cognitive ability).
Figure 1, which is my adaptation of the regression results in Table 2 of Asadullah and Chaudhury’s paper, shows the predicted percent competence on oral mathematics for each grade completed.[1] The results are stunning. A child who had completed grade 5 is only 11.7 percent more likely to be competent (51.7 versus 40.2) than a child with no schooling at all (after statistically controlling for other factors).
One interpretation of that statistical fact is that only 1 in 9 children actually learned mathematics in 5 full years of schooling--as 40 percent could do it without the five years of schooling, 48 still couldn’t do it even with the five years of schooling, so for only 12 percent of children did five years of primary school make the difference between having or not having this level of competence in mathematics.
Put another way, the average gain in “competence” between grade 1 and grade 8 is just 4 percentage points per year ((67.0-38.8)/7=4.02). So in a classroom of 25 children only about 1 child per year would reach this very basic level of competence.
Figure 1: The learning profile is very flat—children who just complete primary school (grade 5) are only 12 percent more likely to be competent in an oral assessment of mathematics than those with no schooling at all
Source: Author’s adaptation of Asadullah and Chaudhury (2013) Table 2 (oral mathematics, pooled (male and female), column 2).
To be fair, the results are modestly better for the written mathematics assessment (Figure 2), perhaps because it combines an assessment of some reading and ability to do math. Those with no schooling, even if they can do arithmetic learned from experience, struggle with a written test. But still, the gain per grade completed is only 5.4 percent—only about one child in 20 gains competence per year of instruction.
Figure 2: If the assessment of mathematics is written then children who just complete primary school (grade 5) are still only 20 percent more likely (47.6 vs 27.1) to be competent than those with no schooling
Source: Author’s adaptation of Asadullah and Chaudhury (2013) Table 2 (written mathematics, pooled (male and female), column 2).
As distressing as they are, these simple grade learning profiles are likely to overstate the actual learning from year to year as, especially after grade 5, more and more children drop-out (Figure 3). If children with greater competency are less likely to drop out (as seems plausible) then the increase in competence of those who complete grade 8 , could be due to the fact that those with lower competence did not complete grade 8, so the increase in competency between year 7 and year 8 could easily overstate the learning gains. Hence the learning profile may appear to be steeper in the later years not because of more learning but just because of more learning selective drop-out.
Figure 3: Attainment profile of a recent cohort shows accelerating drop-out after grade 5—especially among poorer children (from lowest 40 percent of households by assets) among whom only 20 percent reach grade 9 or higher
These results from Bangladesh add to a growing body of evidence about flat learning profiles. Results from the ASER in India and Pakistan and UWEZO in East Africa show slow progress by grade on a simple literacy and mathematics instruments. The LEAPS study in Pakistan has also tracked specific children and can examine the learning gain child by child. Recently, Karthik Muralidharan has been able to link children from Grades 1 to 5 on a common vertically scaled IRT (Item Response Theory) scale in Andhra Pradesh, India. The Young Lives study is also beginning to be able to trace out learning trajectories across its four study countries.
Unfortunately, this new study adds Bangladesh to the list of countries with evidence from learning profiles indicating a serious learning crisis.
[1] This level is set to the average percent competency of those who had “some secondary” or “secondary completed” percent from Table 1.
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
