Educational Attainment in Punjab – Have Learning Outcomes Improved?

 This post is the second in a series discussing findings from the Punjab Education Sector Programme (PESP) II evaluation, a performance evaluation funded by the UK’s Department for International Development (DFID). In the first interim phase, survey and administrative datasets were reviewed to understand changes in Punjab’s education landscape since 2012. The first post in this series discussed participation and access. This post will focus on educational attainment.

School-level averages of learning outcomes for Punjab show marginal improvement in educational attainment in the province. The analysis of raw data from different data sets reveals that there have been marginal gains in learning outcomes in the province in all basic literacy and numeracy competencies assessed i.e. English, Math and Urdu[1]. For the purpose of this post, we have used DFID data which has been collected through a school base survey, using learning outcomes data for grade 3 pupils.

The average overall student score has increased from 70.2% in 2015 to 78.4% in 2018, resulting in an increase of 8.2 % in the three-year assessment period. In terms of average student scores in subjects, mathematics has shown the largest increase in this period, going up by 9.8%, followed by Urdu at 8.9% percent, and finally English at 5.5%.

Analysis of this data raises several important issues. Firstly, the overall limitations of all collected data are in themselves an important finding. They indicate the need for nuance, perspective and context while analyzing the raw data. They also highlight crucial gaps in data collection which need to be addressed in order to gain an accurate assessment of changes in learning outcomes. The second point is especially crucial in terms of developing and implementing appropriate and targeted policy interventions.

Secondly – and more specifically – the available data shows us marginal progress only for the limited set of literacy and numeracy skills that have been assessed. This means that when interpreting the results, it is necessary to remember that we lack evidence about other aspects of children’s learning and development. For example, a supportive and literate home environment, access to home tuitions, and even basic nutrition and health, among others. Thirdly, it is also helpful to remember that learning progresses, and learning outcomes change, in a complex and non-linear manner. Data snapshots for a specific period in time are therefore not helpful in conducting accurate analysis. It is much more helpful and informative to collect data and observe potential gains over a longer time period. The current time period for which data is available is only able to give us a surface level look into improvements in educational attainment in the province.

Lastly, and most critically, the current data reveals the need for a great deal of context about who is being assessed. Simply put, this means that we need much more information about children under assessment in order to have a deeper understanding of improvements in learning outcomes. Variations such as location, gender, socio-economic status, disability, school-type, and other potential marginalization, are integral in order to accurately understand learning outcomes. Understanding whose learning is being assessed is especially more important when it comes to the enrolment and consequent progress of disadvantaged children. Disadvantaged children – be it wealth, location, gender, disability, or an intersectionality of all of these – tend to have poorer learning outcomes than their counterparts. Increased enrolment of disadvantaged children therefore has a negative effect on measured learning outcomes, particularly if school quality remains unchanged. The inability of current data sets to categorize socioeconomic profiles of students being assessed is therefore a significant drawback, and greatly impacts the ability of researchers and policy makers to gain a meaningful understanding of success in educational attainment.

Despite the data limitations, it is important to note that in this case, limited improvement does not necessarily signify failure. This is because improvements in learning tend to be cumulative. Not observing large improvements in learning outcomes over a relatively short period is by itself not indicative of a system-wide failure in educational attainment. There is significant literature that suggests that being in school matters for learning for children  One potential path to success is through increased enrolment, and then retaining students in the educational system long enough to identify meaningful improvements in learning beyond their particular socio-economic context.

[1] The PESP II evaluation studied educational attainment across three different data sets i.e. Learning and Numeracy Drive data 2015-2017 (LND), DFID’s Six-Monthly Assessment data (2014-2017), and ASER data. For both LND and DFID, data are mainly collected on basic literacy and numeracy (English, Urdu and Maths).

[2] See work done by the Institute of Development and Economic Alternatives on this under the Teaching Effectively All Children (TEACh) project: http://ideaspak.org/human-development/teaching-effectively-all-children-teach/

Maheen Saleem Khosa is the Assistant Manager Communications at the Institute of Development and Economic Alternatives (IDEAS).