(Image: DFID – UK Department for International Development)
Rabea Malik
Recent policy reform efforts in Pakistan have given immediate priority to the enrollment of out-of-school children, retention of those at high risk of dropping out and ensuring learning gains for those in school. Data are required for understanding who is benefiting, who is excluded, links between teaching, learning and disadvantage and ways in which reforms impact classrooms, schools and communities.
Children from poor households – particularly girls from poor households – and children with disabilities are much more likely to be at risk of dropping out or never entering school at all. Children from poor backgrounds are likely to face considerable learning challenges while they remain in school.
Also high on the policy agenda is teacher effectiveness, which plays a crucial part in increasing student enrollment and ensuring classrooms are inclusive of all backgrounds. Policy research over the past couple of decades has made it clear that experience and qualifications alone cannot explain why some teachers are more effective than others. Pedagogical expertise (classroom practice), teacher attitudes and motivation matter most for student learning and engagement. These are also areas with the most significant data gaps.
Current sources of data and gaps in data
A number of survey-based and administrative data sources are available and being used for monitoring teaching and learning, and in some cases for policy responses. A mapping of existing data sources, undertaken as part of the background work for Teaching Effectively All Children (TEACh), reveals a long list of sources with information on relevant variables:
| Surveys | Gender | Income/wealth | Disability | Learning | Teacher experience /qualifications | Teacher attitudes /practices |
| Pakistan Social &Living Standards Measurement (PSLM) | ✔ | |||||
| Household Integrated Economic Survey (HIES) | ✔ | ✔ | ||||
| Neilsen survey (McKinsey) | ✔ | ✔ | ||||
| Punjab Examination Commission (PEC) | ✔ | ✔ | ||||
| Directorate of Staff Development (DSD) | ✔ | ✔ | ||||
| Education Management Information System (EMIS) | ✔ | ✔ | ||||
| Pakistan Poverty Alleviation Fund (PPAF) | ✔ | ** | ||||
| Learning & Educational Achievement in Punjab Schools (LEAPS) | ✔ | ✔ | ✔ | ✔ | ||
| Annual Status of Education Report (ASER) | ✔ | ✔ | ** | ✔ | ||
| Multiple Indicator Cluster Survey (MICS) | ✔ | ✔ | ||||
| Early Grade Reading Assessment (EGRA) | ✔ | ✔ | ✔ | |||
| Teaching Effectively All Children (TEACh) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
Note: ** Surveys piloted tools for identifying disability
The mapping also reveals some gaps. Sources that collect information on individual/household socio-economic characteristics (such as Pakistan Standards of Living Measurement Survey (PSLM) or Household Integrated Economic Survey (HIES)) do not collect information on learning levels. Administrative data sources that provide information from large scale standardized learning assessments do not provide student background information (such as Punjab Examination Commission results for 3rd, 5th, 8th and 10th grade, and the more recent Sindh Assessment Tests, as well as the monthly assessments undertaken by the Directorate for Staff Development in Punjab). Those that collect information on both learning and disadvantage (such as LEAPS, ASER), do not collect data on teacher attitudes and pedagogical practices.
The biggest gaps emerge for data on disability (inside and outside classrooms), information on teachers’ attitudes (regarding children with disabilities, those from poor backgrounds and slow learners), their level of preparedness for identifying and managing diversity in classrooms and the practices they undertake. Policy research on themes of learning, teaching and disadvantage can potentially bridge these gaps: proving missing data, and instruments for collecting data that can improve the effectiveness of policy responses.
Using data for policy impact
The silver lining is that policy makers today see the benefits of having information available to them for planning and policy design – a preference reiterated during a policy dialogue held recently in Lahore. They’re keen to – and in some ways they have already – put in place mechanisms that make available data on enrollment, teacher attendance, student attendance, school expenditures, etc. Digital dashboards highlighting key indicators and road maps tracking progress across regions are examples of such mechanisms.
However, steps can be taken to improve the effectiveness of these foundational measures. Chief among them are: 1) ensuring the right indicators are tracked, 2) data are being utilized by empowered local agents, rather than being retained centrally, and c) data help form a feedback loop between policy and practice, rather than being used exclusively for high-stakes assessments (when teachers’ promotions and appraisals are linked to trends in enrollments and assessments).
The systemic reforms needed for effective intervention
Currently, learning assessments are undertaken every month in government schools to inform teacher appraisals and school rankings by comparing students across schools, communities and regions. These comparisons do not consider student background or information on other challenges the child might be facing. Local education departments currently do not track children at risk of dropping out or those who need special attention in schools. A mechanism for systematic identification of learning challenges and other disabilities at scale is currently not in place.
Assessments can be restructured to track improvements in the same children over time. Information on teacher attitudes and practices, as well as the challenges they face, can inform the design and focus of support mechanisms (such as in-service training and school resource decisions). Internationally developed and validated survey modules that help identify the nature and severity of disability for all children in and out of school can provide accurate numbers for planning.
The ultimate goal should be to use the information to inform change in practice. An effective policy-practice feedback loop requires making information and authority to act on information available to local decision makers. In other words, it is important to make sure the data are used in a way that empowers teachers and school leaders, and enables them to change their practice willingly rather than being utilized for high-stakes accountability.
Rabea Malik is a research fellow at the Institute of Development and Economic Alternatives (IDEAS).