Implementing Complex Reforms: The Roadmap Approach in Pakistan

On the 6^th of July, the Consortium for Development Policy Research (CDPR), held a policy talk covering Pakistan’s experience of reforms in public sector service delivery. A core area of focus for the panel was the Roadmap approach towards performance improvement in particular social sectors, which has gained popularity with various subnational governments since 2011. Most notably, the Roadmap was first introduced to the country in Punjab’s education sector back in 2011, and it has remained the driving instrument of reform in the province since then.

What is the Roadmap Approach?

At its simplest, the Roadmap approach is a system of setting targets and goals for a defined sector or sub-sector, and a way to manage its implementation through accountable practices. This is done, usually but not exclusively, at higher-levels of policymaking, such as at the level of the Chief Executive, or the head of a particular department. In that sense, it remains a top-down strategy, designed to harness support of key decision-makers and generate momentum from above.

Although the Roadmap approach draws many of its basic premises from general management theory, part of its design also comes from the experience of delivery units, and the general practice of ‘deliverology’, pioneered during the Tony Blair administration in the UK. Delivery units are conceived of as small teams, which draw on the explicit backing of the key decision-maker for the purpose of mobilizing high-level political support and coordination to see through a particular set of reforms. Shostak et al. (2014) further state that such units carry the mandate to use the authority of the chief executive to: 1. focus on improving results, as measured by citizen outcomes in a limited number of priority areas; 2. unblock obstacles when monitoring shows that progress is off track; and 3. build understanding and capability for strengthening the underlying actors and systems/processes’.

In this backdrop, the Roadmap approach can be viewed as both a way to resolve the pressing issue of ‘political will’ in development processes – a concern that has gained increasing centrality in recent years – as well as a technical intervention to bypass capacity constraints in outdated or under-resourced bureaucracies. In particular, the ability to collate and analyse data for benchmarking, and use it to strategize and prioritise reform interventions is the added-value that Roadmap teams bring to the table.

The Case of Punjab

 Punjab’s experience with the Roadmap approach echoes many of the general underpinnings highlighted above. It was first implemented in the education sector, as the Chief Minister’s Roadmap for Education, launched to drive effective delivery of education reform in Punjab by harnessing high level political commitment and accountability. The Chief Minister himself chairs the education reform agenda, with management support from a specialist Roadmap team and UK Special Representative for Education Sir Michael Barber through the stocktake meetings, which double as an accountability forum. Progress is delivered through the School Education Department (SED) and attached departments, with results measured at the district level. Crucial to the functioning of this system is the data collection regime instituted through the Program Management and Implementation Unit (PMIU), which uses field-based monitoring officers to collect school-level data across a wide range of learning/teaching quality, human resource, and infrastructure indicators.

In 2015, the Chief Minister inaugurated a set of ambitious 3-year targets to be tracked and implemented through the Roadmap:

1. Enrolment and access: 95% minimum participation rate for 5 to 9 year olds;
2. Teaching quality: 75% score on six-monthly assessment;
3. Schools and teachers: 36,000 new classrooms, 46,000 new teachers, 100% functioning facilities in schools;
4. And Public private cooperation: 2.6 million students enrolled by Punjab Education Foundation (PEF) schools by 2018

To achieve these, a number of interventions were piloted and then scaled up, including many that involved reforms in key organizations related to teacher training, textbooks and curricula, and examinations. The data regime allowed for close tracking of progress, keeping province-level goals as the collective end-point. Each district was assigned targets in consultation with the provincial team, the DCOs, and the EDOs/DEA CEOs, keeping in mind their respective district baselines. A system of incentives (bonuses – typically worth twice their monthly salary – and other material rewards) and punitive actions (reprimands and transfers to low-status designations) was introduced to motivate district managers to meet these targets.

By 2018, the Roadmap has achieved varying degrees of success in the 4 target areas identified. Gains have been made in both school participation, retention, and learning outcomes, though all have fallen short of the targets set in 2015. Other areas of success have been in improving infrastructure – such as the provision of basic facilities to all schools – and improved human resource management, which has seen teacher and student attendance rise to an average of above-90%.

The Way Forward

The sector gains identified as a result of the Punjab Education Roadmap has led to its replication in a number of sectors and areas in Pakistan. In Punjab, Roadmaps are in place for water and sanitation, healthcare, and solid waste management. Similarly, there is an education roadmap in operation in Khyber-Pakhtunkhwa, led by the provincial education minister, and one currently being designed for the province of Sindh as well.

At this stage, there are few independent evaluations of the Roadmap approach, and the data from Punjab’s experience shows a modicum of success, which has driven the expansion and replication process. However, there are several key questions that emerge from Pakistan’s experience, which need to be interrogated more thoroughly.

The first relates to the long-term value of a top-down, centralized approach in sectors that require more diverse set of reforms and devolved decision-making. This is an issue that has been highlighted for the education sector, where targets being measured against a fixed set of variables may be insufficient in capturing localized school-level dynamics and nuances.

Secondly, there are concerns that high-intensity accountability structures, such as the stocktake process with its system of reprimands, may generate incentives to ‘game’ the system. This remains an empirical question, and one that has to consider the quality of data being generated through the monitoring process.

Lastly, given its reliance on political will and the buy-in of senior decision-makers, there remain valid question marks over the sustainability of the approach beyond one election cycle or shifts in key personnel. Given that an election is around the corner in Pakistan, it would be interesting to see how the approach interacts with a change in government, and whether it is able to adapt to the new circumstances.

Umair Javed is the Program Manager at Consortium for Development Policy Research.