How to gauge the impact of empathy-building initiatives

(Image courtesy of Rabtt)

By Haseeb Sher Bajwa

Pakistan and prejudice

Prejudice towards women and minorities is deeply entrenched in Pakistani society. Education plays a vital role in perpetuating these prejudices through one-sided narratives and a failure to inculcate adequate critical thinking skills for students to objectively analyze the information they receive. This results in a society comprised of adults heavily prejudiced towards women and minorities that lacks the analytical skills to change those prejudices.

Several social enterprise organizations, such as Rabtt, The History Project and Ravvish, have emerged in the last five to six years to incorporate empathy and critical thinking into education. Where some use a holistic approach to engender these characteristics, such as teaching courses in various subjects that range from world history to public speaking, others use a more targeted approach of teaching special courses that offer multiple perspectives on history. As these firms iteratively improve how they teach empathy, it is equally important to develop adequate measurement techniques to gauge empathy levels in children. Without them, they cannot ensure whether children are becoming more empathetic critical thinkers. Benchmarking students’ performance using tools designed specifically for gauging empathy would also legitimatize their efforts in front of donor organizations.

For six months, I worked with Asad Liaqat, a PhD candidate at Harvard University, to develop surveys measuring empathy in children for The History Project, a history education enterprise. We hope that other enterprises can use these tools to develop their own surveys that gauge empathy in children.

Using the hammer: Interpersonal Reactivity Index

The Interpersonal Reactivity Index (IRI) is an empathy-measuring tool consisting of 28 questions that are divided into four subscales: perspective taking (PT), fantasy scale (FS), empathic concern (EC) and personal distress (PD). Each subscale consists of six to seven questions that ask students how much a statement describes them as a person. Student scores are then calculated for each subscale both before and after the workshop. Previously, IRI has been conducted with various groups of people including adolescents as well as adults.[1] And in all cases, IRI seems to show consistent internal validity, making it less prone to error.

Our application of IRI in low and high income schools in Punjab followed a similar pattern but excluded the PD scale, since it is the only scale in IRI that does not give consistent results. We conducted a pre-post intervention, in which students took the IRI survey before and after the workshop. We carried out the survey with roughly 650 students, which was sufficient for our purposes. We did not see any significant improvement in children’s empathy levels.

While IRI is currently being used by firms like Rabtt to evaluate empathy levels in children, they are running into sampling problems and are not yielding significant results. Their student sample is too small to give statistically significant results even if their workshop increases empathy in children. In addition, they have not excluded the PD scale from IRI which may also distort the consistency of their results.

Using the scalpel: Implicit Association Test

A problem with IRI is that respondents often answer questions for a desired outcome, rather than provide an honest, accurate reflection of their biases. The Implicit Association Test (IAT) tries to solve this problem by measuring the implicit bias of people towards various groups in society.

IAT runs like a computer game in which negative and positive words appear on the left and right hand sides of the screen. Respondents allocate words pertinent to separate ethnic groups to either the positive or negative side and the computer records the respondent’s time for making each word association. The bias is then calculated by the difference in average time it takes for respondents to associate bad words with their own ethnicity and the average time it takes for them to associate bad words with other ethnicities. If they are much faster at making negative associations with other ethnicities than their own, then bias towards other ethnicities exists. Multiple studies using IAT show its internal validity as an effective tool for measuring implicit prejudice.[2] IAT can be used by social enterprises in Pakistan to gauge the baseline prejudice of students against minorities.

We used IAT in high income schools in Lahore to gauge students’ biases towards Indians. The game included a list of 10 Pakistani and 10 Indian words that students had to assign to either the positive word category or negative word category. As expected, we found a significant bias towards Indians, which did not decrease significantly after the workshop. However, due to the small number of students who took IAT, we cannot be conclusive about these results.

Conclusion

In 2013, the National Commission of Justice and Peace issued a report saying that:

“Over the past 33 years (1970 – 2013), at least eleven different governments [in Pakistan] have come up with their national educational policies, education sector reform action plans, policy review teams, and a whole host of white papers. Somehow, statements of prejudice, descriptions of biases, bulletins of violence, and cannonballs of hate have remained in every educational policy, every education sector reform action plan, every policy review team and every white paper.”[3]

Prejudice towards minorities is deeply entrenched in Pakistani society and efforts to change these attitudes needs to start in schools. Fortunately, quite a few social enterprises in Pakistan have realized this issue and have started developing curricula to reduce existing prejudices. However, it is crucial for these organizations to develop tools that measure the empathy they are trying to build in students. Otherwise they could fall into the same trap as most government organizations in Pakistan that are not only unclear about the exact nature of their service delivery, but also do not have any tool to measure the performance of their services.

Haseeb Sher Bajwa graduated from Swarthmore College with a double major in Economics and Political Science. He is currently working on a project with the Institute of Development and Economic Alternatives. He can reached at haseeb.sher.bajwa@gmail.com. 

[1] See Katherine Péloquin, Marie-France Lafontaine (2010), Ana Maria Fernández, Michele Dufey, and Uwe Kramp (2011).

[2] Baron & Banaji, 2006; Craeynest et al., 2005; Dunham, Baron, & Banaji, 2006; Skowronski & Lawrence, 2001

[3] National Commission for Justice and Peace, “Education vs Fanatic Literacy: A Study on the Hate Content in Textbooks in Punjab and Sindh Provinces”, March 2013.