Measuring impact in Haiti: How big of a change are we making?

Share this page: 

A few days ago, our Statistics Consultant, Sylvia, sent me this WSJ article in which Bill Gates states that the first step towards solving the world’s problems is to measure them. This is not a nerdy statistician’s point of view; it echoes the question that haunts Sylvia and all of us, humanitarian professionals: how big of a change are we making?

Last year, as I was walking around a neighbourhood I worked in, an elderly lady grabbed me by the arm and insisted on offering me a plate of fried plantains in thanks for all the good work the international community was doing in the area. As I sat down eating (how can one turn delicious Haitian food down?), she told me in a hushed voice that a brand new solar lamp has been installed exactly where her niece had “problems” a few months ago. Maybe we could measure the impact of gender-based violence reduction programs in plates of plantains, but I doubt this would be very scientific.

Being accountable

As an international NGO, we are accountable to our beneficiaries, our donors, our partners and the institutions of the countries we work in. They demand specific and objectively verifiable indicators, especially in Haiti where the massive NGO presence (even before the 2010 earthquake) has not led the country out of extreme poverty and vulnerability.

My task as Urban Coordinator is to improve the lives of the most vulnerable city dwellers and ensure urban challenges are understood and addressed in Haiti. How do you measure this? Bill Gates takes the example of child mortality in Somalia, which seems like an easy thing to measure: the baby is either dead or alive. I can measure the number of people who have left camps to safe homes. But I can’t really ask residents of any given neighbourhood where we work in if their “vulnerability” has decreased from 80 to 60% in the past two months, it just doesn’t make sense.

Measuring quality of life

Quality of life indicators are not only an NGO issue. The trusted Mercer Quality of Living survey – designed by an HR consulting firm – calculates through a series of indicators expatriate benefits in different cities. This is quite a different focus from Oxfam’s, but the indicators are interesting nonetheless. They include elements that are at the heart of humanitarian and development action such as water quality, level of violence, availability and access to health and social services.

Another famous survey (Global Liveability Report from the Economist Intelligence Unit) uses the number of English speakers in a city as an indicator to determine its quality of life. As a native French speaker, I don’t mind if no one in my hometown speaks English, I find it really liveable. This goes to prove that any set of criteria is determined by the surveyor’s priorities.

We should never forget this subjectivity when we choose the indicators for our projects; and this is why indicators are discussed with the local population beforehand to make sure they are relevant and real to them. Only then we know we are working 100 per cent for the right change and we know how to measure it.

You may also like

Slideshow: Haiti three years after the earthquake

The From Poverty to Power "MEL wonkwar debates":

Theory’s fine, but what about practice? Oxfam’s MEL chief on the evidence agenda

So What do I take Away from The Great Evidence Debate? Final thoughts (for now)