Warning
The following is dry. I thought I’d let you know off the bat. The writing is not meditative. It is not a journey into the mind. Those things come later, in my romanticized vision where I am laying on my cot in Les Cayes, Haiti, sweating, sipping water (or scotch, or, more likely, rum), piecing together the events of a day, probably dumbfounded. That makes good writing, even if the words are inexact, the reader forgives, because it is personal. What follows was written in New York City, in my parents’ apartment, in the week leading up to my departure. If you are able to pick up the scent of uncertainty and fear I congratulate you, for while these feelings were not consciously placed in the following passages, I would not be surprised if they lay there nonetheless.
Background
Haiti is the poorest country in the Western Hemisphere (measured by GDP, the Gini Co-efficient, and HDI—see my paper “Development, Aid, and Inequalites” for explanations of these terms). Although the country’s government has been a vocal advocate for improving its’ educational system, there is little evidence to suggest that Haitian children have access to, or excel in, Haiti’s schools. This is partially due to Haiti’s small GDP, and subsequently small monetary investment in education. A less obvious reason for Haiti’s poor educational track record is due to language: the official language of Haiti is French, and all government schools teach in French. The rural population of Haiti, however, speaks Creole, and it has been suggested that the persistence of French in rural classrooms is used as a form of political control.
Due to the lack of resources devoted to education by the Haitian government, NGOs (non-governmental organizations–essentially, non-profits) have stepped in to fill the void. Of the approximately 15,000 schools the Haitian government can account for, 90% are private, run by communities and NGOs. “The general level of trust in schools and in teachers’ skills is great. This trust motivates most parents to send their children to school although public provision is scarce and quality is unregulated.” (Salmi, 2000)
Haitian teachers are at the mercy of a poor Haitian government, which often results in salaries that are sporadic at best, nonexistent at worst. The inconsistency of teacher salaries filters down to students, where it often impacts attendance (students don’t come to school if they’re unsure of whether their teachers will be there). Poor attendance is one of several factors that are known to have detrimental effects on student achievement.
A Theory of Change
From the end of August 2009, to the beginning of April 2010, I’ll be monitoring and ultimately evaluating an educational program designed to provide consistent (both in amount and regularity) salaries to approximately 500 private school teachers. If teachers receive regular paychecks, they are more likely to be professional, showing up on time, staying at the school throughout the school day, and generally being reliable figures in an otherwise tenuous environment for children. If teachers are more reliable, student attendance should increase as a result. Currently, students pay tuition to attend private school (generally a modest figure, around $7-$8 per month, $US) to raise money for the school’s teacher salaries. When the NGO implements this program, student tuitions are reduced and in some cases eliminated. It is the policy of the NGO I’m working for that any school with this program in place cannot turn away any student due to lack of tuition.
The explicit theory of change (TOR) here is that by subsidizing teacher salaries, both student attendance and gross enrollment at schools will increase. Implicit in this TOR, however, is that by increasing their attendance and enrollment, students ultimately benefit educationally. A bit harder to prove, but we’ll give it a shot nonetheless.
Methods
The program I’ll be evaluating covers 37 schools, 500 teachers, and 12,000 students. That’s a lot of people to track, a lot of people to survey, considering this program has never had an official monitoring and evaluation (M&E) system put in place, and that I’ll be evaluating this entire program alone. I will not BE alone, but my co-workers will be focused on the NGO’s other programs, most notably a health clinic in Les Cayes that they recently constructed. It’s impossible, logistically, to monitor 12,000 people alone, so instead I’ll be focusing on the school directors (15), the schools themselves, and a sampling of the teachers. Right now I’m aiming for 100-150 teachers, but that number will change once I’m on the ground and get a realistic picture of how many schools I can closely monitor in the 7-odd months I’m there. A good first step, I believe (I hope!), is creating a database within which all those stakeholders (15 school directors; 37 schools, 100-150 teachers) can be tracked, within which all the data I collect can be organized and dissected. I’ve never created a database, so I decided to go with a relatively easy one, Filemaker Pro, which I taught myself how to use using tutorials I found on the Internet, and which I’ve built from scratch over the past 3 months. A true database is relational, which is what separates it from what most of us use to keep track of data, namely a spreadsheet of some kind. Picture a spreadsheet for each group, for the school directors, for the schools, for the teachers, which are all able to communicate with each other. You update data in one field, say, an individual teacher’s number of sick days taken, and this data is reflected in the school’s overall data (say, number of total teachers’ absences), and a school director’s data. Below you can see what a relational database looks like visually: the school directors are at the top of the pyramid. To their right is a “beginning of year survey” I’ll be administering to all of them. Below them are the schools they’re in charge of, and the teachers each school employs. If you were to scroll down you’d see another window for the students in each teacher’s class, though I’m undecided as of now if I’m going to venture into surveying individual students. I’ll certainly collect overall student data—their attendance, their performance on the national 6th grade exam (when applicable—many of the schools are primary and not subject to this test), and if possible, their location in comparison to the schools they attend. The distance a student must walk to reach school is one of the more straightforward pieces of data I can collect. (As I write this, it feels like one of those assumptions we make before actually doing fieldwork—it may turn out to be an incredibly hard or inaccurate piece of data to collect.)
I’ll be inputting data into this database at the school director, school, and teacher level, by surveying these groups, and collecting hard data from them directly.
Problems that keep me up at night
For one, all the data I collect will be self-reported. There will be no independent organization, no department of education, collecting this data; school directors will provide it. If I’m a school director, getting a couple thousand dollars a year to pay for my teachers from an NGO, and that NGO (more specifically, some white kid I’ve just met who I don’t trust) will be evaluating me based on the data I give them, my data will sometimes be biased. I’m sure there are some school directors who don’t give a shit, who will provide honest data about the state of their school, because they know it’s the right and honorable thing to do, and because the white kid has explained to me that my funding will not be cut no matter what the numbers say. Then there are the rest of the directors.
Another problem…you could call this one the doomsday problem, is that no one has data on these schools BEFORE THE PROGRAM STARTED. In other words, we have nothing to compare the data we’re getting to what these schools were like before the program started. That’s pretty much a killer. There’s no way around that one. What I can do, however, is collect data on a control group, on schools not in the program currently, but who have similar geographic and socio-economic indicators. This could at least provide some context as to what affect the program is having, good or bad. It doesn’t solve the problem of not having the pre-program data, which, again, really really is a killer, but it at least adds some value to the data I’m collecting. One problem with this is that the schools in the control group will most likely be ones that WANT to be in the program, and so might skew their data to appear more…appealing? Organized? Something. I don’t know how this will skew the data, but I’ll bet it will. School directors will be afraid of being unflinchingly honest, despite my protests, because they will think that we will be accepting or rejecting them based on the data they provide. (I honestly don’t know if their fears are well-founded are not, but I hope not. Schools should be rewarded for brutal honesty, not punished, and everyone’s better off with more realistic data—if anything it helps our program because it gives us more room for improvement on attendance and test scores and other things, rather than trying to improve on 100% attendance etc.)
So, to summarize: I have a system in place for collecting data from the schools this program is involved with. I’ll travel a lot to these schools, administering surveys and collecting the data the schools have about the teachers who work there and the students who attend. Then I’ll disseminate this data and try to reach some conclusions about whether or not the program is having an affect on the schools, teachers, and most importantly, students. That’s what I’m doing.
