Two Types of Evaluations

The two major categories of measurement are impact evaluations and process evaluations.

• Impact evaluations measure your organization’s outcomes. If your nonprofit distributes mosquito nets, an impact evaluation might determine the effect of your work on rates of malaria.

• Process evaluations measure your organization’s internal effectiveness.

Let’s say your organization trains volunteers. A process evaluation might determine how many volunteers you trained, how much information they learned during the training process, or how many of those trainees went on to become actual volunteers. Process evaluations don’t measure the effect those volunteers made, but rather your own organizational efficiency training them.

Although impact evaluations may sound more compelling, process evaluations are just as important. A charity can’t accomplish what it wants to if its internal mechanics don’t function properly.

If the delivery is flawed, it doesn’t matter that the causal logic is solid, says Rachel Glennerster, executive director of the Abdul Latif Jameel Poverty Action Lab at the Massachusetts Institute of Technology.

For some organizations, a process evaluation may be far more important than an impact evaluation.

Paul Niehaus, a professor at the University of California at San Diego and co-founder of the nonprofit GiveDirectly, gives the example of an organization that distributes pills. Imagine that academics have clearly established a link between taking the pills and preventing a certain disease. They’ve also calculated the economic benefit of preventing this disease. In other words, the charity doesn’t need an impact evaluation because the effect of the work is already known.

But the remaining question is: How effective is your nonprofit at distributing the pills? A process evaluation answers that.

Marc Epstein, a professor at Rice University and author of the book Measuring and Improving Social Impacts, says there are four ways to measure impact:

• Trained judgment, which is a half-step up from an anecdote. This is when someone with established and credible expertise in a relevant field evaluates your program. That could be a doctor or an academic surveying your operation and offering his or her opinion.

• Qualitative research, which entails a trained researcher conducting a rigorously designed experiment in which the outputs are expressed in observation, not numbers. This could be a structured observation along the lines of what an anthropologist does or a survey in which subjects answer open-ended questions and a researcher synthesizes the responses.

• Quantitative research, which is similar to qualitative research, except the outcomes are expressed in numbers. For example, the number of people who got jobs as a result of your job-training program or the average increase in test scores among the youth you work with.

• Monetization, which is a type of quantitative research that takes numerical results and tries to convert them into a common metric, often dollars. This can be helpful because it takes an output and turns it into a variable that can then be easily compared with other activities. Monetization is particularly helpful for comparing the social good of two unrelated causes. For example, Population Services International developed a metric, DALY, that measures a “year of healthy life lost” due to an individual’s death or disability. The nonprofit aims to avert DALYs through its programs.