What makes a good evaluation?
The Joint Committee on Standards for Educational Evaluation (www.jcsee.org) has defined four main principles that underlie good evaluation:
- Utility: This ensures that the evaluation is collecting credible, useful, timely information. The purpose of an evaluation is to determine what works and how, and to inform decision-making. If it doesn’t address current needs and realities, then there’s really no sense in moving forward.
- Feasibility: This principle prioritizes evaluation that is practical, cost-effective and politically viable.
- Propriety: This relates to legal and ethical standards that should govern an evaluation, including careful consideration of those involved as well as those who might be impacted by the results.
- Accuracy: The data yielded from an evaluation must be accurate in order to be useful (and to ensure your organization’s credibility). This is connected to the rigor of your evaluation plan, data collection methods, and willingness to report the bad just as much as the good.
To the last point, as much as it might be tempting to focus on only the positive, doing so erodes the credibility of your evaluation and your organization. Most donors, and people in general, understand that failure is instructive. For a nonprofit program, it can lead to the refinement or refocus that ultimately creates positive change. On the other hand, you’d be hard-pressed to find anyone understanding of an organization that swept negative findings under the rug. Our point? Be comprehensive in your reliability. Just as in life, it doesn’t mean much to be half reliable.
The USDA’s Food and Nutrition Service (www.fns.usda.gov) provides a range of useful guidelines in Principles of Sound Impact Evaluation. While the full publication is intended for nutrition educators, the general guidelines put forth will apply to many nonprofits. Specifically:
- Make certain that the program components can realistically be evaluated.
- Build on available research.
- Choose measures that fit what you’re doing and approach existing standards for credible assessment.
- Observe ethical standards for the fair treatment of study participants.
- Measure both process and outcome.
- Report both positive and negative results – but do so accurately.
- Share results to maximize their value.
Finally, your evaluation design should flow from your logic model. Since your logic model explains how you’ll create change, the evaluation is linked to it in that it’ll confirm where you are on target and what needs to be refined.
Adapted in part from the Joint Committee on Standards for Educational Evaluation’s “Program Evaluation Standards Statements.”