Table of Contents >
Chapter 37. Some Operations in Evaluating Community Intervent... >
Section 1. Choosing Questions and Planning the Evaluation >
Choosing Questions and Planning the Evaluation |
|
| Contributed by Steve Fawcett and Phil Rabinowitz |
Checklist
Here you will find a checklist summarizing the important points of the section.
What do we mean by choosing questions?
__ Evaluation questions are the questions your evaluation is meant to answer about your work.
__ Evaluation questions help set the direction of the work, as well as assess its effectiveness.
__ Ideally, choosing evaluation questions is part of the planning of the overall program.
Questions to ask yourself as you choose evaluation questions:
__ What do you want to know?
__ Why are you interested?
__ Is the issue you’re addressing important to the community or to the society?
__ How does the issue relate to the field?
__ Is the issue general, rather than specific to your population or community?
__ Who might use the results of your evaluation?
__ Whose issue is it?
Why is it necessary to choose evaluation questions carefully?
__ It helps you understand what effects different parts of your effort are having.
__ It makes you clearly define what it is you’re trying to do.
__ It shows you where you need to make changes.
__ It highlights unintended consequences.
__ It guides your future choices.
__ In participant evaluations, it involves stakeholders in setting the course of the program, thus making it more likely that it meets community needs.
__ It provides focus for the evaluation and the program.
__ It determines what needs to be recorded in order to gather data for evaluation.
When should you choose questions and plan the evaluation?
__ If possible, choosing questions and planning the evaluation should be an integral part of planning your program.
__ If your reality makes that impossible, choosing questions and planning the evaluation should take place as soon as possible after the program starts.
Who should be involved in the process?
__ To the extent possible, the process should involve all stakeholders, including program participants and beneficiaries.
How do you choose questions and plan the evaluation?
Choosing questions:
__ Describe the issue or problem you’re addressing.
__ Describe the importance of the problem.
__ Describe those who contribute to the problem.
__ Assess the importance and feasibility of changing those behaviors.
__ Describe the change objective.
__ Make sure that the expected changes would constitute a solution or substantial contribution to the problem.
Planning:
__ Take into account the issues raised by multiple or very different settings.
__ Take into account the issues raised by participant groups that differ in culture, ability to complete the program, geographical location, and other factors.
For outside evaluators, specifically:
__ Choose a setting.
__ Learn as much as you can about the organization you’ve chosen.
__ Contact the appropriate person(s) and request an interview.
__ Plan and prepare for the initial meeting.
For all evaluators:
__ Find out all you can about the context.
__ Establish trust with program administrators, staff, and participants.
__ Aim for a participatory evaluation.
__ Plan the evaluation in collaboration with stakeholders.
__ Consider all the elements of an evaluation in your planning:
__ Information gathering and synthesis.
__ Designing an observational system.
__ Developing and testing a prototype intervention.
__ Selecting an appropriate experimental design.
__ Collecting and analyzing data.
__ Gathering and interpreting ethnographic information.
__ Collecting and using archival data.
__ Encouraging participation throughout the research.
__ Refining the intervention based on the evaluation.
__ Preparing the evaluation results for dissemination.