Here you will find a checklist summarizing the important points of the section: What do we mean by answering evaluation questions? __ Evaluation questions refer to what stakeholders – the community and funders, for example – want to know about the functioning of the program or initiative. Some illustrative evaluation questions: __ Are participants satisfied with the program? (Process and Implementation Issue) __ How well is the initiative meeting its stated objectives? (Attainment of Objectives) __ How much and what kind of difference did it make for participants? (Impact on Participants) __ How much and what kind of difference did it make on the community? (Impact on the Community) Why answer the key questions? __ To improve your work. __ To understand what affects the work in what ways. __ To see how to accelerate results. __ To understand specifically how broader actions, events, or conditions – e.g., a crisis or concentrated poverty – affect the work. __ To understand what works to bring about community change, and adjust accordingly. __ To understand how to address specific events and changes within your organization or effort so they will have the most positive or least negative effects. __ To show the community the value of your work. Who should be involved in answering these questions? __ This type of evaluation works best as a participatory effort. Those who might be involved include: __ Participants in or beneficiaries of the effort. __ Residents of a geographic area you’re focused on. __ Professionals and volunteers carrying out the work. __ Those whose jobs or relationships bring them into contact and involvement with the population and/or issue you’re concerned with. __ Funders and local officials. When should you set up and use an evaluation system to answer key questions about the effort? __ Start at the very beginning of the effort, so you can record the whole of your process – outreach, planning, implementation, and evaluation. __ Continue gathering, recording, analyzing, and using data throughout the course of the effort. How do you use an evaluation system to answer key questions about the effort? __ Decide what information you need to answer each question. __ Decide how to gather that information. __ Devise a method for recording and setting up your data that makes it easy to analyze. __ Graphing is particularly good because it allows you to easily compare different sets of data. __ Consider each question separately: Is the initiative serving as a catalyst for community/system change related to its mission? Changes to look for include: __ New or modified programs. __ New or modified practices. __ New or modified policies. What factors or processes are associated with the rate of community or system change? __ The processes by which you conduct the assessment, planning, implementation, and evaluation aspects of the effort. __ People served or benefited. __ Significant program events. These might include: Changes in leadership. Staffing changes. Changes in method or direction. Milestones of the effort. Increases or cuts in funding. Unforeseen circumstances. __ Community events, such as: A change of local political leadership or government administration. Racial or ethnic conflict. An economic windfall. An economic downturn. A community tragedy or celebration. __ Changes in broader conditions. How are community/system changes contributing to efforts to promote community health and development? __ Examine community changes in relation to various aspects of your work. Group goals or aims. The strategies of intervention you’re using. Risk and protective factors. The expected duration of change. The populations benefited. The sectors addressed. The ecological level addressed. Place. Are community/system changes related to improvements in population-level outcomes that reflect the objectives of your effort? __ Community-level indicators of an issue are markers of success at the level of the community as a whole, rather than for particular individuals. __ Population-level outcomes can be found or inferred by consulting publicly available statistics and records, including: Census data. Public health statistics. Records of health and human service organizations. Although most of these organizations shield individual participants with confidentiality, their general records – number of people served, units of service, general outcomes, etc. – are often open to scrutiny. Statewide data and data from other communities (for comparison purposes). Police and court files. Educational data – standardized test averages, truancy, dropouts, high school completion rates, incidence of school violence. Environmental statistics – pollution rates, bad air days, amount of open space, water quality, etc. __ You can also collect other sources of data, using observation, surveys, and other methods. __ Graphing and comparing sets of data on community-level indicators can show connections among them. Did the effort lead to improvements in population-level outcomes? __ The effort may have led to improvements by influencing changes in the community or system (intermediate outcome) and (more distant) community-level indicators of success. that brought them about. __ Comparing data on population-level outcomes, community changes, and the timing of various phases and events of your effort can help you understand the connections among them. __ Begin early and continue to gather, record, analyze, and use data indefinitely to understand how to adjust your effort for greatest effectiveness.
