Search form

Section 1. Using an Evaluation System to Answer Key Questions About Your Initiative

Here you will find a checklist summarizing the important points of the section:

What do we mean by answering evaluation questions?

__ Evaluation questions refer to what stakeholders – the community and funders, for example – want to know about the functioning of the program or initiative.

Some illustrative evaluation questions:

__ Are participants satisfied with the program? (Process and Implementation Issue)

__ How well is the initiative meeting its stated objectives? (Attainment of Objectives)

__ How much and what kind of difference did it make for participants? (Impact on Participants)

__ How much and what kind of difference did it make on the community? (Impact on the Community)

Why answer the key questions?

__ To improve your work.     

__ To understand what affects the work in what ways.

__ To see how to accelerate results.

__ To understand specifically how broader actions, events, or conditions – e.g., a crisis or concentrated poverty – affect the work.

__ To understand what works to bring about community change, and adjust accordingly.

__ To understand how to address specific events and changes within your organization or effort so they will have the most positive or least negative effects.

__ To show the community the value of your work.

Who should be involved in answering these questions?

__ This type of evaluation works best as a participatory effort.

Those who might be involved include:

__ Participants in or beneficiaries of the effort.

__ Residents of a geographic area you’re focused on.

__ Professionals and volunteers carrying out the work.

__ Those whose jobs or relationships bring them into contact and involvement with the population and/or issue you’re concerned with.

__ Funders and local officials.

When should you set up and use an evaluation system to answer key questions about the effort?

__ Start at the very beginning of the effort, so you can record the whole of your process – outreach, planning, implementation, and evaluation.

__ Continue gathering, recording, analyzing, and using data throughout the course of the effort.

How do you use an evaluation system to answer key questions about the effort?

__ Decide what information you need to answer each question.

__ Decide how to gather that information.

__ Devise a method for recording and setting up your data that makes it easy to analyze.

__ Graphing is particularly good because it allows you to easily compare different sets of data.

__ Consider each question separately:

Is the initiative serving as a catalyst for community/system change related to its mission?

Changes to look for include:

__ New or modified programs.

__ New or modified practices.

__ New or modified policies.

What factors or processes are associated with the rate of community or system change?

__ The processes by which you conduct the assessment, planning, implementation, and evaluation aspects of the effort.

__ People served or benefited.

__ Significant program events. These might include:

  • Changes in leadership.
  • Staffing changes.
  • Changes in method or direction.
  • Milestones of the effort.
  • Increases or cuts in funding.
  • Unforeseen circumstances.

__ Community events, such as:

  • A change of local political leadership or government administration.
  • Racial or ethnic conflict.
  • An economic windfall.
  • An economic downturn.
  • A community tragedy or celebration.

__ Changes in broader conditions.

How are community/system changes contributing to efforts to promote community health and development?

__ Examine community changes in relation to various aspects of your work.

  • Group goals or aims.
  • The strategies of intervention you’re using.
  • Risk and protective factors.
  • The expected duration of change.
  • The populations benefited.
  • The sectors addressed.
  • The ecological level addressed.
  • Place.

Are community/system changes related to improvements in population-level outcomes that reflect the objectives of your effort?

__ Community-level indicators of an issue are markers of success at the level of the community as a whole, rather than for particular individuals.

__ Population-level outcomes can be found or inferred by consulting publicly available statistics and records, including:

  • Census data.
  • Public health statistics.
  • Records of health and human service organizations. Although most of these organizations shield individual participants with confidentiality, their general records – number of people served, units of service, general outcomes, etc. – are often open to scrutiny.
  • Statewide data and data from other communities (for comparison purposes).
  • Police and court files.
  • Educational data – standardized test averages, truancy, dropouts, high school completion rates, incidence of school violence.
  • Environmental statistics – pollution rates, bad air days, amount of open space, water quality, etc.

__ You can also collect other sources of data, using observation, surveys, and other methods.

__ Graphing and comparing sets of data on community-level indicators can show connections among them.

Did the effort lead to improvements in population-level outcomes?

__ The effort may have led to improvements by influencing changes in the community or system (intermediate outcome) and (more distant) community-level indicators of success. that brought them about.

__ Comparing data on population-level outcomes, community changes, and the timing of various phases and events of your effort can help you understand the connections among them.

__ Begin early and continue to gather, record, analyze, and use data indefinitely to understand how to adjust your effort for greatest effectiveness.