Section 1. Measuring Success: Evaluating Comprehensive Community Initiatives

Whenever you begin a new job or start a project, you will probably want to understand how well your work is going. You might ask yourself: "Is my work meeting its goals and the expectations of the people involved or affected?"

Community initiatives also benefit from evaluation throughout their work. Asking community members, participants, staff, and other stakeholders whether plans are working as intended is one important part of evaluation. Positive findings can help confirm that an initiative is moving in a useful direction, while other findings can identify where adjustments may be needed.

This chapter explains why evaluation is important and offers step-by-step approaches for evaluating your group's work. Evaluation can help you recognize progress, learn from challenges, and make informed decisions about what to continue, strengthen, or change.

What does it mean to evaluate a community initiative?

Evaluation is the systematic process of understanding the value, implementation, and effects of your work. After developing and carrying out an initiative, you need information about how well it is working, for whom, under what conditions, and where improvements may be needed.

In some areas of life, feedback is immediate. You press the brakes and the car slows down. You shoot a basketball and see whether the ball goes in. Community initiatives are more complex. Changes may take time, multiple factors may influence outcomes, and different people may experience the same initiative differently. Evaluation provides a structured way to understand what is happening rather than relying only on impressions or assumptions.

In this chapter, evaluation refers to gathering and interpreting information about your group, its activities, and the changes connected to its work. This information can help you determine which parts of your action plan are working well, which are creating barriers, and which should be adapted. Evaluation findings can then inform changes to goals, strategies, implementation, and future planning.

The sections in this chapter focus specifically on ways to gather useful information. The next chapter discusses ways to use that information to strengthen and improve your initiative. Organizations sometimes hesitate to evaluate their work, so it is helpful to recognize and address some common concerns.

Common concerns about evaluation

People may feel uncertain about evaluation because they are unfamiliar with the process, concerned about the time it requires, or worried about what the findings might show. These are understandable concerns, but they do not have to prevent useful evaluation.

I don't know how to conduct an evaluation.

Evaluation skills can be learned. This chapter walks through the basic steps involved in planning an evaluation, gathering information, interpreting findings, and using what you learn to improve your work.

I don't have the time.

Evaluation does require time and attention. However, identifying problems, barriers, or implementation challenges early can prevent larger difficulties later. Evaluation can also help organizations focus limited time and resources on approaches that are most useful.

Once useful information becomes part of routine program operations, evaluation may require less additional effort. Attendance records, participant feedback, implementation notes, community indicators, and other information can often be collected as part of regular work rather than through a separate process.

The findings may be disappointing or create challenges for us.

Evaluation may show that some parts of an initiative are not producing the outcomes you expected. That information can still be valuable. Identifying limitations gives you an opportunity to understand what is happening, make changes, and improve the quality and responsiveness of the initiative. Evaluation should be approached as a tool for learning and accountability rather than simply as a judgment of success or failure.

Why should you evaluate a community initiative?

Effective community work requires attention throughout planning, implementation, and follow-up. Consider a musician learning a new piece for a performance. Practice is essential, but feedback can help identify whether tempo, technique, interpretation, or another element needs adjustment. Without feedback, it may be difficult to recognize what is working well and what needs additional attention.

Community initiatives also need feedback throughout their work. An organization may devote substantial time, energy, and resources to pursuing its goals, but without evaluation it may be difficult to know whether the effort is reaching the people intended, responding to community priorities, being implemented as planned, or contributing to meaningful change. Evaluation helps identify progress and informs adjustments along the way.

There are many reasons evaluations are valuable. Some of the ways evaluation can benefit a community initiative include:

• Evaluation can document progress and strengthen support. Evidence of meaningful outcomes can help community members, partners, funders, and decision-makers understand the value of the work. For example, an initiative might document increased volunteer participation, improved access to preventive health services, stronger community partnerships, or other accomplishments. Clear evidence can support continued investment and help organizations communicate what they have achieved.
• Evaluation can turn challenges into opportunities for learning. When an initiative does not meet its goals, evaluation can help identify why. The issue may involve implementation, accessibility, resources, outreach, program design, changing community conditions, or another factor. Understanding these challenges provides a stronger basis for improvement than simply labeling the initiative unsuccessful.
• Evaluation can recognize accomplishments and sustain motivation. Seeing evidence of progress can reinforce the value of people's efforts. Findings that identify areas for improvement can also support motivation when they are treated as information for learning rather than as personal or organizational failure.
• Evaluation can strengthen future action. Once you understand what has worked, what has not, and why, you can refine your strategies and make better-informed decisions about future efforts. Sharing these lessons may also help attract new partners or strengthen community support.
• Evaluation can help you understand important aspects of the initiative. Suppose a free health screening is well attended while a monthly healthy-cooking program has low participation. Evaluation can help you explore why. Scheduling, transportation, location, childcare, cost, communication, cultural relevance, accessibility, or competing responsibilities might all influence participation. Evaluation can help explain not only what happened, but why.

When should you evaluate a community initiative?

Evaluation should take place throughout the life of an initiative.

When your plan is being implemented:

• Establish baseline information for the behaviors, conditions, experiences, or outcomes you hope to influence. Understanding where things began makes it easier to assess change over time.
• Examine the effects your work is having on participants and the broader community, including intended and unintended outcomes.
• Continue revising and updating action plans as new information becomes available.
• Keep the group focused on its goals while remaining responsive to community priorities. You may want to use a survey that assesses community goals and use the findings to refine priorities.

When some of your action plans are complete:

• Use evaluation findings to understand the initiative's outcomes, identify what should be sustained or adapted, and plan future work. For example, community-level indicators can help show whether broader community conditions are changing in ways related to the initiative.

How do you begin an evaluation?

Evaluation can seem complex when viewed as a single large task, but it can be broken into manageable steps. Begin by identifying what you most need to understand about your work.

Before moving into specific methods, keep one principle in mind: your evaluation should address questions that matter to community members, program participants, staff, partners, decision-makers, and funders. Time, funding, staffing, and available information may limit what you can examine, so focus on questions that will help your organization learn and make useful decisions. Every initiative is different, but the following questions are relevant to many community efforts:

• Who participated in the initiative, and whose perspectives may be missing?
• What programs, policies, practices, services, relationships, or community conditions changed?
• What changes occurred in people's experiences, behaviors, access to resources, or other relevant outcomes?
• What evidence suggests that your initiative contributed to those changes, and what other factors may also have influenced them?

Tips for implementing a successful evaluation

Build a commitment to evaluation.

Evaluation is most useful when the organization sees it as part of learning and improvement rather than simply as a requirement. Be clear about why you are evaluating the initiative, what you hope to learn, and how the findings will be used.

For example, if an initiative seeks to reduce tobacco use among young people, evaluation questions might examine changes in tobacco use, access to tobacco products, exposure to prevention efforts, enforcement of age-of-sale requirements, or other community and policy conditions that influence use.

Your evaluation should be grounded in your stated objectives or goals.
When you planned your initiative, you should have identified what you hoped to accomplish. Clearly defined objectives provide a foundation for deciding what information to collect and how progress will be assessed.
For example, if your objective is to reduce tobacco use among local high school students to 5% by May 2032, you can compare later estimates with that target while also examining trends, differences among groups, and other factors that may influence the result. Clear objectives make evaluation more focused, but they should not prevent you from noticing important outcomes that were not originally anticipated.

For each objective, identify indicators that provide meaningful and appropriate measures of progress.

Indicators should reflect what you are actually trying to change. Whenever possible, use multiple measures rather than relying on a single number to represent a complex issue.

For example, if your objective is to prevent violence involving young people in a local school or community, possible indicators might include violence-related injuries, reports of feeling safe at school, participation in conflict-resolution or mentoring programs, school attendance, access to supportive adults, reported incidents, and other measures of community safety and well-being. Arrest data may sometimes provide additional context, but they should be interpreted carefully because they also reflect reporting and enforcement practices.

If your objective is to increase access to heart-healthy foods, possible indicators might include the availability and affordability of fruits, vegetables, whole grains, and other nutritious foods; the number of stores or restaurants offering healthier options; participant-reported access; or changes in organizational food policies. Measures should reflect access and opportunity rather than assuming that individual choices alone determine health.

You need to collect information on each of these indicators.

Sometimes the information you need already exists. For example, library records might help evaluate efforts to increase borrowing, while transportation or public safety records might provide information about traffic conditions. In other cases, relevant information may not be available and you may need to collect it directly through observation, surveys, interviews, sensors, administrative tracking, or other appropriate methods.

Whenever possible, gather information from before and after implementation and, when useful, at several points in between. These data can help you determine whether meaningful change occurred. However, a simple before-and-after difference does not by itself prove that the initiative caused the change. Consider trends, outside events, differences among groups, and other factors when interpreting the findings.

Use the results to adjust the program or intervention as necessary.

Ask whether the initiative is moving toward its objectives and whether participants and communities are experiencing the intended benefits. If progress is limited, use the findings to understand why before deciding what to change. The issue may involve program content, outreach, accessibility, implementation quality, community conditions, or the objective itself.

For example, if a tobacco-prevention initiative is not producing the expected change among high school students, you might examine whether young people find the program relevant, whether cessation support is available, how social and commercial environments influence tobacco use, whether new nicotine products are affecting behavior, and whether policies and enforcement practices support prevention. Findings like these can guide changes that address both individual behavior and the broader conditions influencing it.

In Summary

Begin by asking, "What do we need to understand about our initiative?" Evaluation can then help you gather and interpret the information needed to answer those questions.

Evaluation should be an ongoing part of community work rather than something conducted only at the end. It can help you recognize progress, understand participant and community experiences, identify barriers and unintended effects, strengthen accountability, and make informed decisions about what to continue, adapt, expand, or discontinue.

The following sections in this chapter describe methods you can use to evaluate your initiative systematically and use what you learn to strengthen future work.