Table of Contents >
Chapter 38. Some Methods for Evaluating Comprehensive Communi... >
Section 2. Gathering Information: Monitoring Your Progress >
Gathering Information: Monitoring Your Progress |
Main Section | Contributed by Aimee Whitman and Eric Wadud Edited by Bill Berkowitz and Jerry Schultz |
What does it mean to monitor your progress?
Why do you need to monitor your progress?
How do you monitor progress?
What does it mean to monitor your progress?
Monitoring your community initiative can help you weigh your actions against the results to see if you are meeting the goals of the community and your initiative. In a sense, monitoring data helps you understand how well the initiative is functioning. That is, monitoring can help pinpoint where the actions of the initiative are not producing the desired effects. Additionally, the monitoring system can help you
- Better understand the initiative
- Make decisions concerning the programming of the initiative
- Promote awareness of accomplishments
- Recruit support
- Secure funding
Despite the scary sound of "monitoring system," you have probably already observed examples of monitoring in a variety of ways. Political candidates monitor the status of their campaigns by conducting polls and analyzing the results. Teachers monitor the progress of their students by giving tests at the beginning and end of the school year to see if they have mastered the secrets of long division. You might monitor your utility bills by keeping track of the monthly increases and decreases. Monitoring has a wide variety of applications. As a member of a community initiative, monitoring means a way of tracking major events and accomplishments of the initiative.
In Chapter 38, Section 1: Measuring Success: Evaluating Comprehensive Community Health, we introduced three key parts to the monitoring system. To quickly refresh your memory, these parts are:
- Process and outcome measures
- Observational system
- Regular feedback on performance
In this section, we'll explain what we mean by process and outcome measures, and the observational system. Later, in Chapter 39, Section 2: Providing Feedback to Improve the Initiative, we'll tackle ways to provide feedback to members of your coalition and your community.
Process and outcome measures
Process measures are the activities that take place during the initiative which help you determine how well things are going. Process measures can include many aspects of your initiative, such as:
- Members who participate: the number and type of participants, frequency of attendance, and turnover rate of the members.
Example: At the last general meeting of the initiative, 17 people attended. Of these folks, 10 were regular members, and 7 were sitting in on a meeting for the first time. - Planning products: written objectives, by-laws, or committees that contribute to the initiative.
Example: The action plan for the coalition was approved by the coalition and will be implemented immediately. - Media coverage: by radio, television, and print media.
Example: Several five minute radio spots describing one of your group's projects aired on a local FM radio station. - Financial resources: grants or donations. Financial resources also can include "in-kind" services, such as free advertising or products that an individual or business might offer instead of money.
Example: $8000 was received at a $50 per plate lunch that was held to raise money for local drug and alcohol abuse efforts. - Services that are ultimately provided: classes, programs, workshops, publications or other services or communications provided for the community by the initiative.
Example: Nutrition education workshops were conducted by child care providers. - Community actions: actions taken to encourage change in the community.
Example: Merchants were asked to display signs describing the penalty for selling alcohol to minors and the need for proper identification.
Later, you will incorporate process measures into event logs to better measure the progress of your group's work. We'll walk through an example of an event log later in this section.
While process measures document the specific methods you use to create change, outcome measures explain the overall impact that occurs as a result of these individual actions. Outcome measures highlight the changes that happen in the community as a result of the work done by your initiative. These include:
- Changes in programs, such as a new or modified service program.
Example: A parenting class was implemented by the initiative. - Changes in policies, such as a new or modified policy.
Example: A city ordinance was passed requiring owners of cigarette vending machines to place on every machine a sign that states "No cigarette sales to minors." The legislation was introduced at the urging of the Law Enforcement and Government Committee.
- Changes in practices, such as a new or modified practice.
Example: Merchants displayed signs describing the penalty for selling alcohol to minors and the need for proper identification.
As you can see, the monitoring system involves several components that will help your group determine how it's doing, what it's doing correctly, and what can be improved. Sound interesting? Read on for more!
Why do you need to monitor your progress?
Now you might be thinking to yourself, "You want me to gather information about our initiative. But what will I do with all of those numbers and comments? Why is this important for the success of our group?" Don't worry; the data you gather by using the monitoring system can help the group in a variety of ways.
- Data can tell you where the initiative places its emphasis. For example, the monitoring system might reveal that your initiative focuses on services, rather than change. If change is what you want, the monitoring system will help you detect this at an early date.
- Data can point out which groups in the community are affected by your initiative. Is your initiative producing a lot of change in the schools, but little change in the criminal justice system? Is this what you and the community want and need? Who is being targeted? With the results of the monitoring system, you might be able to better answer these questions.
- Data can reveal which strategies are being addressed. The monitoring system can determine whether your initiative is offering information without following through with peer support or access to other resources.
- Data can be used by the staff to achieve a variety of results. Staff and leadership can use the data to promote community awareness of the initiative's activities and accomplishments, recruit community support, and secure financial resources.
How do you monitor your progress? A step-by-step guide
Like many aspects of building a community initiative, the monitoring system involves several important steps. Before you can start analyzing your data, you first have to collect it.
1. Data can be collected by completing event logs and other forms on a regular basis. Event logs are written accounts of the major activities of the initiative. They might also be used to record any changes in the community brought about by the initiative, such as new programs, policies, or practices related to the initiative's goals and mission.
The event log might include important information such as:
- the month/day/year of the event
- a description of the event itself, including
- why it was important
- what happened as a result
- a description of the details of the event, including
- who was involved
- what organizations contributed people and resources
- what community sector or objective this relates to
- if this is the first time this event happened
| Example A group called the Adolescent Pregnancy Prevention Initiative in Franklin County, Kansas, participated in an event to help bring about changes in the community that are related to reducing adolescent pregnancy. At the next coalition meeting, Maggie B., a coalition member who helped organize the event, completed the information in the event log. She wrote: "Shannon P. and Gary M. participated in a meeting at Ottawa High School on February 14, 1996. They met with the high school principal to get permission to organize a Teen Action Committee in the school. The Teen Action Committee was approved by the principal. This was important because it will create leadership opportunities for youth." Maggie recorded this information in a chart where she identified details such as the month, day, and year of the event, a description of the event itself, and a description of the details of the event. The chart she made is shown as an example at the end of this section. |
2. These event logs can be completed by several people, such as key project staff and active members of the initiative. The people who fill out logs will be those members who are taking action on behalf of the initiative. Ideally, everyone who is doing something for the initiative will complete event logs. Event logs should be completed no later than one week after the event took place. That way, the important details will be fresh in everyone's mind! The completed event logs can then be given to the people in the group who are in charge of evaluation.
3. We've found that data collection in own work takes about two hours to a week to complete. Of course, if your initiative is working on a particularly involved project one week, you might spend twice or even three times that amount of time gathering data. Carefully gathering your information will mean that the results that are generated will be more accurate, and therefore more valuable to your initiative.
4. Once the information has been gathered, it needs to be turned over to the people in your group who will put the data into some kind of organized form so that the initiative can use it in a beneficial way. This step in the process might take a good deal of time; but, don't be scared! As you have heard many times already, the benefits you receive from the evaluation process will only be helpful if you spend the time to carefully analyze the results. While this might take a few hours, in the end, you'll be glad you did it!
5. Those who organize the information that is gathered will then take time to summarize the event logs. The results will then be distributed to members of the group. For details on feedback, see Chapter 39: Using Evaluation to Understand and Improve the Initiative.
Congratulations! You've just finished learning one of the most important ways to evaluate a community initiative. Keep reading to see examples of event logs, and other monitoring charts. But don't stop there! More exciting sections await you and your initiative. The next chapters will introduce the constituent survey of goals, process, and outcomes, and other methods to measure the progress and success of your initiative. Remember, monitoring your initiative helps your group be the best that it can be!
We encourage reproduction of this material, but ask that you credit the
Community Tool Box: http://ctb.ku.edu
Resources
Internet Resources
The Magenta Book - Guidance for Evaluation
This resource provides an in-depth look at evaluation. Part A is designed for policy makers. It sets out what evaluation is, and what the benefits of good evaluation are. It explains in simple terms the requirements for good evaluation, and some straightforward steps that policy makers can take to make a good evaluation of their intervention more feasible. Part B is more technical, and is aimed at analysts and interested policy makers. It discusses in more detail the key steps to follow when planning and undertaking an evaluation and how to answer evaluation research questions using different evaluation research designs. It also discusses approaches to the interpretation and assimilation of evaluation evidence.
Print Resources
Fawcett, S. B., Paine-Andrews, A., Francisco, V.T., Schultz, J.A., Richter, K.P., Lewis, R.K., Williams, E.L., Harris, K.J., Berkley, J.Y., Fisher, J.L., & Lopez, C.M. (1994). Work group evaluation handbook: evaluating and supporting community initiatives for health and development, Lawrence, KS: Work Group on Health Promotion and Community Development, University of Kansas.