Bookmark and Share

Table of Contents >
   Part J. Evaluating Community Programs and Initiatives
      Chapter 38. Some Methods for Evaluating Comprehensive Communi... >
         Section 9. Gathering and Using Community-Level Indicators >
             Main Section - Introduction, what, why, when, who, and how. >

Gathering and Using Community-Level Indicators

  

Main Section

 Contributed by Aimee Whitman Edited by Bill Berkowitz and Jerry Schultz

This section is based on an article in the Work Group Evaluation Handbook: Evaluating and Supporting Community Initiatives for Health and Development by Stephen B. Fawcett, Adrienne Paine-Andrews, Vincent T. Francisco, Jerry Schultz, Kimber P. Richter, R.K. Lewis, E.L. Williams, K.J. Harris, Jannette Berkley, Jacqueline L. Fisher, and Christine M. Lopez of the Work Group on Health Promotion and Community Development, University of Kansas, Lawrence, Kansas.


What are community-level indicators?

Why should you use community-level indicators?

When should we evaluate community-level indicators?

How do we find community-level indicators?



What are community-level indicators?


Imagine for a few moments that you play basketball for a Division I college. Despite the presence of some solid players, your team finishes with an average record, and does not make any of the brackets come NCAA tournament time.

Next summer, the team practices long and hard on defense, offense, and free-throws. Then, during regular season play, everyone plays consistently well, and you match the numbers you had hoped to reach at the beginning of the season. And, come March, your team earns a spot in one of the brackets to compete in the Big Dance.

In this scenario, you might say that one of your team objectives when you began the next season was to start hitting 50% from the field and to shoot 75% from the charity stripe. The improvement of the team as a whole would be equivalent to a community-level indicator.

For community initiatives, community-level indicators can be seen as one ultimate outcome of any initiative. While the monitoring system, constituent surveys, goal attainment reports, and interviews can help pinpoint the success of individual events, community-level indicators try to determine how these individual events affect the community as a whole.

One way to discover the big picture effect is to review proven "indicators" recommended by experts in your group's field of interest. For a group focusing on substance abuse, for just one example, one indicator might be the number of emergency medical transports related to alcohol use. By examining these numbers, your group will have a better sense of the ultimate effect of your group's work on the community at large.


Why should you use community-level indicators?


While the other evaluation tools can offer you valuable insights into the effectiveness of specific actions and objectives of the initiative, community-level indicators provide the major "big picture" perspective. But the benefits don't stop there!

  • Community-level indicators provide "bottom-line" evidence of the impact of the initiative.

    "Bottom-line" evidence describes the ultimate effect of your group's work on the initiative. Community-level indicators give you an objective way to measure and assess the bottom-line evidence. Examples might include single-nighttime vehicle crashes (as a way to track efforts to reduce substance abuse) or the number of hospital admissions for violence-related injuries as a way to account for efforts to reduce youth violence). In addition, community-level indicators help you measure progress towards the goal. Measurements are ideally taken at multiple stops along the road, so that you (and the community) can see how you are doing and adjust your course, if need be.

  • Community-level indicators help determine the effects of key components of the initiative.

    If you decided to implement several bold ideas in your action plan, community-level indicators can help you determine the effectiveness of these ideas, and of specific objectives. That way, you can see which ideas and strategies helped you achieve some of your goals, and which might need some reworking in a future action plan.

  • Community-level indicators help push issues to the forefront of the public agenda.

    If the community-level indicators reveal negative results (i.e. the number of single nighttime-vehicle crashes stays the same even after you have implemented several of your objectives), this can be the perfect time to remind the community that the issue still exists. It still needs to be addressed through continued action. Although negative results may not be the results you want to see, try to envision them as a golden opportunity to push your issues to the forefront of the public agenda.

  • Community-level indicators that show positive results can help secure more support for the initiative.

    But, if your community-level indicators suggest that your work has been resulting in many positive changes for the community, you can use these glowing reports to gain more support for your initiative and the good work that you do. This might include financial support, which can be used to improve and extend the work of the initiative. So get out there and tell it to the world; your group deserves recognition and support for all of its hard work and contributions!


When should we evaluate community-level indicators?


The sooner you start to evaluate your community-level indicators, the sooner you'll get a better understanding of how your community is affected by different variants. This way, you can plan in advance so that the effects are the ones that you desire.

It's advisable to start measuring community-level indicator as soon as you start a new project or event, so that you can change things around as necessary. To get a broader picture is always valuable and the sooner the better.

Of course, after you conclude your event or project, run a detailed community-level indicator survey so that you can measure the results and effects of your activity, and learn more for the future.


How do we find community-level indicators?


It's time to dip your toes in the crystal blue waters of community-level indicators. Soon you'll be swimming with the big fish as you navigate the waves of availability, accuracy, feasibility and sensitivity. Let's dive in!

There are five main steps to evaluating community-level indicators. They involve:

1. Selecting indicators that meet the criteria of being:

  • Available;
  • Accurate;
  • Possible to collect;
  • Relevant to the initiative.

2. Securing data from relevant local and state agencies

3. Summarizing and graphing the data

4. Presenting the data to community leadership, trustees, and funders

5. Using the data to push the issue on the public agenda and to redirect the initiative's efforts

Now, let's get our feet wet!


1. Select indicators that are available, accurate, possible to collect, and sensitive to the initiative.

The key to success here starts with knowing the right questions to ask. First, your group will want to hold several meetings with staff and also with groups of community members to identify which community-level indicators will best serve your needs.

In the Examples section, you will find a variety of recommended community-level indicators for:

  • Substance abuse coalitions
  • Adolescent pregnancy prevention initiatives
  • Tobacco control initiatives
  • Injury control initiatives
  • Violence prevention initiatives

Let's say, for example, that your group wants to focus on reducing the rate of teenagers who smoke marijuana in your community. One of the community-level indicators includes the number of juvenile arrests for drug offenses. (See Examples)

2. Contact relevant local and state agencies to find data.

Once you have decided upon the appropriate community-level indicators, members of the staff can pool their knowledge of local, regional, state, and federal sources who might be good resources for information. These sources might include health departments, law enforcement agencies, or transportation offices. Your exact choices will be determined by what you want to study.

Let's continue on with the above example. If you know that the community- level indicator you want to measure is the number of juvenile arrests for drug offenses, you could contact the local police department. Then, gather information about the number of arrests for drug offenses for community members between the ages of 14 -20 over the last several years.

3. Compile, summarize, and graph the data.

Members of the evaluation team put together all of the crucial information in a report, and also in graph, chart or table form.

How do we graph the data from the number of drug arrests? One way would be to graph the numbers of arrested people at six-month intervals. Then, you can compare your group's presence in the community with the changes in the numbers. If, for example, you began implementing your action steps one year ago, you would want to compare the levels of drug arrests before your work with the levels during and after your presence.

4. Present the data to community leadership, trustees, and funders.

Data obtained from the community-level indicators can then be used to inform and educate your organization and the community. As a tool to encourage public support, community-level indicators can highlight the positive results of your group. Feel free to spread the word far and wide about the results you have obtained. Post your information everywhere; let the community know what is going on in their world.

5. Use the data to elevate the issue on the public agenda, and to redirect the initiative's efforts.

If the community-level indicators yield negative results, this can provide a chance to push the issue to the top of the public agenda. Additionally, data, regardless of the results, can give the group a chance to redirect its work in a more effective direction.

If, for example, your research shows that levels of drug arrests among juveniles continues to hold steady, this would be an excellent opportunity to remind the community that drug abuse exists and continues to be a significant issue in the community. In this case, you might want to distribute your results to local schools, local community leaders, and other groups or organizations that focus on substance abuse.

Finally, remember that results can be viewed in a variety of ways. Positive results might mean that your group needs to redirect its energy towards other areas if you have achieved your original goals. Negative results should not be viewed as failure. Instead, these results should simply mean that another strategy might work better. Then, rather than continuing to focus your energy in the wrong direction, you can try to find a more effective way to work towards change.


To sum it up


You deserve a round of applause! You have successfully worked your way through all of the evaluation steps! Good work! Keep reading to find some helpful examples of community-level indicators. Then, prepare to enter the wonderful world of Feedback! (Want to know more? Check out Chapter 39: Using Evaluation to Understand and Improve the Initiative!)

 



We encourage the reproduction of this material, but ask that you credit

the Community Tool Box: http://ctb.ku.edu



Resources

Berkowitz, William.(1982) Community impact: creating grassroots change in hard times. Cambridge, MA: Schenkman Publishing Company, Inc.

Cox, Fred and eds. (1984) Tactics and techniques of community practice. Itasca, IL: F.E. Peacock Publishers, Inc.

Fawcett, S.B., Paine-Andrews, A., Francisco, V.T., Schultz, J.A., Richter, K.P., Lewis, R.K., Williams, E.L., Harris, K.J., Berkley, J.Y., Fisher, J.L., & Lopez, C.M. (1994). Work group evaluation handbook: evaluating and supporting community initiatives for health and development, Lawrence, KS: Work Group on Health Promotion and Community Development, University of Kansas.

Fetterman, David and eds. (1996) Empowerment evaluation: knowledge and tools for self-assessment and accountability. Thousand Oaks, CA: SAGE Publications, Inc.

Francisco, V. and Wolff, T. (1990). Evaluating coalition efforts. Amherst, MA: AHEC Community Partners.

Pietrzak, J., Ramler, M., Renner, T., Ford, L., and Gilbert, N. (1990) Practical program evaluation: examples from child abuse prevention. Newbury Park, CA: SAGE Publications, Inc.

Rutman, Leonard, ed. (1984) Evaluation research methods: a basic guide. Newbury Park, CA: SAGE Publications, Inc.


Materials

Healthy People 2000: National Health Promotion and Disease Prevention Objectives
Superintendent of Documents
Government Printing Office
Washington, DC 20402-9325
(202) 783-3238