Evaluating community health initiatives in Kansas (U.S.A.)
This example talks about a composite case study that draws on work with several community health initiatives in Kansas with which our Center for Community Health and Development at the University of Kansas had evaluation and support responsibilities. Their missions included preventing teen pregnancy, preventing teen substance use, reducing risks for cardiovascular diseases and some cancers, and promoting health in rural communities. All of the initiatives had the aim of improving community members' abilities to better their quality of life.
Background and context
A variety of communities participated in these initiatives with grant support from the Kansas Health Foundation. Initiatives were located in urban areas, communities with military bases and prisons, and rural communities.
In each community, the local partnership attempted to be a catalyst for change. The initiatives involved community members in deciding how to change programs, policies, and practices to address local concerns. Task forces and action committees developed the initiatives' action plans.
Who was involved?
Participants in these initiatives included community members, service providers, community leaders, youth, and parents. They involved people affected by the problem and community leaders who had the ability to bring about changes. Members included targets of change, such as youth or elected officials whose actions (or inaction) contributed to the problem, and agents of change, such as religious leaders or peers, who could contribute to the solution. Other players included members of the steering committees and staff of the initiatives; program officers representing the grantmaker; and evaluation and support staff of the Center for Community Health and Development at the University of Kansas.
Stakeholder interests for the evaluation included understanding whether the efforts were having an effect and using this information to improve the initiative and its projects. Major stakeholders posed several questions to be addressed by the community evaluation:
- Is the initiative making a difference?
- Is the initiative serving as a catalyst for change?
- How is the initiative distributing its efforts?
- What makes a difference in how well the initiative functions?
The evaluation system used different measurement tools to address these and other questions.
Implementation of the model
Here, we'll look at how the catalyst and logic models we described in the section played out in these communities in Kansas.
Supporting collaborative planning
In the rural health initiative, our KU Center for Community Health and Development collaborated with rural communities in using concerns surveys, listening sessions or town meetings, and statistical information relevant to local concerns.
First, the concerns survey allowed community members to rate the importance of different health issues, such as teen drug abuse or water quality, and to say how well they thought each of these issues was being addressed in their community. Data were summarized to highlight major strengths and relative problems in the community. The issues chosen by each community varied, and included promoting good health among older adults, preventing substance use, and preventing teen pregnancy.
Second, the KU Center for Community Health and Development helped community members conduct listening sessions. In these informal focus groups, members of the community came together to define issues, identify barriers and resources, and brainstorm solutions. Third, statistical information was obtained by the Center for Community Health and Development to determine if what community members perceived as problems were confirmed by statistical records.
The Center's supported action planning by community members. Staff helped each community initiative to form a vision, create a mission, set objectives, and develop strategies and action steps. It's important to recognize that achieving consensus on what to do in the face of many problems and without always having all of the necessary information (or resources) involves artistry. Conflicts may arise when what people care about isn't supported by available "hard" data. For example, community members may feel strongly about working to end child violence in the community, even if statistics show that many other problems occur more often.
People throughout the community were encouraged to participate so that changes could be brought about throughout the community. The Center developed practical guides to support action planning for teen substance use, teen pregnancy, youth violence, chronic disease, child abuse and neglect, and health promotion for older adults.
Documenting community implementation, action, and change
The KU Center for Community Health and Development worked with members of the initiatives to document the changes in the community that were facilitated by each community partnership. Ratings of importance and feasibility by community members and outside "experts" helped inform members of the initiative about the importance of discrete community changes. Graphs and regular reports of accomplishments were used to inform stakeholders about the progress on intermediate outcomes. The Center also helped develop ways to get the word out about the initiative's accomplishments, and provided information about how to incorporate such information in status reports and future grant applications.
Implementation of initiatives was also documented. For example, with the teen pregnancy prevention initiatives, sexuality education was tracked using teacher reports and records. The Center also worked with the initiatives to determine if particular changes brought about by the initiatives were effective in reducing risks for the problem. For example, researchers and members of one initiative studied whether adjustments in physical education requirements improved the physical fitness of school children.
Assessing community adaptation, institutionalization, and capacity
The KU Center for Community Health and Development helped make necessary adjustments to important parts of the initiative. For example, communities working to prevent teen pregnancy adapted specific elements of an approach first implemented in South Carolina. To provide support and education' communities implemented a variety of different programs, including male-to-male support groups, abstinence clubs, and social events. Institutionalization was examined by looking at whether new programs, policies, and practices were still in place some months after they began. Community capacity was studied by looking at the pattern of community changes over time.
Evaluating more distal outcomes
The KU Center for Community Health and Development obtained information from behavioral surveys and archival records to help determine if changes in the environment were associated with corresponding changes in the bottom line. For example, the initiatives working to prevent teen substance use used school surveys to determine youths' reported use of alcohol, tobacco, and other drugs. The Center also gathered data on outcomes such as the rate of teen pregnancy. Finally, staff examined possible relationships between rates of community change and changes in the bottom line to help determine if the initiative had a real impact on its long-term goals.
Promoting dissemination
Center for Community Health and Development staff used local and national presentations, journal articles, and book chapters to get the word out about what works, what makes things work, and what doesn't appear to work. Staff also prepared handbooks for guiding action planning and community evaluation, and conducted workshops and training sessions. Finally, the Center team helped develop this website -- The Community Tool Box -- to share information about promising practices for improving our communities.