Table of Contents >
   Part J. Evaluating Community Programs and Initiatives
      Chapter 38. Some Methods for Evaluating Comprehensive Communi... >
         Section 4. Rating Member Satisfaction >
             Main Section - Introduction, what, why, when, who, and how. >

Rating Member Satisfaction

  

Main Section

 Contributed by Aimee Whitman Edited by Bill Berkowitz and Jerry Schultz

What is the member survey of process?

Why should you use the member survey of process?

When should you use the member survey of process?

How to conduct the member survey of process



What is the member survey of process?


How many times have you visited a restaurant and seen a mini-questionnaire tucked between the salt and pepper shakers on your table? These surveys usually ask you to rate the service, the food, the atmosphere, and the overall quality of the meal.

With that in mind, picture this scenario. Last weekend, you went to a fine Italian restaurant to celebrate a friend's birthday. When you arrived, everything from the simple yet beautiful candles on your table to the friendly service met or surpassed your expectations.

But maybe you wished the waiter or waitress would have been better informed about the evening's specials, or about the different varieties of wine. After finishing your meal, you filled out one of the surveys where you offered suggestions for improvement while still complimenting the many enjoyable aspects of the meal. If the restaurant truly wants to please its customers, it will look at your comments and try to incorporate them into its operation.

With the member survey of process, groups ask members of the group to evaluate their "service" in the hopes of improving weak spots and continuing on with those methods that are proving to be successful. In other words, the member survey of process asks members of the group to indicate their satisfaction with the group's day-to-day operations. Like the restaurant owner who wants to please her customers, a community group can use the survey of process to better meet the needs of the community.

You may be asking yourself, "But, how is this different from the member survey of goals?" (For more information on the member survey of goals, see Chapter 38, Section 3: Rating Community Goals) Although the two sound similar, they are actually quite different.

  • The member survey of goals asks community members to respond to action plans during the planning stages of the group.
  • The member survey of process gathers opinions from members of the coalition once it has begun work in the community.

The survey of process might ask questions about such areas as planning, leadership, services, community involvement, and progress towards accomplishing goals.


Why should you do this type of survey?


The importance of the survey of process

The survey of process reminds members of your group that you care about their ideas, suggestions, hopes, and dreams. If members feel like they can actively participate in all aspects of the group--including sharing their feelings of satisfaction or disappointment--their desire to devote time and energy to the group will probably increase. In this way, both your group and its members benefit from the member survey of process. While your group learns more about its strengths and weaknesses, members feel like an important and necessary part of the work that you do.


When is the right time to start?


Timing the member survey of process

We recommend that this survey be done annually. That way, you can continually receive the input of your members concerning the work done by your group. Our experience with coalitions is that this often takes a real time commitment from staff, and that the amount of time that will be devoted to this survey will vary depending upon the nature of the group.


How to conduct the member survey of process


Like many other aspects of evaluation, the member survey of process involves several steps that can easily be broken down into manageable parts. The four steps are:

  • develop a survey;
  • obtain the ratings;
  • assess satisfaction with different aspects of the group;
  • use the data to improve the functioning of the initiative.

1. Develop a survey

You'll want to develop a survey that will help you understand the level of satisfaction that members of the community feel towards your initiative. For this, you'll need some members of the coalition to propose a survey form and help design the survey. You might hold a small meeting of those who might be interested in developing a survey and brainstorm possible areas for questions. Another thing you could do is send out a preliminary draft of the survey to your most active members and ask them to comment, add, and take out questions.

Develop the questions which you will want to ask on your survey. You will want to prepare the questions in such a way that you can determine the satisfaction and the overall approval with the different aspects of the group.

You might ask questions concerning both planning and implementation and leadership.

Example

One of the questions under planning and implementation might read:

Concerning the follow through of the group's leaders on various projects, I felt...

a) very dissatisfied
b) somewhat dissatisfied
c) somewhat satisfied
d) satisfied
e) very satisfied

If a respondent to the survey circled "e" for this question, your group would know that this individual's satisfaction was high for this particular area of the group. Then, after having looked over the entire survey, you might see that the respondent answered "d" or "e" for every question. This would lead you to believe that his or her overall approval for the coalition's activities was strong.

Try to come up with questions that assess satisfaction with the following aspects of the group:

  • planning and implementation;
  • leadership;
  • services;
  • community involvement with the group;
  • progress and outcome.

2. Obtain the ratings.

Your survey might address any or all of the above issues. Depending upon the nature of your group, you might feel like you could use more input in one area than another. In any case, you should cater your survey to meet the needs of your group. At the end of this section is a sample survey to help you better understand the ways to break down the questions.

After the surveys have been written, members of the group can distribute them to all the other members of the group. Sometimes this can be done through the mail, but there are other options depending upon the size of your group and what works best. One option would be to distribute them at meetings and have a "drop off" box at later meetings and events.

3. Assess satisfaction.

Once the respondents have completed and returned the anonymous surveys to the evaluation team, you can begin to look at the information to determine what changes might need to happen in the structure of the group. One way to help accomplish this includes tabulating the results of the survey.

4. Use the data to improve the functioning of the initiative.

Once again, the most brilliant research in the world won't help anyone if it is not used to make changes for the good of the coalition and of the community. Because this survey focuses on the thoughts and ideas of members, it is crucial that these members know that their suggestions are being heard and used by the group. For more ideas on feedback, see Chapter 39: Using Evaluation to Understand and Improve the Initiative.

To help complete the member survey of process, feel free to use and adopt example of the sample cover letter, a sample survey, and a sample collection of results. Hopefully, these samples will help you create your own successful survey.

Now, get ready for just one more survey: the member survey of outcomes. We'll discuss it in further detail in Chapter 38, Section 5: Constituent Survey of Outcomes: Ratings of Importance. Then, while you are taking a breather, pat yourself on the back for the great work you've done as you learn about evaluation your group!



We encourage the reproduction of this material, but ask that you credit

the Community Tool Box: http://ctb.ku.edu



Resources

Berkowitz, W. Community impact: creating grassroots change in hard times. Cambridge, MA: Schenkman Publishing Company, Inc. 1982.

Cox, F. and eds. Tactics and techniques of community practice. Itasca, IL: F.E. Peacock Publishers, Inc. 1984.

Fawcett, S. B., Paine-Andrews, A., Francisco, V. T., Schultz, J. A., Richter, K.P., Lewis, R.K., Williams, E.L., Harris, K.J., Berkley, J.Y., Fisher, J.L., & Lopez, C.M. (1994). Work group evaluation handbook: evaluating and supporting community initiatives for health and development, Lawrence, KS: Work Group on Health Promotion and Community Development, University of Kansas.

Fetterman, D. and eds. Empowerment evaluation: knowledge and tools for self-assessment and accountability. Thousand Oaks, CA: SAGE Publications, Inc. 1996.

Francisco, V. T. and Wolff, T. (199 ). Evaluating coalition efforts. Amherst, MA: AHEC Community Partners.

Pietrzak, J., Ramler, M., Renner, T., Ford, L., and Gilbert, N. Practical program evaluation: examples from child abuse prevention. Newbury Park, CA: SAGE Publications, Inc., 1990.

Rutman, L., ed. Evaluation research methods: a basic guide. Newbury Park, CA: SAGE Publications, Inc. 1984.