Chapter 40. Maintaining Quality Performance | Community Tool Box

• Why is quality important to you?

• What are the basic principles of TQM?

• What is TQM and its relevance to your organization?

• How do you achieve quality performance, using TQM and other principles?

• How do you maintain quality performance?

• The Fourteen Points

W. Edwards Deming, a statistician and management thinker who influenced American industry during World War II and Japanese industry afterward, developed management principles centered on quality and continual improvement. These principles encouraged flexible, dynamic systems that involved people throughout an organization in producing goods and services that met customer needs, performed reliably, and were continually improved. His ideas are often associated with Total Quality Management (TQM), and they contributed to a number of related management approaches and innovations around the world.

Why is quality important for you?

Exactly what does quality mean in the context of advocacy, community development, health, or human service organizations or initiatives?

A quality program:

• Responds as effectively as possible to the needs it was designed to meet
• Is totally consistent with the mission and philosophy of the organization or group carrying it out
• Is responsive to the priorities, needs, strengths, and cultural contexts of the people and communities involved
• Is a model of ethical behavior

But why is quality important for a grassroots or community-based organization?

• Quality makes a group more effective at addressing the priorities and needs it is working on
• Quality adds strength and credibility to your organization or initiative
• Ethically, organizations should strive to provide the highest quality of service, support, or advocacy they can
• Investing in quality can reduce avoidable problems, strengthen effectiveness, and make better use of resources over time

Developing a "culture of quality" can have a number of positive effects on your organization itself

• When staff members and volunteers see that they and the organization are doing high-quality work, it can strengthen morale, confidence, and commitment to the organization
• Striving for quality helps to develop organizational and individual competence, thus continually improving the organization
• A quality program continually improves its performance and service delivery, which can strengthen organizational credibility and better serve participants, community members, and partners

What are the basic principles of TQM?

(Much of the following discussion is based on material contained in Introduction to Total Quality: Quality Management for Production, Processing, and Services, 2nd Edition. Full source citation can be found under Resources.)

There are some basic assumptions that underlie the idea of TQM. In this section, we'll look at how they might relate to your organization or initiative.

Key elements of total quality

• Customer Focus: In TQM, the needs of the customer are the starting point for quality. In community-based work, however, the people most directly affected by an issue are more appropriately understood as participants, partners, community members, or constituents rather than simply as "customers." What priorities, needs, strengths, and experiences should guide the work? How can the organization respond effectively and respectfully while involving community members in shaping programs, services, and decisions?
• Commitment to Quality: Quality should be considered from the beginning and built into everything a business or organization does. Careful planning, monitoring, evaluation, reflection, and adjustment are all important. Quality is strengthened not only by identifying problems after they occur, but also by designing systems that prevent avoidable problems and support learning when challenges arise. Everyone in the organization should understand and contribute to this commitment to quality.
• Continual Improvement of Systems: The work of an organization should be viewed as an ongoing process of learning and improvement. Programs and initiatives should adapt as community priorities, needs, strengths, circumstances, and available evidence change.
• Unity of Purpose: In order for quality to be achieved, everyone in an organization or business has to work together toward common goals. That means mutual support throughout the organization, not turf battles, not jealousy, not unnecessary competition. All interactions among people in the organization should be mutually helpful and aimed at achieving the best possible performance of the organization as a whole.
• Teamwork: Working collaboratively can strengthen connections among colleagues, distribute responsibility, bring together different perspectives, and improve results. Effective teamwork also creates opportunities for shared learning and support.
• Staff Involvement: If everyone in an organization is expected to contribute to quality performance, staff members should have meaningful opportunities to shape how the work is done. People need enough autonomy and support to do their jobs effectively, and their knowledge, perspectives, and ideas should be respected and taken seriously.
• Education and Training: Achieving quality requires ongoing learning for everyone in an organization, and that learning should be part of the organizational culture. Staff members can learn from colleagues, community partners, participants, research, training, workshops, courses, and other organizations. Ongoing learning can strengthen skills and bring new ideas and perspectives to the work.
• Evidence-Informed Approach: For grassroots and community-based organizations, this means drawing on the best available research, practice-based knowledge, community knowledge, lived experience, and lessons from others to design and improve programs or initiatives. Using multiple forms of evidence is more likely to support effective, appropriate work than relying on assumptions or intuition alone.

The founders of a Massachusetts community-based adult literacy program, with backgrounds in both developmental psychology and reading theory, based their program on the best available research in both areas. They worked to align the educational and support elements of the program and trained staff accordingly. Initially, because the program used an approach that had not been widely applied with adults, it was strongly criticized by others in the field. Critics argued that the program should use a stricter phonics-based approach and give less attention to students' concerns and community issues. Over time, however, the program maintained a low dropout rate and strong student outcomes, and others began experimenting with similar ideas. Fifteen years later, the program had become a model for the state while continuing to adapt in response to student feedback, changing needs, and new research findings.

• Long-Term Commitment: Quality requires sustained attention. Organizations should continue learning and improving even after reaching a strong level of performance. Rather than treating quality as a fixed endpoint, view it as an ongoing commitment to effectiveness, responsiveness, equity, and learning.

The Deming Cycle

The assumptions above underlie the "Deming Cycle," which is really a process for creating and selling a quality product. We'll revisit the Deming Cycle later to examine how it can be used in an advocacy, community development, health, or human service context.

• Plan - conduct consumer research and use it for planning the product
• Do -  produce the product
• Check - check the product to make sure it was produced in accordance with the plan
• Act - market the product
• Analyze - analyze how the product is received in the marketplace in terms of quality, cost, and other data

What is TQM and its relevance to your organization?

It is important to note that the principles of TQM were designed for the business sector, so while some aspects are relevant to organizations concerned with advocacy, community development, health, and human services, other aspects may be contrary to the goals of those organizations.

Some elements of TQM that would work toward quality in any environment include:

• The need for careful planning, monitoring, evaluation, learning, and adjustment
• Teamwork, shared decision-making, and meaningful participation throughout the organization
• Constant education and training for all staff
• Attention to the priorities, needs, strengths, and outcomes of community members and participants
• Identifying and changing what doesn't work well
• Encouraging and rewarding, rather than discouraging, new ideas
• Developing an organization-wide culture of quality
• Keeping at it over the long term

Some elements which may not work toward quality for your organization are those that assume that the goal is the success of a business, such as:

• The emphasis on products and production, which may take the focus off the human needs and consequences your organization is concerned with
• The assumption of a hierarchical structure where those in authority control when and how others participate in decisions, and where leadership always comes from the top, which may conflict with collaborative or community-centered ways of working
• The definition of everyone as either a supplier or a consumer/customer, which may not fit grassroots and community-based work where people may also be participants, partners, leaders, contributors, and decision-makers

How do you achieve quality performance, using TQM and other principles?

Using the Deming Cycle while keeping some of the basic TQM principles in mind can help you design, deliver, refine, and maintain an effective program or initiative.

Plan

In Deming's original business framework, the Plan stage involves consumer research and product planning. In community-based work, this can be adapted as a collaborative assessment of community priorities, needs, strengths, resources, and context, followed by planning with the people and groups who will be affected by or involved in the initiative.

Thus, the "Plan" part of the cycle might include the following:

• Conducting a community assessment that examines needs, strengths, assets, and resources and meaningfully involves people affected by the issue
• Identifying desired outcomes collaboratively from the perspectives of community members, participants, the organization, partners, and the broader community
• Determining ways to reach those outcomes that are feasible, consistent with the organization's guiding principles, inclusive and equitable, and responsive to the priorities, strengths, cultural contexts, and experiences of the people and communities involved
• Developing indicators that can help show progress toward desired outcomes and identify meaningful milestones along the way
• Creating meaningful opportunities for people affected by or involved in the work to participate in the development of the plan

Do

In Deming's original framework, the Do stage focuses on producing the product. In community-based work, this stage involves designing and implementing the program, outreach effort, service strategy, advocacy activity, or other action identified during planning. The work can draw on relevant TQM principles—such as teamwork, staff involvement, evidence-informed practice, commitment to quality, and responsiveness to participants and community members—along with community organizing and engagement principles.

The following are important elements of designing an effective program:

• Finding out what has already been tried in the community, and how well it worked
• Learning whether past experiences with certain methods, approaches, organizations, or individuals created concerns or mistrust that should be understood before similar approaches are proposed again
• Using as examples other communities that have successfully mounted similar programs, while remaining aware that not everything that works in one place will work in another
• Consulting relevant research, practice-based evidence, community knowledge, and lived experience to understand what has worked in similar situations
• Meaningfully involving people affected by, participating in, and carrying out the program or initiative in its development
• Taking care of the logistics: accessible space or platforms, equipment and supplies, appropriate staffing and volunteer capacity, and other resources needed for implementation

Check

Review the program or initiative to determine whether it aligns with the plan and with what was learned during community assessment and engagement. Does it reflect identified priorities, strengths, and needs? Is it likely to support the desired outcomes in appropriate ways? Is it inclusive, equitable, accessible, and feasible? Were people affected by the work meaningfully involved in its development? Is it ready to move forward?

Act

In Deming's original framework, this stage involves bringing the product to market. In community-based work, it means putting the planned program or initiative into action.

If it's going to work well, there are some non-TQM standards that need to be applied:

• Everyone involved should understand the process that led up to this program, as well as the philosophy, concept, and workings of it
• Everyone involved should understand their responsibilities and have the support, resources, and accountability needed to carry them out. Mistakes and challenges are inevitable; the goal is to create conditions where people can learn from them, address barriers, and follow through on agreed-upon responsibilities.
• Strong planning must be matched by effective implementation, clear roles, adequate resources, supportive working conditions, and people with the knowledge and skills needed to carry out the work

Analyze

Analyze how the product is received in terms of quality, cost, and other data. Analysis in this context - looking at what you're doing, evaluating it, and trying to improve it - needs to be conducted on the basis of the original plan, with discussions among participants, staff, and others.

Consider:

• Does the program or initiative address the priorities, needs, strengths, and opportunities identified through assessment and engagement? Have those priorities or circumstances changed since the original assessment?
• Does the program or initiative reach, or help participants and communities move toward, the desired outcomes? Are those outcomes still appropriate and meaningful, or should they be revised? The indicators you developed can help you examine both questions.
• Is the plan feasible? Can the program or initiative be carried out with the time, resources, staffing, and capacity available? Is it accessible to participants? Are staff and volunteers able to do their work without unreasonable workloads or expectations? Is the program or initiative trusted, supported, or viewed as useful by community members and partner organizations?
• Is the program or initiative consistent with the vision, mission, philosophy, and guiding principles of the organization, and are those still appropriate? Do its internal practices—such as working conditions, shared decision-making, and relationships among staff, participants, volunteers, partners, and community members—reflect the values and changes the organization hopes to support in the broader community?
• Is the program or initiative inclusive, equitable, accessible, and respectful of participants and community members?
• Is the program or initiative ethical? Are its decisions and practices consistent with the organization's stated principles? Does the work uphold fairness, honesty, respect, participation, accountability, privacy, and responsibility for its effects?

Although the two are usually congruent, ethical is not always exactly the same as legal. The exercise of civil disobedience may be profoundly ethical, while being, at the same time, inarguably illegal. The civil rights marches and actions led by Martin Luther King and others during the 1950's and 60's often fell into this category.

Restart the cycle

Your analysis should naturally lead into a new planning session and, where necessary, to rethinking and revising the program or initiative, or even the organization's role and priorities.

How do you maintain quality performance?

While the maintenance of quality is, to some extent, built into the Deming Cycle, it requires some particular commitments and action.

Institutionalization of dynamism

An organization needs to be dynamic, always moving and always seeking continued improvement, and to institutionalize its dynamic character.

This means:

• A commitment to learning and adaptation should be part of the organizational culture, with people throughout the organization understanding and contributing to it.
• Encouraging and providing support -- to staff, volunteers, and participants -- for learning.

Organizational support for education includes providing, or coordinating with another organization's provision of, professional development or university courses, training, certification, etc., that help staff and volunteers to build their skills. If financial resources are not available, other means of institutional support -- release time or leave time, special recognition, a library for the organization, study circles or reading groups, etc. -- need to be considered. Everyone in the organization, including administrators and Board members, should be encouraged to take advantage of learning opportunities and to model learning behavior.

• Listening to and carefully evaluating ideas from everyone.
• Encouraging openness to change and experimentation with new ideas and strategies

Administrators and Board members need to model such openness by being willing to reexamine and change procedures, policies, etc. when needed. Staff and volunteers should be given room to try out even things that others may be skeptical about, as long as they can justify the attempt ("It worked elsewhere" and "I learned about it in a course" are both reasonable justifications.). If such an attempt is honestly carried out, it should be seen as a positive even if it fails: it provided new information, and is another building block in the construction of a quality program.

• Never being complacent and always being open to the idea that the work could be done differently -- and better.
• Incorporating ongoing reevaluation, including feedback, ideas, and knowledge from participants and community members.
• Always being aware of the original mission, but not being afraid of change. While the mission itself may change as the community and circumstances do, it should nonetheless remain consistent with the principles and philosophy upon which the organization was founded.

Long-range strategic planning

To maintain quality, an organization needs to continually examine its work with a long-term perspective.

It needs to ask some questions about its role and its future:

• Is it meeting an ongoing need effectively? If not, what does it need to do to become more effective?
• Have community priorities, needs, strengths, or circumstances changed? Are they likely to? If so, how can the organization adapt in partnership with the community?
• Are there different or additional roles the organization should consider? Should it adapt or expand its current activities in response to current or emerging community priorities?
• Does it need more resources, or will it in the future? What are some likely sources?
• Is its structure appropriate to what it is doing and consistent with its mission and guiding principles? For example, an organization committed to shared power and community participation should examine whether its own decision-making structures reflect those values.
• Are its goals, vision, and philosophy still relevant to community realities and priorities and consistent with its organizational mission and guiding principles?

SWOT Analysis

One way to explore these questions is through the use of another planning tool often used in organizational settings: SWOT analysis. SWOT stands for Strengths, Weaknesses, Opportunities, and Threats. Each of the questions above can be examined through this framework. What internal strengths and limitations affect your organization's work? What opportunities exist? What external threats or challenges may affect the organization's ability to sustain quality performance?

Strengths and weaknesses: Strengths and weaknesses may be trends, rather than specifics. A level of service that is currently appropriate, for example, is not a strength if it is more or less than will be needed in a year. A new program that's not ready to implement yet is not a weakness if it's unready because the developers are taking the time necessary to make it effective.

By the same token, strengths and weaknesses don't necessarily lie only in the success of programs or the skills of staff members, but in such areas as relationships, contacts, and reputation. An organization running a great program may still be have serious weaknesses because it lacks some of these other features, no matter how well it carries out its day-to-day activities.

Opportunities: Opportunities can take many forms.

• An organization may be able to address additional priorities with its current structure. For instance, an organization that connects pregnant adolescents with prenatal care may also be well positioned to provide or connect families with vaccinations, nutrition information, parenting resources, and other supports after a child is born.
• It may be possible to expand into other areas of service, or into a larger arena (another town, another county, national instead of just one state).
• Increased funding may be available from new sources, or because of changed circumstances. A new census, for example, can result in an increase in federal funds to a region, or an economic downturn may bring a demand -- and increased funding -- for adult education or retraining.
• Collaboration with other groups, leading to increased resources, may become a prospect.
• Invitations, awards, partnerships, or positive media attention may strengthen awareness of and confidence in your organization's work.

Taking advantage of an opportunity can have both benefits and tradeoffs for your organization, so it is important to analyze the situation carefully before making a commitment.

Threats (Challenges): Some opportunities also bring significant challenges and tradeoffs that should be considered carefully. Growth or expansion may require organizational restructuring, additional capacity, and substantial planning. Some opportunities may even lead an organization to reconsider its purpose or scope. As an organization grows or gains wider recognition, it is important to remain connected to its mission, guiding principles, community relationships, and accountability.

Other challenges may arise independently of new opportunities. Your organization may have difficulty securing and sustaining funding and other resources, recruiting and retaining qualified staff, maintaining capacity across areas such as outreach, communications, programming, and evaluation, navigating controversy, or responding to opposition from individuals, organizations, or parts of the community.

Applying SWOT analysis across the organization's work can help identify strengths to build on, limitations to address, opportunities to pursue, and external challenges to anticipate and prepare for.

Other facets of the planning process

Some specific areas that long-range strategic planning needs to address at regular intervals:

• Reexamining the organization's vision, guiding principles, and mission statement. Are they still relevant to what the organization does, the communities it works with, and the values it seeks to uphold? Do they need to be restated or redefined?
• Reexamining the goals of the organization. Are they still relevant to the community priorities and needs they were originally meant to address? Are they consistent with the vision and mission statement?
• Reexamining the current strategies of the organization for meeting those goals. Are current methods effective? Are they consistent with the organization's vision, mission statement, and guiding principles? Are they feasible, accessible, equitable, and responsive to community priorities and perspectives? Are they ethical?

Keeping at it

The single most important thing to understand about maintaining quality performance -- or maintaining an organization, for that matter -- is that you can never stop working at it. No effort at maintaining quality will work any longer than it is applied. No matter how institutionalized dynamism becomes, no matter how good your planning process is, they take constant care.

The Fourteen Points

Deming used the key elements listed above in "Basic principles of TQM" to define the "Fourteen Points," fourteen things that he felt businesses needed to do in order to achieve quality performance.

1. Create constancy of purpose toward the improvement of products and services in order to become competitive, stay in business, and provide jobs.
2. Adopt the new philosophy. Management must learn that it is a new economic age and awaken to the challenge, learn their responsibilities, and take on leadership for change.
3. Stop depending on inspection to achieve quality. Build in quality from the start.
4. Stop awarding contracts on the basis of low bids.
5. Continuously improve the system of production and services to enhance quality and productivity, and thus constantly to reduce costs.
6. Institute training on the job.
7. Institute leadership. The purpose of leadership should be to help people and technology work better.
8. Drive out fear so that everyone may work effectively.
9. Break down barriers between departments so that people can work as a team.
10. Eliminate slogans, exhortations, and targets for the work force. They create adversarial relationships.
11. Eliminate quotas and management by objectives. Substitute leadership.
12. Remove barriers that rob employees of their pride of workmanship.
13. Institute a vigorous program of education and self improvement.
14. Make the transformation everyone's job and put everyone to work on it.

In Summary

Achieving and maintaining quality performance is important to participants, community members, partners, funders, and the organization itself.

Using some TQM principles and, specifically, the Deming Cycle (Plan, Do, Check, Act, Analyze) can be helpful in getting to a high level of quality and continuing to improve.

In general, achieving and maintaining quality is a result of:

• Careful planning
• Program or initiative development that adheres to that planning
• Implementation that takes quality into account
• Constant reevaluation of implementation and of the organization
• An assumption of the dynamic character of the organization, and a willingness to change continually in striving for a better way to accomplish goals
• Keeping at it indefinitely

If your organization can sustain these practices over time, it will be better positioned to maintain quality, learn from experience, and adapt as circumstances change.

What is the Total Quality Management system?

___Total Quality Management refers to principles developed by W. Edwards Deming that encourage the development of a flexible, dynamic system which involves everyone in an organization in the production of goods that exactly meet the customer's needs, do precisely what they are supposed to do as effectively as possible every time at the best possible price, and are constantly being improved

Why is quality important for a grass roots organization?

___It increase the group's effectiveness at reaching its goals

___It adds strength and credibility to your organization/initiative

___It helps the organization fulfill the ethical obligation to provide the best service or advocacy possible

___It is more economical over the long run

___It has many positive effects on your organization

What are the basic principles of TQM?

___Customer focus

___Obsession with quality

___Continual improvement of systems

___Unity of purpose

___Teamwork

___Employee involvement

___Education and training

___Scientific approach

___Long-term commitment

What are the steps of the Deming Cycle?

___Plan: Conduct research and plan for improvement

___Do: Produce the product

___Check: Check to be sure that the product meets planning goals

___Act: Market the product

___Analyze: Analyze how the product is received in the marketplace

How should you use TQM for your organization?

___Determine which elements of TQM are relevant to your situation, and which are not

___Commit your organization to maintaining quality performance through:

• Institutionalization of dynamism
• Long-range strategic planning
• Keeping at it forever

___Know what SWOT analysis is, and how to use it

A new community health initiative was about to start, funded by anti-smoking money from the state Department of Health. The program was intended for people participating in a number of community service programs – employment training and job development, adult basic education, pregnancy prevention among adolescents, and others. Its focus was supposed to be on sharing information about the health effects and addictive nature of smoking and supporting people who wanted to quit.

Before the program started, however, its organizers decided that they needed to understand which health issues potential participants considered most important.

The health educators interviewed a large number of people involved in the community programs they were working with and found that many both smoked and were interested in quitting. More immediate concerns for many participants, however, included neighborhood violence and substance use. Smoking ranked much lower among their priorities.

After conferring with its funder, the community health program began by addressing violence and substance use as community health issues. It attracted a large number of participants who, through a variety of strategies and collective efforts, helped reduce violence in the area and supported the establishment of a substance use treatment center in the neighborhood. During this work, participants were also introduced to information about tobacco addiction and resources for smoking prevention and cessation. Many participants who smoked subsequently joined smoking cessation groups and were able to quit. Although the program had begun by addressing health concerns that might initially have seemed unrelated to smoking, the health educators ultimately developed one of the most successful smoking cessation programs in the state. They realized that this success depended in large part on asking participants about their priorities, experiences, and what would make the program relevant to them.

Health and community service programs often seek to support community well-being, but organizations do not automatically know what community members consider most important or what approaches will be useful. One of the best ways to understand these priorities is to ask people who are directly affected by or involved in the work. Chapter 40 is about attaining and maintaining quality in your work. In this section, we'll discuss what feedback from participants consists of, how you can obtain it, and how you can use it to create and maintain a high-quality program, initiative, or intervention that is responsive to community priorities, strengths, and needs.

What do we mean by feedback from participants?

In simplest terms, feedback from participants consists of reactions, perspectives, experiences, and opinions about your effort from people who are affected by or involved in it. If you've read the first section of this chapter, you'll remember that an important element of building quality into programs is responsiveness to the people and communities involved. In the original Total Quality Management framework, this concept is described as customer-centeredness. In community-based work, participant- and community-centered practice means understanding what people actually experience, value, and need rather than assuming you already know. It also means meaningfully involving participants in planning, implementing, and evaluating the work so that programs can adapt in response to their knowledge and experiences.

This emphasis on participant- and community-centered practice fits closely with the Community Tool Box's emphasis on participatory processes. We believe that involving participants from the beginning in a program or intervention greatly increases the possibility of its success. W. Edwards Deming's Total Quality Management framework emphasized listening to customers; in community-based work, that principle can be expanded to recognize participants and community members as partners whose lived experience and knowledge should help shape decisions. Understanding what people experience, value, and need makes it easier to develop programs that are relevant and responsive.

Feedback is not simply criticism, constructive or otherwise. Rather, it reflects the experiences and perceptions of the person or group providing it. You may believe that you are presenting something in a particular way or accomplishing a particular purpose, but participants may experience it differently. Understanding those differences can provide important information for improving your work.

Feedback can be positive, negative, mixed, or neutral.

Its nature may also be influenced by a number of factors:

• Feedback may draw on facts, observation, past experience, personal reactions, relationships, cultural context, trust, and other sources of knowledge. Rather than treating some forms of feedback as inherently more valid than others, consider what each perspective can tell you and what additional information may be needed.
• Participants may bring important knowledge about the history of the community, specific people and relationships, culture, local institutions, previous initiatives, and other factors that may not be visible to program staff.
• Feedback may also be shaped by the information participants have received, how accessible that information is, and their experiences with the issue or process. When perspectives differ, additional conversation can help clarify what people know, understand, or have experienced.
• Feedback might be solicited (asked for) or unsolicited. Unsolicited feedback may sometimes be expressed strongly, particularly when people have felt unheard or frustrated. Even when its delivery is difficult, it may contain important information worth considering.

In addition to its range of possible characteristics, feedback can be communicated in a number of ways, and it's important to pay attention to as many of them as possible.

• Direct verbal feedback. Direct feedback can be especially useful because it gives participants an opportunity to describe their experiences and perspectives in their own words. It may be offered face to face, virtually, by phone, or in writing. Not everyone will feel comfortable providing direct feedback, however, particularly when there are differences in power or concerns about how the information will be used.

In general, this section focuses on obtaining intentional, direct feedback. We include some other ways of recognizing feedback immediately below. Direct feedback can often be easier to clarify and use, but organizations should also pay attention to other ways participants communicate their experiences.

• Indirect verbal feedback. This might include remarks that aren't intended specifically as formal feedback, comments made during everyday interactions, or observations shared by participants with one another.

A conversation with a participant might include a remark about how helpful the program has been for her, or how a particular activity or process wasn't useful. Participants may also discuss experiences with one another that they have not felt comfortable sharing directly with staff. When these concerns reach staff appropriately and without violating privacy, they may provide useful information about the participant experience.

Comments that initially seem broad or unclear may become more informative through respectful follow-up. A participant might say, for instance, "This is stupid!" Rather than assuming the comment reflects cynicism or lack of understanding, you might ask what specifically isn't working for them or what would make the activity more useful. The response may identify confusing instructions, inaccessible material, embarrassment, frustration, or another issue that can be addressed. Asking rather than assuming can produce useful information for both participants and staff.

• Direct non-verbal or behavioral feedback. Patterns such as missed appointments, difficulty completing tasks, reduced participation, or leaving a program may provide information about participants' experiences. These behaviors should not automatically be interpreted as rejection of the program; they may reflect barriers such as transportation, work schedules, caregiving responsibilities, accessibility issues, health concerns, stress, previous negative experiences, or other circumstances. When safe and appropriate, changes in behavior can be an invitation to ask what is happening and what support or changes might be useful.

It is important not to assume that every action is feedback about a program. A person's behavior may reflect many factors, including experiences within the program, conditions outside the program, stress, trauma, health, family responsibilities, safety concerns, or other circumstances. For example, an adolescent who appears guarded or disengaged may be responding to an interaction that felt disrespectful, to previous experiences of abuse, neglect, discrimination, or other trauma, or to circumstances unrelated to the program. Similarly, several people missing appointments may point to a program-level barrier, such as inconvenient scheduling or transportation difficulties, but individual circumstances may also play a role. The most useful response is usually to avoid assumptions and seek additional information where appropriate.

• Indirect non-verbal feedback. Facial expressions, tone of voice, body language, participation patterns, and other behaviors may provide clues about how people are experiencing a program. For example, continued signs of discomfort, anxiety, disengagement, or reluctance to participate may suggest that something about the environment, relationships, expectations, or activities deserves closer attention. Non-verbal cues vary greatly across individuals and cultures, so they should not be interpreted in isolation. Instead, use them as a reason to check in respectfully and create opportunities for participants to describe their experiences in their own way.

It's important to remember that feedback reflects participants' experiences and perceptions. Different people may interpret the same situation differently, and feedback should be considered alongside other information and perspectives. At the same time, participants' perceptions are themselves important information. If many participants report that they don't feel respected, supported, safe, or well served, the organization should seek to understand why and consider what changes may be needed rather than simply trying to change those perceptions. Even feedback that is difficult to hear can provide an opportunity for reflection and improvement.

Why do you need feedback from participants?

• It's part of being participant- and community-centered. Paying attention to participants' priorities, experiences, needs, strengths, and ideas is a crucial element in creating a quality program.
• It gives you tools to improve your program. Understanding what's useful and what isn't, what's working and what isn't, and what participants would change can help increase the effectiveness and responsiveness of your work.
• It allows you to respond to changes in the community or in participants' circumstances. Social, cultural, economic, demographic, environmental, or policy changes may affect participants and the community. For example, changes in migration patterns, employment opportunities, housing, neighborhood safety, or access to services may influence what participants need from a program. Participant feedback can help your organization understand these changes and respond appropriately.
• It can give you information about the history of the community, the issue, or previous efforts. Participants may know that a particular approach carries negative associations, has been tried previously, or encountered barriers that are still present. Understanding that history can help an organization avoid repeating past mistakes and build on previous learning.
• It can help you understand relationships and community dynamics. Participants may have valuable knowledge about relationships among organizations, neighborhoods, leaders, networks, and community groups. This knowledge can be important when building partnerships or creating planning and decision-making groups.
• It can tell you when your methods or approach aren't working. If participants consistently report that they aren't seeing progress or that an approach isn't useful, that feedback should be examined alongside other evidence.
• It can tell you when people feel uncomfortable, unheard, excluded, or disrespected. Participant feedback can identify an intake process that feels impersonal or intimidating, communication that is inaccessible, or staff attitudes and practices that make participants feel that their knowledge or experiences are not valued.
• It can help you address what is important to participants, even when those priorities differ from your original focus. The smoking cessation example that opens this section shows how responding to participants' priorities can strengthen engagement and ultimately support the program's broader goals.
• It can help ensure that your objectives are meaningful to participants as well as to your organization. Programs are more likely to be relevant and sustainable when their goals are informed by the priorities, strengths, and experiences of the people involved. If participants see little connection between a program and what matters to them, participation is likely to decline.

Who are the participants that can offer feedback?

All participants can offer meaningful feedback when organizations provide appropriate and accessible ways for them to communicate. People differ in age, language, communication style, literacy, cognitive abilities, disability, comfort with institutions, and preferred ways of expressing themselves. These differences may affect the methods you use to gather feedback, but they should not be treated as reasons to exclude people from the process. The key is to match feedback methods to participants' communication preferences and support needs and, where appropriate, offer multiple ways to participate.

Considerations for making feedback opportunities inclusive include:

• Participants who use languages other than the dominant language. Provide interpretation, translated materials, bilingual staff, or other language-access supports when possible. Also consider how power, previous experiences with institutions, immigration experiences, cultural norms, and concerns about confidentiality may affect whether someone feels comfortable giving candid feedback. Avoid assuming that everyone from a particular cultural or language group will communicate in the same way; ask participants what would help them feel comfortable and heard.
• People who have left the program before reaching their goals. Their perspectives may provide especially valuable information about barriers to participation. Why did they leave? Was the program relevant and accessible? What barriers did they encounter? What might have made participation easier or more worthwhile? When appropriate and respectful, following up with former participants can help an organization understand where changes may be needed.

Sometimes a participant's decision to leave has less to do with the program's content than with barriers surrounding participation. Participants may need childcare, transportation, disability accommodations, scheduling flexibility, language access, financial support, or other resources that are not available. Understanding these barriers will not make every program appropriate for every person, but it can identify changes that make participation more accessible and equitable.

• Children. Children can provide meaningful feedback about how they experience a program, even when they may not describe those experiences in the same way adults do. Use developmentally appropriate questions and methods such as conversation, drawings, visual scales, activities, or observation, and make sure children understand that their perspectives are welcome. Behavioral changes may also signal comfort, distress, confusion, or unmet needs, but they should be explored rather than interpreted automatically.
• Adolescents facing significant barriers or challenging circumstances. Young people experiencing violence, difficulties at school, substance use, unstable housing, discrimination, family stress, or other challenges may have important perspectives about the services and environments around them. Trust may take time to develop, particularly when young people have had negative experiences with adults or institutions. Create respectful, confidential, youth-friendly opportunities for feedback and avoid assuming that reluctance to speak reflects a lack of insight or interest.
• People with mental health conditions. People with mental health conditions can provide valuable information about the services and programs they use and about their own experiences, priorities, and needs. Organizations should provide respectful opportunities for feedback and make reasonable accommodations when needed rather than assuming that a diagnosis limits someone's ability to contribute.
• Adolescents and adults with intellectual or developmental disabilities. People with intellectual or developmental disabilities can provide meaningful feedback about their experiences. The best method will depend on individual communication preferences and support needs. Plain language, visual aids, supported communication, accessible surveys, additional time, trusted communication partners, or other accommodations may help people express what they like, dislike, need, or would change.

When might you obtain and use feedback from participants?

In thinking about when feedback is necessary, it's helpful to refer to the Deming Cycle, which consists of five phases: Plan, Do, Check, Act, Analyze. Feedback from participants can and should be used at every phase of the Deming Cycle.

• Before a program or initiative starts (Plan): Participant feedback is crucial to ensuring an accurate understanding of community strengths, assets, priorities, needs, and circumstances and to identifying outcomes that are meaningful to the people involved.
• During the development of the program or initiative itself (Do): Invite people who will participate in or be affected by the effort not only to provide feedback but, where possible, to help shape its goals, design, activities, and implementation. Meaningful participation can strengthen relevance, accessibility, trust, and community ownership.
• When your program is ready to begin (Check): Asking potential participants to review the plan can identify barriers, omissions, strengths, and opportunities for improvement before implementation. Ask whether the program reflects the priorities participants identified, whether it is accessible, and whether its intended outcomes are meaningful and realistic.
• During the running of the program (Act): Continue creating opportunities for feedback throughout implementation so you can respond to emerging issues, improve accessibility and participation, and make appropriate adjustments as you learn.
• As part of a formal or informal evaluation (Analyze): Conduct evaluation regularly at intervals appropriate to your program. Participant feedback is an important part of understanding strengths, challenges, outcomes, and unintended effects. Combining it with other forms of evidence can guide improvements and begin the cycle again.

How do you obtain feedback from participants?

Obtaining honest feedback from participants may sometimes be as simple as asking, but creating conditions where people feel comfortable answering candidly often requires more intentional work. Differences in power, previous experiences with institutions, language, culture, disability, age, confidentiality concerns, and the consequences participants believe could follow from criticism can all affect what people feel comfortable sharing.

Avoid assuming that a person's culture, gender, age, or other identity determines how they will respond. Instead, consider how communication norms, relationships, social expectations, institutional experiences, and power dynamics may affect participation. Language access and literacy can also affect which methods work best. Participants in involuntary programs, such as court-mandated substance use treatment, may have particular concerns about confidentiality or about whether criticism could affect their status. Others may simply hesitate to criticize a staff member or organization they value.

Obtaining honest feedback is generally easier when participants have been treated from the beginning as people with valuable knowledge and a meaningful stake in the program. When participants help shape planning, implementation, services, education, or treatment, feedback becomes part of an ongoing partnership rather than a separate activity. People are more likely to share candid perspectives when they know their knowledge is respected, understand how their feedback will be used, and see evidence that the organization takes their ideas seriously.

We present below several ways to ask for and obtain useful feedback from participants, but one of the strongest foundations for honest feedback is to involve participants as partners in the work from the outset wherever possible.

Methods of obtaining feedback

There are a number of methods you might use to obtain feedback. Which is best will depend on your situation, and you may use more than one method in order to make participation accessible to different groups of people.

Surveys. Using a survey to collect feedback is a common practice. Surveys can be administered in several ways.

• Written. This often takes the form of a short questionnaire that people can complete and return. Some surveys are longer and may include opportunities for open-ended comments in addition to yes-no answers or rating scales.
• Face to face. An interviewer asks questions and records the participant's responses. A longer or more open-ended conversation may become an interview, a method discussed below.
• Phone. Phone surveys can be useful when participants prefer or need to respond remotely and can allow an interviewer to clarify questions when necessary.
• Online surveys. These are now common and convenient for many participants. However, digital access, reliable internet service, device availability, digital literacy, disability access, data costs, and privacy may affect who can participate. If these barriers are likely, offer additional ways to provide feedback rather than relying exclusively on an online survey.

Email and online surveys can make feedback easier for some people, but digital access is not universal. Communities with limited economic resources, rural communities, older adults, people with disabilities, and others may face barriers involving devices, internet access, data plans, accessibility, or digital literacy. Offering multiple feedback options can help avoid excluding people.

Like all the methods described here, surveys have advantages and disadvantages. Short surveys are relatively easy to complete and can be designed to require little time. Written or online surveys may also be structured so that participants can respond anonymously, which may make it easier to share candid feedback.

Getting people to complete surveys can be difficult. Reasonable incentives may increase participation, as may providing time during a program activity for participants to respond. Be clear about whether participation is voluntary and avoid designing incentives in ways that create undue pressure to participate.

Surveys can also provide limited information when they rely primarily on fixed-response questions. Longer surveys may provide more detail but may be more burdensome to complete. Consider accessibility, time, participant preferences, and the amount of detail you need when choosing a format.

Some participants may prefer or need alternatives to standard written surveys because of literacy, language, visual, cognitive, or other accessibility needs. This may include people who left high school before graduating, participants in adult basic education or English-language learning programs, and many others. Options can include plain-language materials, translated surveys, screen-reader-compatible forms, phone or face-to-face administration, large print, visual response scales, or having questions read aloud when the participant chooses that support.

Interviews. These are conversations with a specific purpose – in this case, obtaining feedback about your work. Interviews can take place in person, by phone, or virtually and can be conducted individually or in groups.

An interview differs from a structured survey in several ways. Interviews may use a consistent set of questions while still allowing the interviewer to ask follow-up questions, clarify responses, or explore issues that participants raise. Interviews also tend to provide more space for explanation and reflection than brief surveys.

Most interviews emphasize open-ended questions rather than questions that can be answered only with yes or no. Rather than asking "Did you like X?", for example, an interviewer might ask, "What worked well for you about X? What didn't? Why?" A good interviewer can also ask respectful follow-up questions when an important theme emerges.

Individual interviews can provide detailed information and give the participant privacy and the interviewer's focused attention. These conditions may make it easier for some people to discuss experiences that they would not raise in a larger group.

Individual interviews can also be time-consuming and may limit how many people can participate. As with any form of qualitative feedback, one person's account represents that person's experience and should not automatically be treated as representative of everyone. The quality of the interview also depends on trust, accessibility, interviewer skill, and the participant's comfort with the setting.

Small-group interviews and focus groups. A small-group interview includes several people and gives participants opportunities to respond to and build on one another's ideas. Questions may be prepared in advance while still allowing space for follow-up discussion and issues raised by participants.

Focus groups are structured small-group discussions organized around a particular topic. A trained facilitator uses prepared questions while allowing participants to elaborate, respond to one another, and raise related concerns. Explain the purpose of the discussion honestly while avoiding questions that lead participants toward a predetermined response. When deciding who should be in a group, consider whether relationships, power differences, confidentiality concerns, or group dynamics could make some participants less comfortable speaking openly.

Town-meeting or whole-program format. In this situation, a larger gathering – perhaps even all participants in a program – meets to discuss experiences, concerns, ideas, and possible improvements. With skilled facilitation and clear expectations for respectful participation, this format can generate useful feedback, new ideas, and opportunities for collaboration. It may be less appropriate when people need confidentiality, when power differences make open discussion difficult, or when unresolved conflict could prevent some people from participating safely and meaningfully.

If you use a town-meeting or whole-program format, provide skilled facilitation, establish expectations for respectful discussion, and consider whether some participants may need other ways to share concerns privately. Significant disagreements or unresolved problems do not necessarily mean that a group discussion should be avoided, but they may require additional preparation, facilitation, or opportunities for smaller or confidential conversations.

Journals. A way of obtaining ongoing feedback is to invite participants to keep journals. These may be devoted solely to feedback or may have a larger programmatic purpose, depending on the nature of the program. Regular entries can provide an ongoing record of participants' reactions, experiences, questions, and observations.

If journals are interactive – participants' entries are read and responded to by an educator or another appropriate staff member – participants can be invited to elaborate on particular comments and concerns. Be clear about privacy, who will read the journal, and how the information will be used. This method works best when participants are willing and able to use the format and when alternative methods are available for those who prefer another way to communicate.

There are many other ways of obtaining feedback. For example, a program might use a brief anonymous rating after an activity, suggestion cards, text-message surveys, accessible digital forms, participant advisory groups, listening sessions, or creative methods designed with participants themselves. The most useful method is one that fits the people, context, and type of information you need.

Which method you decide to use will depend on a number of factors, including time, accessibility, confidentiality, available resources, participant preferences, and the amount of detail you need. Individual interviews may be useful for detailed feedback; surveys may work well when participants are geographically dispersed; group discussions can generate interaction and collective ideas; and journals can provide ongoing reflection. Using more than one method often helps organizations hear from people who communicate in different ways.

Ways to improve your chances of getting good feedback

Regardless of what methods you use to gather feedback from participants, there are some things you can do to increase the likelihood that people will feel comfortable sharing honest and useful information. Not every strategy will be appropriate for every method or group, so choose approaches that fit your participants and context.

Let people know from the beginning that you want their feedback, and will ask for it

This lets participants know that reflection and constructive criticism are expected parts of the program, establishes them as partners in improvement, and makes clear that their perspectives are valued. Explain that feedback is welcome throughout the program, not only when formally requested.

Specify what kind of feedback you want

The clearer you are about the information you're seeking, while still leaving room for issues you may not have anticipated, the more useful the responses may be.

Some of the questions you might ask or things you might want to know about:

Assessment / planning:

• How important is this service or issue in the community? Who is most affected, and what strengths, resources, or barriers should we understand?
• Has this service or issue been addressed in the community before? What happened, and what can we learn from that experience?
• Who should be involved in discussing, planning, or making decisions about this effort?
• Are there relationships, histories, or power dynamics that could make collaboration difficult? Who has worked well together in the past?
• What would make people want to participate in this effort or program? What might discourage participation?
• Are any proposed approaches culturally inappropriate, inaccessible, disrespectful, or inconsistent with how people in the community prefer to engage?

Ongoing feedback and program evaluation:

• What do you like about the program? What works well for you?
• What would you change about the program? What doesn't work well for you?
• What makes it easier for you to continue participating? What has helped you stay involved?
• What circumstances, barriers, or program experiences might make it difficult for you to continue participating?
• Was/is intake welcoming, informative, accessible, and respectful? Were your questions answered? Was the process of determining eligibility or enrollment clear and manageable?
• Were you connected with activities or services that fit your goals and circumstances? Was there flexibility when your needs changed?
• Are you receiving the services or support that were explained or promised? Are they useful to you? Are there other services or resources you need?
• Are there support services or accommodations that would make participation easier, such as childcare, transportation, language access, disability accommodations, technology, or scheduling flexibility?
• Are staff supportive, knowledgeable, helpful, and respectful?
• Do you feel respected and treated as a partner whose experiences and ideas matter?

This question is really about the philosophy of your organization and how it views participants. Staff may be polite while still communicating, intentionally or unintentionally, that professionals know best or that participants lack the knowledge or ability to make decisions about their own lives. Respect involves recognizing participants' agency, experience, knowledge, and right to make informed choices. Honest conversations can include difficult information or accountability, but they should still preserve dignity and avoid humiliation or paternalism. If participants consistently report that they do not feel respected or heard, take that feedback seriously and examine the organization's assumptions, practices, policies, and power relationships.

• Are program expectations clear and consistent? Do staff demonstrate care and respect? Can you get help when you need it?
• How are your relationships with other participants? Do you feel comfortable and included? If problems arise, are there safe and effective ways to address them?
• Have there been problems related to payment, insurance, benefits, or eligibility? Are these processes easy to understand and address? Do you feel comfortable asking staff for help?
• Does the schedule work for you? What changes, such as evening, weekend, remote, or flexible options, might make participation easier?
• Is the location convenient and accessible? Are transportation options available? Do you feel safe traveling to and using the location?
• Is the program site comfortable, welcoming, and appropriate for the activities taking place? Is there enough space and privacy? Is it physically accessible, and are the restroom and other facilities adequate for the people who use them?

Make clear exactly what you're going to use the feedback for

Whether the feedback is being gathered for assessment, planning, ongoing improvement, or evaluation, explain how the information will be used. Participants may worry that criticism could affect their services, relationships with staff, eligibility, or other outcomes. Be clear about confidentiality, any limits to confidentiality, who will see the information, and how decisions will be made. Explain that the purpose is to understand experiences and improve the program rather than punish participants or individual staff members.

Openness about why you're asking for feedback also supports participatory practice by recognizing participants as contributors to planning, evaluation, and improvement and demonstrating that their knowledge is valuable.

Protect anonymity or confidentiality when appropriate

Concerns about being identified can prevent people from providing candid feedback. When possible, offer anonymous methods such as surveys that do not collect identifying information or confidential methods in which identifying information is protected and separated from responses. Explain clearly whether feedback is anonymous, confidential, or neither; these terms are not interchangeable.

When participants are concerned about being identified, choose methods that provide the greatest realistic protection. An independent facilitator, anonymous survey, confidential interview, secure suggestion process, or other approach may help. Group discussions cannot guarantee anonymity among participants, so make sure people understand those limitations before deciding whether to participate.

Let people know that you'll share the results of feedback with them, and be sure to do it

Participants should be told what you learned from the feedback and, when appropriate, what changes you plan to make in response. If recommendations cannot be implemented, explain why. Closing the feedback loop demonstrates that participation mattered and gives people an opportunity to respond to how their input has been interpreted and used.

How do you use feedback from participants?

Once you've gathered feedback, the next step is to use it. Soliciting people's perspectives and then filing them away without meaningful consideration can undermine trust. Participant feedback should inform reflection, decision-making, improvement, and, where appropriate, changes in organizational practice.

Understand the priorities, strengths, and needs of the community

Feedback from potential participants and other community members before planning begins can help identify community priorities, strengths, assets, needs, existing resources, barriers, and the issues people believe deserve attention. It can also help determine what level and type of activity or service would be useful.

Understand changes in the community or in participants' circumstances

Communities continually change because of migration, economic conditions, housing patterns, demographic shifts, policy changes, development, displacement, environmental conditions, and many other factors. For example, the arrival of new residents may bring new languages, cultural knowledge, organizations, networks, and service needs, while longtime residents may also experience changes in resources, relationships, housing, employment, or neighborhood conditions. Factory closures, major construction projects, gentrification, or other changes may significantly affect participants and require an organization to reconsider its approach.

Organizations may be aware that community changes are occurring without fully understanding what those changes mean in people's daily lives. Participant feedback can provide that context and help organizations adapt their work in ways that remain useful, equitable, and responsive.

Improve your program itself

Participant feedback can be tremendously useful, both on an ongoing basis and as part of evaluation, in helping you make your program more effective, accessible, equitable, and responsive.

Feedback can alert you to the need to refine techniques and methods, strengthen approaches that are working, and change those that aren't. It can identify program content that is no longer relevant, as well as barriers involving schedules, transportation, language, disability access, technology, childcare, cost, or other conditions affecting participation.

Participant feedback can also have a major impact on program philosophy and organizational culture. What participants say about how they are treated, whether they feel heard, and whether they have meaningful opportunities to influence decisions can be as important as feedback about specific services.

 There are really three areas to look at here, all of which have already been referred to in this section:

• Attitude toward participants. Participants are more likely to engage when they are treated with dignity and when their knowledge, choices, identities, and experiences are respected.
• Approach toward provision of services. When participants are treated as partners and have meaningful opportunities to shape a program, they can contribute to its relevance, quality, and success as well as benefit from what it offers.
• Identifying and addressing barriers to participation or success. Respectful, participant-centered programs pay attention to practical, organizational, and structural barriers such as childcare, transportation, disability access, language, schedules, cost, technology, literacy, benefits, or other supports. Rather than assuming that difficulty participating reflects a personal failure, ask what conditions may be creating barriers and what changes or resources could address them.

Develop an organizational culture that embraces and institutionalizes ongoing feedback from participants

Many Community Tool Box sections emphasize that effective practices need to be sustained rather than treated as one-time activities. The same is true of obtaining and using participant feedback.

Feedback from participants isn't a one-time activity. Participants and staff change, circumstances evolve, new issues emerge, and institutional memory can fade. Programs and organizations need ongoing opportunities to learn and adapt while remaining accountable to the people and communities involved. Regular participant feedback helps ensure that improvement is informed by current experiences rather than assumptions about what worked in the past.

In Summary

Feedback from people with direct experience of your program, initiative, or intervention can be essential to improving its quality. Participants bring knowledge about what is useful, inaccessible, respectful, frustrating, effective, or missing. Honest feedback may be difficult to obtain when people are concerned about confidentiality, power differences, negative consequences, or whether their input will actually matter. One of the strongest ways to build trust is to involve participants meaningfully from the beginning and demonstrate that their knowledge influences planning, implementation, evaluation, and improvement.

Participant feedback can help you understand community priorities and strengths, identify barriers, improve existing work, and respond to changing circumstances. It can also reveal when organizational assumptions, relationships, policies, or practices need to change. The purpose of obtaining feedback is not simply to confirm that a program is working, but to learn how it is experienced and how it can become more effective, accessible, equitable, and responsive.

You might obtain participant feedback through surveys, individual or group interviews, focus groups, community meetings, journals, advisory groups, digital tools, or other accessible methods. Whatever methods you choose, people are more likely to provide meaningful feedback when they are treated as partners whose knowledge and agency are respected rather than as passive recipients of services.

Ultimately, strive to create an organizational culture that continually invites, values, responds to, and learns from participant feedback as a necessary part of maintaining and improving the quality of your work.

What do we mean by feedback from participants?

___Feedback from participants consists of reactions to, opinions about, and/or information on what you do from those to whom it’s offered or who benefit from it

Feedback may be:

___Positive, negative, or neutral

___Objective or subjective

___Enhanced by knowledge of community history, personalities and relationships, culture, etc.

___Limited by lack of knowledge, information, or understanding

___Solicited or unsolicited

___Direct or indirect

Why do you need feedback from participants?

___It’s part of being customer-centered

___It gives you tools to improve your program

___It allows you to respond to changes in the community, the population, or the situation

___It can give you information about the history of the community, the history of your issue in the community, or the history of your population

___It can inform you about personalities and relationships in the community and/or the population you’re working with

___It can tell you when your methods or approach aren’t working

___It can tell you when you’re treating people in ways that make them feel uncomfortable, angry, or otherwise disrespected

___It can help you deal with what’s important to participants, even though that may not be your major goal

___It can assure that your objectives make as much sense for participants as they do for you

Who are participants that can offer feedback?

___Adults and teens with no intellectual, psychological, or social barriers

___People with mental illness

___Children

___At-risk or troubled youth

___Developmentally delayed or retarded teens and adults

___Speakers of other than the majority language

___People who have left the program

When might you ask for feedback from participants?

___Before you begin an effort (Plan)

___As you develop a program, initiative, or intervention (Do)

___Before you implement your plan (Check)

___While the work progresses and in evaluating it (Act)

___In making adjustments and starting the cycle again (Analyze)

What are some methods for obtaining feedback from participants?

___Surveys

___Individual interviews

___Small group interviews and focus groups

___Town meeting or whole-program format meetings

___Journals

How can you increase your chances of getting good feedback?

___Guaranteeing anonymity

___Explaining why you need feedback, and how you’ll use it

___Being clear about exactly what kind of feedback you want

___Sharing the results with participants

How do you use feedback from participants?

___Use it to understand the needs of the community

___Use it to respond to changes in the situation of participants, the community, the political climate, etc.

___Use it to improve your program by responding to the need for change in attitudes, methods, content, etc.

___Use it to create or strengthen an organizational culture that encourages and welcomes feedback, and uses it well

 

Examples of oversight mechanisms include:

• A community or civilian oversight board that reviews complaints about police misconduct or excessive use of force.
• An in-house quality control team that determines whether the company's methods and products meet established standards.
• A system through which students can evaluate their courses and instructors' teaching.
• A diverse committee that reviews the experiences of patients at a local clinic.

In this section, we'll look at ways to establish processes that support accountability and quality.

What do we mean by oversight and oversight mechanisms?

Oversight refers to systematically monitoring and reviewing an activity, process, organization, or institution to support accountability, quality, effectiveness, and compliance with appropriate standards. For instance, in the case of a community or civilian oversight board, an independent group may review the quality, fairness, or conduct of services provided by a public agency. Oversight involves taking responsibility for monitoring the operation, implementation, and/or outcomes of a process, program, organization, institution, or effort.

An oversight mechanism is the system or process used to carry out that monitoring, review, and accountability.

Just as every writer can benefit from an editor, organizations and individuals whose work affects the community can benefit from appropriate accountability and review processes to help ensure that their work is effective, of high quality, ethical, and consistent with applicable laws, regulations, and commitments.

Oversight means paying systematic attention to how work is being carried out and what effects it is having. It can identify potential problems early, help prevent harm, and support timely improvement. For instance, good oversight can detect illegal or unethical practices by a business or government agency, identify and correct poor financial practices in a nonprofit organization, or help ensure that public health and safety are protected.

Oversight may be official – required by law or by external or internal regulations – or unofficial, as in the case of an independent community or watchdog group examining whether a government agency operates fairly and according to its mandate. When oversight is official, the mechanisms for it – as well as the individual or group responsible for exercising it – are often specified by the laws or regulations that require it. When oversight is unofficial, an independent group or individual may initiate the effort or may be asked by community members or others to take on that role.

Where oversight involves responsibility for the functioning and/or results of something, it can be established within an agency, corporation, institution, or organization, or it may be conducted by an external or independent body.

The activities of community health and development organizations can span a broad range, from planning, to intervention, to advocacy, among others. Maintaining the quality of these activities requires oversight mechanisms that reflect the vision, mission, values, and goals of the groups and individuals engaging in them. We'll discuss establishing oversight mechanisms with that in mind.

Why establish oversight mechanisms?

There are a number of reasons why you might want to set up oversight:

• To ensure accountability. Oversight can help hold individuals and groups accountable for operating legally, ethically, and responsibly, and for providing the services, products, and/or outcomes they have promised or are contracted to provide.
• To maintain and improve quality. We often think of quality control as an industrial issue, but principles of quality assurance and continual improvement also apply to services, programs, facilities, and community initiatives.
• To ensure adherence to laws and regulations. Oversight may be used to make sure agencies, corporations, and other entities – or departments or programs within those entities – operate according to requirements established by laws, regulatory bodies, funders, contracts, or the entities themselves.
• To prevent and address unfair treatment, corruption, illegality, and unethical behavior. Oversight can include processes for identifying and addressing practices in which some groups receive unfair advantages or others face exclusion or discrimination. Corruption by public officials—including the theft or misdirection of government funds and goods—can often be prevented, detected, or addressed through careful oversight. The same is true for illegal or unethical actions that may harm individuals, communities, or society, including people affected by or using the services involved.
• To ensure the effectiveness of programs or efforts for which you have responsibility. If you're responsible for an effort, you need to understand whether the agreed-upon process is being followed and whether the work is achieving its intended purposes. Oversight can also help identify when a plan isn't working as intended and support participants, staff, leaders, and other partners in determining how to improve it.
• To help improve well-being for everyone. Community well-being is strengthened when community interventions and initiatives are effective, public institutions fulfill their responsibilities, employers protect workers' dignity and safety, businesses and governments safeguard the environment, and community members have equitable access to the resources and opportunities they need to thrive.

Who should be involved in establishing oversight mechanisms?

The goal of effective oversight is to improve quality, strengthen accountability, and help people and organizations do their best work with and for the community. It therefore often makes sense to involve a variety of people and groups in designing and carrying out the oversight process. Inviting participants or students to evaluate the services or education they receive and involving community members in oversight of local government services are two of many ways to accomplish this.

Those who might be involved in oversight to make the process more comprehensive and useful include:

• People and communities most directly affected by the issue, institution, service, or situation involved.
• People providing the services or products being reviewed.
• Those with formal responsibility for the effort – program directors, school principals, elected or appointed officials, board members, or others with decision-making authority.
• Funders and others contributing resources or support to the implementation of the effort.
• Community advocates and organizations concerned with the issue or effort in question.

The oversight process often works best when it is participatory and includes meaningful roles for both people affected by the work and people responsible for carrying it out. Selection of participants can be more challenging when an organization or individuals subject to oversight are resistant to scrutiny, particularly when the purpose of oversight is to investigate potential misconduct or harm.

When should you establish oversight mechanisms?

The obvious answer here is "whenever oversight is necessary." But when is that? The short answer is "whenever there is an effort or situation that requires ongoing accountability, monitoring, or review." Let's examine what that means.

• When an effort, new program, or organization is starting. Oversight should be considered from the beginning so that quality, accountability, equity, and ethical practice are built into the work and problems with implementation or unfair treatment can be identified and addressed as early as possible.
• When a problem needs to be prevented from occurring again. For instance, if a community has experienced patterns of police misconduct, excessive use of force, or discrimination in housing, one response may be to establish an independent community or civilian oversight committee. Such a body can strengthen accountability and establish channels of communication between affected community members and the institutions responsible for serving them.
• When the quality or effectiveness of an effort or program is in question. Ideally, appropriate oversight would have been established at the beginning, but if it wasn't, it can begin when concerns arise and continue as long as the effort requires it.
• When there is an agreement, contract, or regulation to be followed. Oversight can help ensure that all parties fulfill their obligations and receive the compensation, services, resources, or other benefits that were agreed upon.

Oversight in this case is often internal, in order to make sure, for example, that money is spent in the ways agreed upon with the funder. When financial oversight is inadequate, organizations may spend funds outside approved purposes and be required to repay them, lose funding, reduce services, or in serious cases close.

• When an entity is engaged in an activity that could potentially harm the community, its members, or society. This can include violations of law, such as illegal dumping of pollutants. It can also include lawful activities – such as construction projects, water treatment, food processing, or other operations – that could create harm if appropriate procedures or safeguards are not followed. Oversight can help prevent harmful situations, identify them quickly when they occur, and support an appropriate response.

How do you establish oversight mechanisms?

In most situations, gather a group of stakeholders to plan and set up oversight mechanisms. As we've discussed, oversight often works best when people understand its purpose and see it as a way to strengthen accountability, quality, safety, or effectiveness. One possible exception is when oversight is intended to investigate suspected illegal, unethical, or otherwise harmful activity. Even then, depending on the circumstances and the need for independence, it may make sense to involve representatives of the entity being overseen while maintaining appropriate safeguards against conflicts of interest.

Decide what kind of oversight you need. The nature of the oversight will help you structure an effective oversight mechanism. There are many different kinds of oversight, including:

• Oversight in terms of accountability. Key questions include: Are people and entities fulfilling their responsibilities? Do programs and initiatives have positive effects, including their intended effects? Who is responsible for making improvements, and what support or resources may be needed? Are government bodies implementing programs effectively, providing necessary public services, and enforcing laws and regulations consistently and fairly?
• Oversight of the operation of a program, effort, or organization. Key questions include: Is it going according to plan? Is it using the methods and procedures it was designed to use? Is it accomplishing what it's meant to do? Are adjustments needed based on experience, evidence, or community feedback?
• Oversight of the conduct of a service, organization, government agency, etc. Key questions include: Are people treated fairly and equitably? Is there evidence of discrimination or inconsistent quality? Is the entity complying with applicable rules and regulations? Is it fulfilling its responsibilities?
• Oversight of the performance of individuals in an organization or institution – supervision of service providers, teachers, managers, etc. When appropriate, this can include formative evaluation designed to support learning and improvement, while still maintaining clear accountability when serious performance, ethical, or safety concerns arise.
• Oversight of the conduct of individuals in organizations and government. Are they following applicable rules and ethical standards, respecting the limits of their authority, avoiding conflicts of interest, and acting responsibly?
• Oversight of processes to make sure they're conducted properly and that policies and procedures are followed and effective. This includes oversight of budgets to ensure that resources are being spent appropriately and to identify potential fiscal problems early enough to address them.
• Oversight of agreements and contracts to be sure that their terms are carried out as specified and that all parties fulfill their obligations and receive what was agreed upon.
• Oversight to safeguard people who may face heightened risk of harm or require additional protections. This can include reviewing the quality, safety, accessibility, and rights protections associated with services involving, for example, children, people with disabilities, patients receiving health care, or older adults with significant support needs. Oversight should recognize the agency and rights of the people involved and include their perspectives wherever possible.

Choose an oversight mechanism that is best suited to the type of oversight you expect to conduct. Oversight mechanisms can take a number of different forms, depending on their purpose:

• Oversight by a designated or responsible individual or group. Oversight of an organization may be conducted by its director. Fiscal oversight of an organization, institution, or business may be conducted by its chief fiscal officer, treasurer, or a board financial committee. In some cases, certain individuals or groups – often boards of directors – are legally responsible for ensuring that their organizations act legally and comply with funder, contractual, regulatory, or other requirements.
• External oversight by a funder, government agency, independent body, or other responsible entity. This kind of oversight may take the form of spot checks or regular reports (see below), or may be carried out by an independent evaluator, consultant, inspector, or other third party.
• An oversight committee. This might be a community or civilian oversight panel for a government agency or service such as law enforcement, environmental services, or public health; or an internal oversight committee within an agency, institution, corporation, or organization. Oversight committees are often formally established either by or in collaboration with the body being overseen, although independence may be essential in some circumstances. Community oversight committees should meaningfully include people and communities most affected by the entity being overseen; for instance, tenants in oversight of a public housing authority or community residents on a local community health board.
• An independent monitoring or watchdog group. This is an organization or less formal group whose mission is to identify and publicize corruption, lawbreaking, discrimination, abuses of authority, or other illegal or unethical behavior by government agencies or officials, businesses and industries, or other institutions whose actions can affect individuals, communities, or society.
• A monitoring and evaluation process. This could be internal or external, with monitoring and evaluation carried out within an organization or institution by staff members or conducted by an evaluator, consultant, or other independent body. Evaluations can be conducted at intervals appropriate to the work and used to assess progress toward objectives, examine the quality and equity of services, identify unintended effects, and guide improvement.

There are many ways to conduct an evaluation process. It can be collaborative, with people affected by and involved in the work contributing; or it can be carried out by an individual or small group with more limited participation. The Tool Box recommends that evaluation be as collaborative and inclusive as possible while maintaining any independence necessary for credible oversight. When people understand the process, have meaningful opportunities to contribute, and trust that their perspectives will be considered, evaluations are more likely to produce useful information and support improvement. When evaluation is experienced only as an adversarial or punitive process, people may be less willing to share information openly.

• Spot checks by staff, supervisors, program directors, regulators, or other authorized reviewers. This method might be used in oversight situations where ongoing intensive monitoring is not feasible or where periodic checks are sufficient.
• Regular oral and/or written reports on the work being done and its results. Because reports prepared by people involved in the work may reflect their perspectives and available information, this mechanism can be strengthened by clear reporting standards, supporting documentation, independent review where appropriate, and opportunities for feedback from people affected by the work.
• Technology-assisted monitoring, such as cameras, audio recording, sensors, digital logs, or automated inspection of activities and products. This approach might be used for environmental monitoring, safety, quality assurance, or other appropriate purposes. Any use of monitoring technology should be proportionate to its purpose and consistent with applicable laws, privacy protections, informed-consent requirements, accessibility considerations, and organizational policies. People affected by monitoring should be informed about its use whenever appropriate and legally required, and access to recordings or data should be limited to authorized purposes.

Any or all of these mechanisms can be strengthened by systematic reflection and feedback from people and organizations subject to or involved in the oversight process, as well as from people and communities most affected by the entity or organization responsible for the work.

Decide who will conduct the actual oversight. This will be determined, at least in part, by the type and purpose of the oversight. Internal monitoring might be done by supervisors, with the involvement of staff members being monitored, or by a committee composed of staff from different parts of the organization. A funder might exercise oversight through an external consultant or evaluator, or through reporting by the funded agency or organization. An organization's treasurer or fiscal officer might oversee the budget to ensure that funding is spent for approved purposes. A community oversight committee should include community members affected by the agency or activities in question and may also include people with relevant technical, legal, or professional expertise.

Decide on the methods you will use to conduct the oversight. They should be appropriate to the situation, risks involved, and character of your organization or effort. The more your organization emphasizes inclusion and participation, for example, the more its oversight should create meaningful opportunities for participation. At the same time, some forms of oversight require independence to avoid conflicts of interest. Any oversight you conduct should reflect your organization's vision, philosophy, ethical standards, and accountability commitments.

Methods can vary greatly depending on the nature of the oversight. You wouldn't oversee an environmental improvement operation – forest management or wildlife habitat restoration – in the same way you would the functioning of a hospital or the conduct of a government agency.

​Some possible methods you might use to implement oversight mechanisms:

• Individual and group interviews with administrators or managers, staff members, current and former participants, funders, community members, and others affected by or familiar with the entity's work.
• Informal conversations with staff members, administrators, participants, or community members, while respecting confidentiality and making clear when information may be used as part of the oversight process.
• Direct observation of the work of the entity. This might entail accompanying staff members periodically, walking through a facility, observing activities, or visiting a location at different times, days of the week, and seasons to understand what happens under different circumstances. Observation should be conducted transparently and ethically, with appropriate attention to privacy, confidentiality, consent, and the rights of participants and staff.
• Participating in or observing the day-to-day work of the entity, sometimes referred to as participant observation. When this method is appropriate, the role of the person conducting the oversight, the purpose of the observation, and any limits on confidentiality should be clear. Avoid deceptive participation unless it is specifically authorized, legally permissible, ethically justified, and necessary for the oversight purpose.
• Study of organizational or agency files, records, and financial reports, as well as its history, funding sources, policies, and philosophy, while following applicable privacy, confidentiality, and records-access requirements.
• Research into practice models, evidence, standards, or theory that underlie the work of the entity.

Train those who will conduct the oversight if necessary. Conducting oversight can be different from conducting an evaluation. You may or may not have the full cooperation of the individuals or entity being overseen, even if they were involved in establishing an oversight mechanism. Some important guidelines therefore concern how oversight is conducted and how people affected by it are treated.

You, or whoever is involved in conducting the oversight, particularly those who have never done something similar before, may need training. Some things those who actually conduct the oversight should be guided by include:

• Get to know the organization or entity you're overseeing – its history, structure, vision and mission, finances, staff and leadership roles, organizational culture, community relationships, and relevant policies. Learn as much as possible about the work and the people who perform it. Try to understand the pressures, constraints, resources, and responsibilities that shape the work without allowing that understanding to replace accountability.
• Understand that people may be cautious about or resistant to oversight, whether because they disagree with it, are concerned about consequences, have had negative experiences with oversight processes, or because the process interrupts their work. Take these concerns seriously while maintaining the independence and accountability required by your role.
• Try to establish respectful working relationships with people you interview or interact with. Be clear about your role and purpose, communicate respectfully, and recognize the knowledge and experience of the people involved.
• Listen carefully to what people tell you and ask follow-up questions where appropriate. Make space for people to explain their perspectives in their own words.
• Use open-ended questions when appropriate – questions that invite explanation rather than requiring only a yes-or-no answer.
• Pay attention to the physical and organizational environment, including accessibility, noise, temperature, safety, working conditions, interactions, and communication patterns. Non-verbal communication can provide useful context, but avoid assuming that body language, eye contact, tone, or other behaviors have a single or universal meaning across people, cultures, or situations.
• Take careful notes using a method that is appropriate, secure, and minimally disruptive. This might include paper notes, a laptop, tablet, smartphone, or audio recording when appropriate. Explain how information will be documented and used. Before taking photographs or making audio or video recordings, obtain informed permission unless recording is otherwise specifically authorized by law or an appropriate oversight process. Protect recordings, notes, and identifying information from unnecessary access.
• Respect people's time. Be on time for meetings and appointments and avoid taking more time than necessary.
• Produce reports regularly and, where appropriate, share findings with people and organizations subject to or affected by the oversight. Explain any confidentiality or legal limits on what can be shared.
• Recognize effective work where appropriate while clearly identifying areas that require improvement. Focus findings on evidence, responsibilities, practices, conditions, and needed changes rather than personal blame, except where individual misconduct or accountability is directly relevant.

Create an analysis and reporting system. Both the oversight process and the information it produces should be documented for two reasons:

• To demonstrate that the organization, institution, or effort is fulfilling its responsibility to monitor what is happening and respond appropriately.
• To use the information to improve both the oversight process and the quality, equity, safety, and effectiveness of the operation.

To accomplish both purposes, you'll need to regularly record and make appropriate information from oversight mechanisms available to the people who need it. You'll also want to develop a process for analyzing that information so that you can identify patterns, potential problems, inequities, or irregularities and determine what action is needed.

Identifying potential financial problems is a particularly important area for oversight. In the spring of 2012, J.P. Morgan-Chase, one of the largest banks in the U.S., announced a loss of more than $2 billion associated with weaknesses in the oversight of its investment activities. While most nonprofit organizations are not likely to lose billions through poor financial decisions, inadequate financial oversight can still lead to reduced services, layoffs, repayment obligations, loss of funding, or even organizational closure. Accurate information, clear accountability, and timely analysis can help organizations identify and address many financial problems before their consequences become more serious.

Determine how you will use oversight to maintain or improve the quality of your work or of the entity or situation you're overseeing. What areas of the work or organization will you oversee? This question is especially important if you have limited resources for oversight. Another important issue is what purpose your oversight will serve. Is it meant to prevent something – harassment or discrimination in the workplace, illegal pollution of the air or waterways, misuse of public funds – or is its purpose to maintain standards of practice, equity, safety, and effectiveness? Is it intended to demonstrate that the organization is exercising appropriate due diligence by monitoring important aspects of its work? How will you use the information from oversight to understand problems and make improvements?

The answers to these questions will help determine how to set up an oversight mechanism that meets your needs. Some forms of oversight or inspection may appropriately be unannounced when necessary to assess compliance or investigate potential misconduct, but any monitoring should still operate within applicable legal, ethical, privacy, and due-process requirements. If the goal is improvement or organizational learning, a more transparent and collaborative process may be appropriate. If your goal is careful financial oversight or monitoring another organizational function, establish procedures that make patterns, irregularities, responsibilities, and decisions easy to review.

Evaluate the effectiveness of your oversight mechanism and adjust it as necessary. Oversight itself should be reviewed to make sure that it is attending to what matters, producing useful information, avoiding unnecessary burdens or harms, and supporting accountability and improvement. If it isn't, the mechanism, like any other aspect of your work, should be adjusted. Regular review can help determine whether the oversight process remains appropriate as circumstances change.

Continue oversight over the long term. As we explained at the beginning, oversight is often a long-term commitment. If things are going well, effective oversight can help maintain progress and identify emerging concerns. If oversight is intended to prevent problems, discontinuing it may allow problems to go unnoticed. Establish oversight mechanisms that can be sustained over time and periodically reassess whether their scope and methods remain appropriate.

In Summary

Oversight mechanisms provide structured ways to support accountability, quality, ethical practice, safety, and compliance. Organizations, institutions, businesses, and governments may exercise oversight over their own operations, while funders, regulators, government agencies, community bodies, or independent organizations may oversee entities for which they have formal responsibility or whose actions affect the public. Community members and community organizations may also participate in oversight of institutions and systems that affect their lives.

Oversight mechanisms often work best when their purpose, authority, roles, methods, and decision-making processes are clear. When appropriate, both people affected by the work and people responsible for carrying it out should have meaningful opportunities to contribute to the design or implementation of oversight, while preserving whatever independence is necessary for credible accountability. Effective oversight may also require training, technical support, accessible reporting systems, privacy protections, regular review, and a clear process for using findings to make improvements.

 

We encourage the reproduction of this material, but ask that you credit the Community Tool Box: /.

Contributor

Phil Rabinowitz

Editors

Andrea Glinn

Stephen Fawcett

Here you will find a checklist summarizing the important points of the section.

 

What do we mean by oversight and oversight mechanisms?

­­__ Oversight is the maintenance of a watchful eye on an individual, entity or process.

Oversight may also be the responsibility for the operation, implementation, and/or outcome of a process, program, organization or institution, or effort.

__ Oversight may be internal or external.

__ Oversight is different from evaluation in that it is constant and concerned with proper functioning as much as effectiveness.

__ An oversight mechanism is the system or process and the procedures used to exercise oversight.

__ Oversight and oversight mechanisms in health and community services and development should reflect the vision, mission, values, and goals of the groups and individuals engaging in them.

 

Why establish oversight mechanisms?

__ To assure accountability.

__ To control quality.

__ To ensure adherence to laws and regulations.

__ To control unfair treatment, corruption, illegality, and unethical behavior.

__ To ensure the effectiveness of programs or efforts for which you have responsibility.

__ To help improve the wellbeing of everyone.

 

Who should be involved in establishing oversight mechanisms?

__ Where possible and appropriate, establishing oversight mechanisms should be a participatory process, and involve:

• Those from the groups most affected by the issue or situation involved.
• Those providing the service or products for which oversight is directed (i.e., those being overseen).
• Those ultimately responsible for the success of the effort – program directors, school principals, elected or appointed officials, etc.
• Funders and others contributing resources or support to the implementation of the effort.
• Community activists concerned with the issue or effort in question.

 

When should oversight mechanisms be established?

__ When an effort or new program or organization is starting.

__ When a problem needs to be prevented from occurring.

__ When the quality or effectiveness of an effort or program is in question.

__ When there is an agreement, contract, or regulation to be followed.

__ When an entity is engaged in an activity that is potentially harmful to the community, its members, or society.

 

How do you establish oversight mechanisms?

__ In most situations, gather a group of stakeholders to plan and set up oversight mechanisms.

__ Decide what kind of oversight you need. Possibilities include:

• Oversight in terms of accountability.
• Oversight of the operation of a program, effort, or organization.
• Oversight of the conduct of a service, organization, government agency, etc.
• Oversight of the performance of individuals in an organization or institution – supervision of service providers, teachers, etc.
• Oversight of the conduct of individuals in organizations and government.
• Oversight of processes to make sure they’re conducted properly, and that policies and procedures followed and effective.
• Oversight of agreements and contracts to be sure that their terms are carried out as specified, and that both parties are fulfilling their obligations and receiving what was agreed upon.
• Oversight to protect vulnerable individuals.

__ Choose an oversight mechanism that is best suited to the type of oversight you expect to conduct. Some types of oversight mechanisms:

• Oversight by a designated or responsible individual.
• External oversight by a funder, government agency, or other responsible body.

• An oversight committee.
• A watchdog group.
• A monitoring and evaluation process.
• Spot checks by staff, supervisors or program directors.
• Regular oral and/or written reports on the work being done and its results.
• Mechanical oversight, as by the use of cameras, audio recorders, or mechanical inspection of products.

__ Decide who will conduct the actual oversight.

__ Decide on the methods you will use to conduct the oversight.

Some possible methods you might use with various oversight mechanisms:

• Individual and group interviews with administrators or managers and staff members, present and past participants, funders, and others affected by or familiar with the entity’s work.
• Casual conversation with staff members and administrators.
• Direct observation of the work of the entity.
• Embedding into the work of the entity, also known as participant observation.
• Study of organizational or agency files, records, and financial reports, as well as its history, funding sources, and philosophy.
• Research into practice models or theory that underlies the work of the entity.

__ Train those who will conduct the oversight if necessary. Some possible topics:

• Get to know the organization or entity you’re overseeing.
• Understand that people may not want you there, either because they don’t want oversight, or because you’re interrupting their work.
• Try to establish a rapport with people you interview or interact with. Be natural and respectful.
• Listen carefully to what people tell you, and ask follow-up questions where it’s appropriate.
• Use open-ended questions.
• Pay attention to body language, the environment, and to the interactions among people.
• Take careful notes in whatever way is easiest and most comfortable for you and others.
• Respect people’s time.
• Produce reports regularly, and share your findings with those who are subject to oversight.
• Praise where appropriate, but don’t blame or criticize.

__ Create an analysis and reporting system.

__ Determine how you will use oversight to maintain or improve the quality of your work or of the entity or situation you’re overseeing.

__ Evaluate the effectiveness of your oversight mechanism and adjust it as necessary.

__ Continue oversight over the long term.

• What do we mean by formal public reporting processes?

• Why create formal public reporting processes?

• Who should be involved in creating formal public reporting processes?

• When should you create formal public reporting processes?

• How do you create formal public reporting processes?

Community efforts often work best when community members have meaningful opportunities to participate. Not everyone can serve on a planning or oversight committee or take part directly in implementation, however. So how do you keep community members informed, involved, and able to contribute?

One important way is to ensure that community members have access to clear information about priority community issues, efforts to address them, progress being made, challenges encountered, and results. This section will help you develop communication and reporting processes that keep the community informed and involved, create opportunities for feedback, and strengthen accountability for the effort.

What do we mean by formal public reporting processes?

Reporting can simply mean sharing information. You might tell a friend about a meeting you attended or a class you took. These types of exchanges represent informal reporting.

A formal reporting process generally has specific characteristics:

• Formal reports are delivered on a regular or established schedule. Recipients may expect to receive periodic updates, such as quarterly and/or annual progress reports. Reporting should use consistent measures where appropriate so that changes and trends can be understood over time. Measures may be quantitative (number-based), qualitative (such as narratives, interviews, observations, or other descriptive information), or a combination of both.
• Formal reports should identify lessons learned, areas for improvement, and possible adjustments, and reporting should continue throughout the life of the effort.

Why create formal public reporting processes?

• To involve the community and build understanding, participation, and support. Formal reporting processes keep community members informed and create opportunities for them to participate, provide feedback, and influence the effort. Involving community members in designing the reporting process can also help ensure that the information is useful, accessible, and responsive to their priorities.
• To increase awareness of the issue and the effort. Clear and accessible communication can help people who are unfamiliar with the issue understand why it matters, what is being done, and how they can learn more or become involved.
• To hold the effort accountable. Public reporting should communicate accomplishments as well as challenges, limitations, unintended effects, and adjustments needed to improve the initiative's effectiveness.
• To keep track of progress. Routine reporting helps both community members and people involved in implementation understand what has been done, what has been learned, what progress has occurred, and what still needs attention.
• To collect new ideas for the effort. Community meetings and other ways of sharing information can also create opportunities for public feedback, which may identify new ideas, concerns, strengths, or adjustments that can improve the work.
• To be open and transparent. Being open about the effort's process, decisions, successes, challenges, and results can build trust and demonstrate accountability, while giving community members information they can use to assess the effort for themselves.

Who should be involved in creating formal public reporting processes?

Formal reporting is most useful when it provides meaningful information to the people and communities affected by an effort. People who are affected by, involved in, or able to contribute to the effort should therefore have opportunities to help shape the reporting process. They can identify what information is most useful, how it should be shared, what questions need to be answered, and how reporting can be more transparent and accessible.

Some of the people and groups you might involve include:

• People and communities most directly affected by the issue or effort
• Local and other relevant officials
• Local businesses and employers
• Community advocates and organizations, particularly those already involved with the issue
• People whose work, responsibilities, or services may be affected by changes resulting from the effort (for example, law enforcement personnel may need additional training related to responses to domestic violence)
• People who will implement the effort

When should you create a formal public reporting process?

Ideally, a formal reporting process should be established when an organization or effort begins implementation so that community members can understand its work, follow its progress, and provide feedback over time. If that has not happened, there are other points when establishing a reporting process may be especially useful.

• When community participation or support is needed. If community advocacy, financial resources, volunteer participation, partnerships, or other contributions are important to the effort, people need clear information about its goals, progress, expected benefits, limitations, and use of resources.
• When there are questions about the effort. Questions may arise about the effort's necessity, cost, effectiveness, fairness, or implementation. A transparent reporting process can provide evidence, explain what is known and not yet known, and invite community members into meaningful discussion.
• When the issue the effort addresses has become urgent or may cause significant harm. Timely reporting can help community members understand the nature and scope of the issue, what actions are being taken, what uncertainties remain, and how people can participate in or respond to the effort.
• To alert and inform the community about emerging or ongoing issues and the actions being considered or taken in response.

How do you create formal public reporting processes?

Assemble a group to plan the reporting process

The group should include people representing the communities and organizations affected by or involved in the effort, as well as relevant advocates and partners. It may also be helpful to include people with communication expertise, such as journalists, communications professionals, interpreters, accessibility specialists, or community media representatives. The group should understand the intended audiences and their preferred ways of receiving information. Its task is to develop a reporting process that can reach and inform key audiences and the broader public while supporting accessibility, transparency, participation, and accountability.

Define the aim of the reporting process

In consultation with people and groups affected by or involved in the effort, decide what you want the reporting process to accomplish.

Some possibilities include:

• To comply with funders' or government agencies' reporting requirements
• To inform the public about the issue and your work
• To make sure that the effort gathers and analyzes appropriate information so that its work, progress, outcomes, challenges, and lessons learned are accurately documented and can inform ongoing improvement
• To hold the effort accountable
• To build public understanding and support for appropriate policy or community action
• To raise funds or other resources

Consult with and collect information from stakeholders

Remember that consultation should include people who are affected by the effort, people involved in carrying it out, and others who can contribute useful knowledge, perspectives, or resources to the reporting process.

There are a number of purposes for consulting with these individuals and groups:

• To understand what they already know about the issue
• To understand what they know about the effort and/or the organization, institution, or coalition
• To identify what information about the process, progress, results, and challenges would be useful to them
• To learn about community priorities, strengths, needs, concerns, and whether modifications to the effort or reporting process may be needed
• To understand what they see as the benefits, limitations, or challenges of public reporting
• To invite them, where appropriate, to participate in the reporting process
• To give community members meaningful opportunities to influence and participate in the effort
• To make the reporting process as fair, accessible, open, and accountable to the community as possible

To accomplish these purposes, you can conduct individual and group interviews, listening sessions, public forums, community meetings, surveys, and informal conversations with community members and other relevant groups.

Identify and prioritize the various audiences you want to reach

It is important to know who you are trying to reach and what kinds of information will be most useful to each audience. Community audiences may differ in how they are affected by or connected to the issue, their level of interest, their existing knowledge, their preferred communication channels, the languages they use, their accessibility needs, their level of trust or support, and their ability to influence or participate in decisions.

Some of the audiences you may want to reach include:

• People and communities affected by the issue
• People who use particular languages or communication formats
• Elected officials, public officials, and policymakers
• Other community leaders, including leaders of businesses and institutions and community members who are trusted or respected for their knowledge, service, relationships, or experience
• The general public
• Public and private funders, which may include local, national, or international foundations, agencies, or development organizations

You may decide to prioritize audiences depending on the goals of the reporting process, while making sure that people most directly affected by the issue are not excluded simply because they have less formal power or influence.

Understand the interests and preferred communication methods of your intended audiences

Different audiences may need different information. People directly affected by an issue may want clear information about what the effort means for them, while policymakers may need information about outcomes, community priorities, costs, implementation, and possible policy implications. Tailoring information should make it more relevant and accessible without changing or selectively presenting facts in ways that could mislead different audiences.

Another important consideration is how different audiences prefer or are able to receive information. Some people may rely on television or radio, others on newspapers, websites, email, social media, text messaging, printed materials, or trusted community sources such as faith leaders, community health workers, educators, or local organizations. In communities where digital or mass-media access is limited, in-person meetings, local gatherings, community radio, or other locally appropriate approaches may be more effective. Ask community members which channels they use and trust rather than assuming that one method will work for everyone.

Timing is also important. Consider when people are most likely to have the time and opportunity to engage with reports. Community events, legislative sessions, organizational planning cycles, or other relevant periods may provide useful opportunities for sharing information, as long as timing does not compromise accuracy or transparency.

Tailor your reporting to each audience

Once you have identified priority audiences and learned about their interests, information needs, languages, accessibility needs, and preferred communication methods, design reporting materials that are clear and useful to them. Use accessible language, relevant examples, appropriate formats, and communication channels that people are likely to use and trust. Tailoring should improve understanding and access rather than manipulate how information is interpreted.

For example, a report on the progress of an immunization program for Spanish-speaking families in a U.S. community might be shared in Spanish through trusted Spanish-language media, community organizations, health workers, and digital channels. A similar effort in a rural community elsewhere might use the locally preferred language and communication channels, such as community radio, local gatherings, health workers, or market-day outreach. The appropriate approach should be determined with the community rather than assumed in advance.

Plan an ongoing communications campaign that includes annual reports

The reporting process should be part of an overall communication strategy that may include:

• Clear and memorable information about the issue and the effort
• Stories and experiences, shared with appropriate consent, that illustrate community strengths, challenges, progress, and lessons learned
• Invitations for people to participate or contribute through advocacy, volunteering, partnerships, funding, or in-kind support
• Regular periodic progress reports about the effort
• An annual comprehensive report on the state of the effort and the issue, including accomplishments, challenges, lessons learned, and possible adjustments for the next year

These communication elements should be shared through multiple channels and formats so that people with different communication preferences and access needs can receive them. When appropriate, trusted community messengers – such as community leaders, faith leaders, educators, health workers, advocates, media figures, athletes, or other respected community members – can help share information. Choose messengers with input from the audiences you hope to reach.

Test your communications and public reports with the intended audiences

This is a three-step process:

• Share draft materials with members of the intended audiences through focus groups, interviews, user testing, or other feedback methods
• Adjust messages and formats to improve clarity, accessibility, accuracy, relevance, and usefulness based on feedback
• Adjust the overall communication plan based on feedback, available resources, effectiveness, and accessibility

Assess and respond to public understanding and perspectives about the effort

Based on your testing and ongoing engagement, determine what community members know, understand, and think about your effort. Do they have access to accurate and sufficient information? What questions or concerns do they have? Do they understand how the issue affects different parts of the community? Do they see value in the effort, and where do they believe it needs improvement? Surveys, public forums, focus groups, listening sessions, and interviews can deepen your understanding of these perspectives.

Once you understand community perspectives, respond to gaps in information, misinformation, questions, criticism, and disagreement with clear and accurate information. Use public education activities, meetings and forums, presentations to community groups, websites, email lists, social media, and other appropriate communication channels. The goal is not simply to persuade people to support the effort, but to create informed discussion and give people meaningful opportunities to raise concerns and influence improvement.

If concerns, objections, or criticism identify legitimate weaknesses in the effort – such as gaps in planning, inequitable effects, barriers to participation, or implementation problems – acknowledge them and determine how they can be addressed. Transparency about challenges can strengthen accountability and trust more than defending approaches that are not working.
 
Developing constructive relationships with local and community media can also help make accurate information about the effort available to the public and create opportunities for community questions and discussion.

Engage public officials about the issues and the effort

Legislators and policymakers can help raise awareness of an issue, connect community concerns with policy discussions, support appropriate funding, and influence policies that affect long-term conditions. Public reporting can provide them with evidence, community perspectives, and information they need to make informed decisions.

• Identify potential legislative or policy champions. An elected or appointed official who understands the issue and is willing to engage with community perspectives may be able to help bring attention to the effort and related policy needs. Personal or professional experience with the issue may strengthen that interest, but community priorities and evidence should remain central.
• Inform policymakers about the issue and your effort. Share evidence, community perspectives, program results, challenges, and policy implications. Supportive policymakers may help arrange presentations, connect organizations with relevant committees or agencies, and bring community concerns into policy discussions.
• Mobilize your local partners. Encourage people affected by the issue, community organizations, advocates, service providers, and other partners to communicate with policymakers when appropriate. Coordinated and informed public participation can help policymakers understand community priorities, experiences, and concerns.

Implement the communication and reporting campaign

Develop action plans for the communication and reporting effort, dividing responsibilities among participants and identifying who will be involved, what the timeline is, what resources are needed, and what outcomes are intended. Then use the communication and reporting formats you have developed and tested to share progress, results, challenges, lessons learned, and implications for future work.

Encourage your intended audiences to use the reports to carry the effort further

Use reports in discussions with community members, partners, decision-makers, and other relevant groups to reflect on and interpret results and discuss what they mean for future work. What adjustments do the findings suggest? What evidence supports those adjustments? What perspectives or information may still be missing?

Use accessible language and explain information in ways that are clear to the intended audience. Invite feedback about how to improve the effort, and when you ask for feedback, explain how it will be considered. Implement appropriate modifications when possible and communicate those changes. When recommendations cannot be adopted, explain the reasoning transparently. Ultimately, the goal is to use reporting and community feedback together to improve the effectiveness, accessibility, equity, and responsiveness of the effort.

Continue the reporting and communications campaign

A public reporting process is not a one-time activity. It should continue over the long term so that community members can follow progress, ask questions, contribute ideas, assess accountability, and remain meaningfully involved as the effort evolves.

In Summary

Throughout a community effort, it is important to establish and maintain communication with people and groups affected by or involved in the work and with the broader community. Effective communication is two-way: organizations share information, and community members have meaningful opportunities to ask questions, provide feedback, contribute knowledge, and influence decisions.

Formal public reporting is a more structured part of that communication. It provides information about the state and progress of an effort, what has been learned, what results have occurred, what challenges remain, and what changes may be needed. When community feedback is taken seriously and visibly informs decisions and improvements, reporting can strengthen trust, shared investment, accountability, and participation. Over time, these processes can help create both a more effective effort and a better-informed community prepared to address future issues collaboratively.

What do we mean by formal public reporting processes?

___It is conducted on a regular basis

___It is based on consistent measures

___It includes suggestions for adjustment and change in order to improve the effort

___It allows for feedback from its audience

___It continues throughout the life of the effort

Why create formal public reporting processes?

___To involve the community and garner support

___To bring the issue to the fore and raise the profile of the effort

___To hold the effort accountable

___To keep track of progress

___To collect new ideas for the effort

___To be open and transparent

Who should be involved in creating formal public reporting processes?

___Those who are most affected by the issue, and who would therefore stand to benefit from a successful effort

___Local and other relevant officials

___The business community

___Community activists, particularly those who have been involved with the issue at hand

___Those who might see changes in their work if the effort gains momentum

___Those who will actually implement the effort

When should you create formal public reporting processes?

___When community support is needed

___When there are questions about the effort

___When the issue that the effort addresses has reached, or threatens to reach, dangerous proportions

___To alert the community to one or more issues that need addressing, and to the fact that something is being done about them

How do you create formal public reporting processes?

___Assemble a representative group to plan the reporting process

___Define the aim of the reporting process

___Consult with and collect information from stakeholders and others who would benefit from and/or could contribute to the public reporting process

___Identify and prioritize the various audiences you want to reach, and what types of messages and information each might receive

___Understand the interests and preferred communication methods of those the report is aimed at

___Tailor your reporting to each audience you want to reach

___Plan an ongoing communications campaign that includes annual reports

___Test your communications and public reports with their intended audiences during development

___Assess and address public opinion about the effort

___Educate public officials about the issues and the effort

___Implement the communication and reporting campaign

___Encourage your intended audiences to use the reports to carry the effort further

___Continue the reporting and communications campaign over the life of the effort