Table of Contents >
   Part J. Evaluating Community Programs and Initiatives
      Chapter 36. Introduction to Evaluation >
         Section 1. A Framework for Program Evaluation: A Gateway to ... >
             Tools & Checklists - A checklist that summarizes the major points contained in the section. >

A Framework for Program Evaluation: A Gateway to Tools

  

Tools & Checklists

 Contributed by Bobby Milstein, Scott Wetterhall, and the CDC Evaluation Working Group Edited by Jenette Nagy and Stephen B. Fawcett

Tools

Tool 1: Tools Related to the Recommended Framework
Tool 2: Checklist for Ensuring Effective Evaluation Reports
Tool 3: Steps in Evaluation Practice and the Most Relevant Standards
Tool 4: Evaluation Standards Checklist

Checklist

Tools

Tool #1: Tools Related to the Recommended Framework.

By clicking on the steps in program evaluation listed below, you will find the Community Tool Box sections that are related to each step.

1. Engage Stakeholders2. Describe the Program3. Focus the Evaluation Design4. Gather Credible Evidence5. Justify Conclusions6. Ensure Use and Share Lessons LearnedFramework Showing Steps in Evaluation


1. Engage Stakeholders

See the following Community Tool Box sections:

To help you understand who stakeholders are, see:

To learn how to involve them, see:

To work together with a diverse group, see:


2. Describe the Program

See the following Community Tool Box sections:

To fully understand the need, problem, or goal the program addresses, see:

To better explain the activities, components, and elements of the program, see:

To be able to describe resources and assets for the program, see:

For examples of logic models, see:


To look broadly at your program and its context, see:

To modify the program to fit the local context, see:

To learn to explain the program to others (and so that they see your point of view), see:


3. Focus the Evaluation Design

To clarify the purpose, see:

To identify potential users and uses, see:

To clarify evaluation questions, see:

For illustrative evaluation questions, see:

For help identifying specific evaluation methods, see:

For guidance about making agreements, see:


4. Gather Credible Evidence

For support in implementing specific evaluation methods, see:


5. Justify Conclusions

To see an illustrative process for considering evidence, see:


6. Ensure Use and Share Lessons Learned

See the following Community Tool Box sections:

To promote the use of what your organization has learned, see:

To learn to share what you have learned with diverse groups, see:


Return to top


Tool #2: Checklist for Ensuring Effective Evaluation Reports

Does the report...

___Provide interim and final reports to intended users in time for intended uses
___Tailor the report content, format, and style for the audience(s) by involving audience members
___Include an executive summary
___Summarize the stakeholders and how they were engaged
___Describe essential features of the program (perhaps in appendices)
___Explain the focus of the evaluation and its limitations
___Include an adequate summary of the evaluation plan and procedures
___Provide all necessary technical information (perhaps in appendices)
___Specify the standards and criteria for evaluative judgments
___Explain the evaluative judgments and how they are supported by the evidence
___List both strengths and weaknesses of the evaluation
___Discuss recommendations for action along with their advantages and disadvantages
___Ensure protections for program clients and other stakeholders
___Anticipate how people or organizations may be affected by the findings
___Present minority opinions or rejoinders where necessary
___Verify that the report is accurate and unbiased
___Organize the report logically and include an appropriate level of detail
___Remove technical jargon
___Use examples, illustrations, graphics, and stories

* Adapted from: Worthen, BR, Sanders, JR, Fitzpatrick, JL. Program evaluation: alternative approaches and practical guidelines. 2nd edition. Addison, Wesley Logman, Inc. New York. 1997.


Return to top


Tool #3: Steps in Evaluation Practice and the Most Relevant Standards

Steps

Most Relevant Standards*

Engage stakeholders

Metaevaluation (A12)
Stakeholder identification (U1)
Evaluator credibility (U2)
Formal agreements (P2)
Rights of human subjects (P3)
Human interactions (P4)
Conflict of interest (P7)

Describe the program

Complete and fair assessment (P5)
Program documentation (A1)
Context analysis (A2)

Focus the evaluation design

Evaluation impact (U7)
Practical procedures (F1)
Political viability (F2)
Cost effectiveness (F3)
Service orientation (P1)
Complete and fair assessment (P5)
Fiscal responsibility (P8)
Described purposes and procedures (A3)

Gather credible evidence

Information scope and selection (U3)
Defensible information sources (A4)
Valid information (A5)
Reliable information (A6)
Systematic information (A7)

Justify conclusions

Values identification (U4)
Analysis of quantitative information (A8)
Analysis of qualitative information (A9)
Justified conclusions (A10)

Ensure use and

share lessons learned

Evaluator credibility (U2)
Report clarity (U5)
Report timeliness and dissemination (U6)
Evaluation impact (U7)
Disclosure of findings (P6)
Impartial reporting (A11)

* Codes following each standard designate the conceptual group and number of the standard.

U=Utility; F=Feasibility; P=Propriety; A=Accuracy.


Return to top

Tool #4: Evaluation Standards Checklist

Use this page to determine how well your evaluation met "good" standards for evaluation.

Evaluation Standards Checklist

Standard

Did the evaluation meet this standard? (Y/N)

Comments

Utility Standards

   

1. Stakeholder Identification

   

2. Evaluator Credibility

   

3. Information Scope and Selection

   

4. Values Identification

   

5. Report Clarity

   

6. Report Timeliness and Dissemination

   

7. Evaluation Impact

   

Feasibility Standards

   

1. Practical Procedures

   

2. Political Viability

   

3. Cost Effectiveness

   

Propriety Standards

   

1. Service Orientation

   

2. Formal Agreements

   

3. Rights of Human Subjects

   

4. Human Interactions

   

5. Complete and Fair Assessment

   

6. Disclosure of Findings

   

7. Conflict of Interest

   

8. Fiscal Responsibility

   

Accuracy Standards

   

1. Program Documentation

   

2. Context Analysis

   

3. Described Purposes and Procedures

   

4. Defensible Information Sources

   

5. Valid Information

   

6. Reliable Information

   

7. Systematic Information:

   

8. Analysis of Quantitative Information:

   

9. Analysis of Qualitative Information

   

10. Justified Conclusions

   

11. Impartial Reporting

   

12. Metaevaluation

   


Return to top


Checklist

Here, you'll find a checklist summarizing the major points contained in the text.


___You understand that evaluation is the systematic investigation of the merit, worth, or significance of an object or effort.

___You understand that a program is used to describe the object or effort that is being evaluated.

___You understand that program evaluation is a way to evaluate the specific projects and activities community groups may take part in.

___You understand that stakeholders are those who care about the program or effort.


You understand that program evaluation:

___Helps clarify program plans.

___Improves communication among partners.

___Gathers feedback needed to improve and be accountable for program effectiveness.


You know how to evaluate a specific program:

___You know what will be evaluated.

___You have decided what criteria will be used to judge program performance.

___You have decided what standards of performance must be reached for the program to be considered successful.

___You know what evidence will indicate performance.

___You know what conclusions about program performance are justified based on the available evidence.

___You have engaged stakeholders.

___You have described the program.

___You have focused the evaluation design.

___You understand that possible focuses are: to gain insight, to improve how things get done, to determine what the effects of the program are, and to effect those who participate in it.

___You have gathered credible evidence.

___You have justified your conclusions.

___You understand how to ensure the use of the information and share lessons learned.

___You know that there are four categories of evaluation standards: utility, feasibility, propriety, and accuracy.

___You understand the standards in each category.

___You understand which standards are most applicable to each step of evaluation practice.


Return to top