Search form

Section 5. Developing an Evaluation Plan

    Example 1: Table of Program Components and Elements

    Here are a few of the items from the table of program components and elements for a fictional teen pregnancy prevention program:

    Risk / Protective Factors Program Components (Strategies) Program Elements (Tactics) Targets and Agents of Change Mode / Context of Delivery
    Risk factor: Unsupervised activities by teens Provide alternative activities
    • After-school activities (sports programs, hobby / craft classes, movies, etc.)
    • Weekend "lock-ins", non-alcoholic teen dance club
    • Daytime activities during school holidays

    Targets:
     

    • Youth
       

    Agents:

    • Youth
    • Teachers
    • Parents
    • Group sessions
    • Workshops
    • Meetings
    Risk factor: Lack of access to contraceptives Increased access to health services
    • Extend hours at school health clinics
    • Increase condom distribution by county health department
    • Brown bag program at area pharmacies (teens wishing to buy contraceptives are able to call ahead and pick them up in a plain brown bag)

    Targets:
     

    • Youth

    Agents:

    • Health providers
    • Walk-in clinics
    • Health Department
    • Businesses
    Protective factor: Involvement in the faith community Encourage youth involvement in the faith community
    • Increase church-sponsored youth activities
    • Institute church-based sexuality education
    • Church-sponsored parent-child communication sessions

    Targets:

    • Youth
    • Parents

    Agents:

    • Youth
    • Parents
    • Religious leaders
    • Discussions
    • Meetings
    • Educational sessions
    • Workshops

    Example 2: Evaluation Questions and Methods

    Here is a sample table of some of the questions from the evaluation plan of Youth Enrichment Services (Y.E.S.), an organization that provides urban young people with services that encourage them to explore, challenge themselves physically and mentally, and interact with positive role models:

        Type of Data Collection* Experimental Design**
    Key Evaluation Question(s) Type of Management Information and Evaluation Measure(s) Survey / Scale Structured interview Self Report / Log Direct Observation Archival Records Case Study Pre- and Post-test Control Group Time Series
    Planning and Implementation Issues Descriptive and Process Measures                
    1. Who participates? Is there diversity among the participants? 1. Basic demographic data (e.g., age, gender, disability, race, economic status, etc.) on participants. X X       X    
    2. Why do participants enter and leave the program? 2. Information on participants' reasons for entering and leaving the program. X X       X    
    3. Are there a variety of service and alternative activities generated? 3. Frequency of different types of service provided and alternative activities generated.     X X X X    
    Assessing Attainment of Objectives Outcome Measures                
    1. How many people participate? 1. Number of people participating.     X X X X   X
    2. How many hours are participants involved in service and activities? 2. Number of hours of participation by type of activity.     X X X X   X
    3. How many people are trained in cultural awareness and sensitivity? 3. Number of people trained in cultural awareness and sensitivity.     X X X X   X
    Impact on Participants                  
    1. How do behavior and attitudes change as a result of participation in the program? 1. Changes in programs, policies, and practices of affiliated organizations, and other outcomes. X X X X X   X X
    2. Does participation affect the incidence of problems? 2. Incidence of problems in living (e.g., substance use, gang involvement). X X     X   X  
    3. Are participants satisfied with the experience? 3. Satisfaction ratings from participants. X         X    
    Impact on Community                  
    1.  What resulted from the program? 1. Changes in programs, policies, and practices of affiliated organizations, and other outcomes. X X X X X     X
    2. Do the benefits of the program outweigh the costs? 2. Cost benefit data.   X X   X X    
    3. Are community members satisfied with the participants and the service they provided? 3. Satisfaction ratings from beneficiaries and community members. X         X    

    * Please note that the categories of types of data collection use broader terms than we've used to describe evaluation methods; here are some examples of methods that fall under these categories:

    • Survey/Scale: i.e., member survey of satisfaction with goals, member survey of satisfaction with process, member survey of satisfaction with outcomes, and behavioral surveys
    • Structured Interview: i.e., interview with key participants
    • Staff Report/Log: i.e. monitoring system, goal attainment report
    • Direct Observation: This refers to the direct observation of behavior by researchers .
    • Archival Records: i.e., community-level indicators of impact

    **Here are explanations of these terms:

    • Case Study: an in-depth, start-to-finish analysis of a person, group, or community that focuses on developmental factors in relation to environment--good for answering "how" or "why" questions
    • Pre-/Posttest Control Group: monitoring a group of people who fit into your group's target population who you don't expose to your initiative, so that you can compare them with the people who were part of the initiative, helping you get a better idea of how much real impact you've made
    • Time Series: effects are measured from a long series of repeated measurements taken both before and after an initiative

    Example 3: Timeline for Evaluation Activities

    This is the timeline for an evaluation being done by Kansas City's Promise on their initiative to promote community and personal caring for youth:

      2010 2011 2012 2013
    Monitoring System J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
    Establish system [ - - - ]      
    Train participants / staff in data entry [ - - - ]      
    Maintain system
    (data collection and reporting)    		 [- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ]
    Critical events        
    Identify key people                  [ - - - ]                      [ - - - ]                        [ - - - ]                   [ - - - ]
    Conduct interviews                       [ - - ]                         [ - - ]                           [ - - ]                       [ - - ]
    Report results                    [ - - - ]                  [ - - - ]                     [ - - - ]                      [ - - - ]
    Community-Level Indicators        
    Identify local indicators                      [ - - - - ]      
    Identify comparison site                     [ - - - - ]      
    Secure data                          [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ]
    Report results                  [ - - - ]                  [ - - - ]                     [ - - - ]                 [ - - - ]
    Other Indicators (e.g., behavioral surveys,
    data observations)    		        
    Identify                   [ - - - - - - - - - - - - ]    
    Secure data                       [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -]
    Report results                  [ - - - ]                 [ - - - ]                     [ - - - ]                     [ - - - ]