Search form

Section 4. Selecting an Appropriate Design for the Evaluation

What do we mean by a design for the evaluation?

___The design of the evaluation is the arrangement that will make sure it reliably tells you what you need to know

___An appropriate design will show you whether you actually got results, and whether those results were likely due to your actions or the circumstances you created, or to other factors

Why should you choose a design for your evaluation?

___So your evaluation will be reliable

___So you can pinpoint areas you need to work on, as well as those that are successful

___So your results are credible

___So you can identify factors unrelated to what you’re doing that have an effect – positive or negative – on your results and on the lives of participants

___So you can identify unintended consequences (both positive and negative) and correct for them

___So you’ll have a coherent plan and organizing structure for your evaluation

When should you choose a design for your evaluation?

___In the ideal, when you’re planning the program and evaluation, before you start any implementation

___In reality, for many organizations, at the beginning of a program cycle, or as a new group of participants enters

Who should be involved in choosing a design for your evaluation?

___It’s extremely helpful to include someone with research experience to help you decide on an appropriate design, and to help you understand what each possible design entails

How do you select an appropriate design for your evaluation?

___Take into consideration the necessary research considerations

  • Understand threats to internal validity ( whether the intervention produced the change)
    • History
    • Maturation
    • The effects of testing or observation on participants
    • Changes in measurement
    • Regression toward the mean
    • The selection of participants
    • The loss of data or participants
    • The nature of change
    • A combination of the effects of two or more of these
  • Understand threats to external validity (generalizability of your findings)
    • Interaction of testing or data collection and the program or intervention
    • Interaction of selection procedures and the program or intervention
    • The effects of the research arrangements
    • The interference of multiple programs or interventions

___Understand common research designs

  • Pre- and post- single group design
  • Interrupted time series design with a single group (simple time series)
  • Interrupted time series with multiple groups (multiple time series)
  • Control or comparison group

___Choose a design

___Consider your evaluation questions

___Consider the nature of your program

___Consider what your participants and staff will consent to

___Consider your time constraints

___Consider your resources