What do we mean by a design for the evaluation? ___The design of the evaluation is the arrangement that will make sure it reliably tells you what you need to know ___An appropriate design will show you whether you actually got results, and whether those results were likely due to your actions or the circumstances you created, or to other factors Why should you choose a design for your evaluation? ___So your evaluation will be reliable ___So you can pinpoint areas you need to work on, as well as those that are successful ___So your results are credible ___So you can identify factors unrelated to what you’re doing that have an effect – positive or negative – on your results and on the lives of participants ___So you can identify unintended consequences (both positive and negative) and correct for them ___So you’ll have a coherent plan and organizing structure for your evaluation When should you choose a design for your evaluation? ___In the ideal, when you’re planning the program and evaluation, before you start any implementation ___In reality, for many organizations, at the beginning of a program cycle, or as a new group of participants enters Who should be involved in choosing a design for your evaluation? ___It’s extremely helpful to include someone with research experience to help you decide on an appropriate design, and to help you understand what each possible design entails How do you select an appropriate design for your evaluation? ___Take into consideration the necessary research considerations Understand threats to internal validity ( whether the intervention produced the change) History Maturation The effects of testing or observation on participants Changes in measurement Regression toward the mean The selection of participants The loss of data or participants The nature of change A combination of the effects of two or more of these Understand threats to external validity (generalizability of your findings) Interaction of testing or data collection and the program or intervention Interaction of selection procedures and the program or intervention The effects of the research arrangements The interference of multiple programs or interventions ___Understand common research designs Pre- and post- single group design Interrupted time series design with a single group (simple time series) Interrupted time series with multiple groups (multiple time series) Control or comparison group ___Choose a design ___Consider your evaluation questions ___Consider the nature of your program ___Consider what your participants and staff will consent to ___Consider your time constraints ___Consider your resources
