Section 5. Collecting and Analyzing Data

Learn how to collect your data and analyze it, figuring out what it means, so that you can use it to draw some conclusions about your work.

 

In previous sections of this chapter, we’ve discussed studying the issue, deciding on a research design, and creating an observational system for gathering information for your evaluation. Now it’s time to collect your data and analyze it – figuring out what it means – so that you can use it to draw some conclusions about your work. In this section, we’ll examine how to do just that.

What do we mean by collecting data?

Essentially, collecting data means putting your design for collecting information into operation. You’ve decided how you’re going to get information – whether by direct observation, interviews, surveys, experiments and testing, or other methods – and now you and/or other observers have to implement your plan. There’s a bit more to collecting data, however. If you are conducting observations, for example, you’ll have to define what you’re observing and arrange to make observations at the right times, so you actually observe what you need to. You’ll have to record the observations in appropriate ways and organize them so they’re optimally useful.

Recording and organizing data may take different forms, depending on the kind of information you’re collecting. The way you collect your data should relate to how you’re planning to analyze and use it. Regardless of what method you decide to use, recording should be done concurrent with data collection if possible, or soon afterwards, so that nothing gets lost and memory doesn’t fade.

Some of the things you might do with the information you collect include:

• Gathering together information from all sources and observations
• Making photocopies of all recording forms, records, audio or video recordings, and any other collected materials, to guard against loss, accidental erasure, or other problems
• Entering narratives, numbers, and other information into a computer program, where they can be arranged and/or worked on in various ways
• Performing any mathematical or similar operations needed to get quantitative information ready for analysis.  These might, for instance, include entering numerical observations into a chart, table, or spreadsheet, or figuring the mean (average), median (midpoint), and/or mode (most frequently occurring) of a set of numbers.
• Transcribing (making an exact, word-for-word text version of) the contents of audio or video recordings
• Coding data (translating data, particularly qualitative data that isn’t expressed in numbers, into a form that allows it to be processed by a specific software program or subjected to statistical analysis)
• Organizing data in ways that make them easier to work with.  How you do this will depend on your research design and your evaluation questions. You might group observations by the dependent variable (indicator of success) they relate to, by individuals or groups of participants, by time, by activity, etc. You might also want to group observations in several different ways, so that you can study interactions among different variables.

There are two kinds of variables in research.  An independent variable (the intervention) is a condition implemented by the researcher or community to see if it will create change and improvement. This could be a program, method, system, or other action.  A dependent variable is what may change as a result of the independent variable or intervention.  A dependent variable could be a behavior, outcome, or other condition.  A smoking cessation program, for example, is an independent variable that may change group members’ smoking behavior, the primary dependent variable.

What do we mean by analyzing data?

Analyzing information involves examining it in ways that reveal the relationships, patterns, trends, etc. that can be found within it. That may mean subjecting it to statistical operations that can tell you not only what kinds of relationships seem to exist among variables, but also to what level you can trust the answers you’re getting.  It may mean comparing your information to that from other groups (a control or comparison group, statewide figures, etc.), to help draw some conclusions from the data. The point, in terms of your evaluation, is to get an accurate assessment in order to better understand your work and its effects on those you’re concerned with, or in order to better understand the overall situation.

There are two kinds of data you’re apt to be working with, although not all evaluations will necessarily include both. Quantitative data refer to the information that is collected as, or can be translated into, numbers, which can then be displayed and analyzed mathematically. Qualitative data are collected as descriptions, anecdotes, opinions, quotes, interpretations, etc., and are generally either not able to be reduced to numbers, or are considered more valuable or informative if left as narratives. As you might expect, quantitative and qualitative information needs to be analyzed differently.

Quantitative data

Quantitative data are typically collected directly as numbers. Some examples include:

• The frequency (rate, duration) of specific behaviors or conditions
• Test scores (e.g., scores/levels of knowledge, skill, etc.)
• Survey results (e.g., reported behavior, or outcomes to environmental conditions; ratings of satisfaction, stress, etc.)
• Numbers or percentages of people with certain characteristics in a population (diagnosed with diabetes, unemployed, Spanish-speaking, under age 14, grade of school completed, etc.)

Data can also be collected in forms other than numbers, and turned into quantitative data  for analysis. Researchers can count the number of times an event is documented in interviews or records, for instance, or assign numbers to the levels of intensity of an observed event or behavior. For instance, community initiatives often want to document the amount and intensity of environmental changes they bring about – the new programs and policies that result from their efforts. Whether or not this kind of translation is necessary or useful depends on the nature of what you’re observing and on the kinds of questions your evaluation is meant to answer.

Quantitative data is usually subjected to statistical procedures such as calculating the mean or average number of times an event or behavior occurs (per day, month, year). These operations, because numbers are “hard” data and not interpretation, can give definitive, or nearly definitive, answers to different questions. Various kinds of quantitative analysis can indicate changes in a dependent variable related to – frequency, duration, timing (when particular things happen), intensity, level, etc. They can allow you to compare those changes to one another, to changes in another variable, or to changes in another population. They might be able to tell you, at a particular degree of reliability, whether those changes are likely to have been caused by your intervention or program, or by another factor, known or unknown. And they can identify relationships among different variables, which may or may not mean that one causes another.

Qualitative data

Unlike numbers or “hard data,” qualitative information tends to be “soft,” meaning it can’t always be reduced to something definite. That is in some ways a weakness, but it’s also a strength. A number may tell you how well a student did on a test; the look on her face after seeing her grade, however, may tell you even more about the effect of that result on her. That look can’t be translated to a number, nor can a teacher’s knowledge of that student’s history, progress, and experience, all of which go into the teacher’s interpretation of that look. And that interpretation may be far more valuable in helping that student succeed than knowing her grade or numerical score on the test.

Qualitative data can sometimes be changed into numbers, usually by counting the number of times specific things occur in the course of observations or interviews, or by assigning numbers or ratings to dimensions (e.g., importance, satisfaction, ease of use).

The challenges of translating qualitative into quantitative data have to do with the human factor.  Even if most people agree on what 1 (lowest) or 5 (highest) means in regard to rating “satisfaction” with a program, ratings of 2, 3, and 4 may be very different for different people.  Furthermore, the numbers say nothing about why people reported the way they did. One may dislike the program because of the content, the facilitator, the time of day, etc. The same may be true when you’re counting instances of the mention of an event, such as the onset of a new policy or program in a community based on interviews or archival records.  Where one person might see a change in program he considers important another may omit it due to perceived unimportance.

Qualitative data can sometimes tell you things that quantitative data can’t.  It may reveal why certain methods are working or not working, whether part of what you’re doing conflicts with participants’ culture, what participants see as important, etc.  It may also show you patterns – in behavior, physical or social environment, or other factors – that the numbers in your quantitative data don’t, and occasionally even identify variables that researchers weren’t aware of.

It is often helpful to collect both quantitative and qualitative information.

Quantitative analysis is considered to be objective – without any human bias attached to it – because it depends on the comparison of numbers according to mathematical computations.  Analysis of qualitative data is generally accomplished by methods more subjective – dependent on people’s opinions, knowledge, assumptions, and inferences (and therefore biases) – than that of quantitative data.  The identification of patterns, the interpretation of people’s statements or other communication, the spotting of trends – all of these can be influenced by the way the researcher sees the world.  Be aware, however, that quantitative analysis is influenced by a number of subjective factors as well.  What the researcher chooses to measure, the accuracy of the observations, and the way the research is structured to ask only particular questions can all influence the results, as can the researcher’s understanding and interpretation of the subsequent analyses.

Why should you collect and analyze data for your evaluation?

Part of the answer here is that not every organization – particularly small community-based or non-governmental ones – will necessarily have extensive resources to conduct a formal evaluation.  They may have to be content with less formal evaluations, which can still be extremely helpful in providing direction for a program or intervention.  An informal evaluation will involve some data gathering and analysis. This data collection and sensemaking is critical to an initiative and its future success, and has a number of advantages.

• The data can show whether there was any significant change in the dependent variable(s) you hoped to influence. Collecting and analyzing data helps you see whether your intervention brought about the desired results

The term “significance” has a specific meaning when you’re discussing statistics.  The level of significance of a statistical result is the level of confidence you can have in the answer you get.  Generally, researchers don’t consider a result significant unless it shows at least a 95% certainty that it’s correct (called the .05 level of significance, since there’s a 5% chance that it’s wrong).  The level of significance is built into the statistical formulas: once you get a mathematical result, a table (or the software you’re using) will tell you the level of significance.

Thus, if data analysis finds that the independent variable (the intervention) influenced the dependent variable at the .05 level of significance, it means there’s a 95% probability or likelihood that your program or intervention had the desired effect.  The .05 level is generally considered a reasonable result, and the .01 level (99% probability) is considered about as close to certainty as you are likely to get. A 95% level of certainty doesn’t mean that the program works on 95% of participants, or that it will work 95% of the time.  It means that there’s only a 5% possibility that it isn’t actually what’s influencing the dependent variable(s) and causing the changes that it seems to be associated with.

• They can uncover factors that may be associated with changes in the dependent variable(s). Data analyses may help discover unexpected influences; for instance, that the effort was twice as large for those participants who also were a part of a support group. This can be used to identify key aspects of implementation.
• They can show connections between or among various factors that may have an effect on the results of your evaluation. Some types of statistical procedures look for connections (“correlations” is the research term) among variables.  Certain dependent variables may change when others do.  These changes may be similar – i.e., both variables increase or decrease (e.g., as children’s proficiency at reading increases, the amount of reading they do also increases).  Or the opposite may be observed – i.e. the two variables change in opposite directions (as the amount of exercise they engage in increases, peoples’ weight decreases).  Correlations don’t mean that one variable causes another, or that they both have the same cause, but they can provide valuable information about associations to expect in an evaluation.
• They can help shed light on the reasons that your work was effective or, perhaps, less effective than you’d hoped.  By combining quantitative and qualitative analysis, you can often determine not only what worked or didn’t, but why.  The effect of cultural issues, how well methods are used, the appropriateness of your approach for the population – these as well as other factors that influence success can be highlighted by careful data collection and analysis.  This knowledge gives you a basis for adapting and changing what you do to make it more likely you’ll achieve the desired outcomes in the future.
• They can provide you with credible evidence to show stakeholders that your program is successful, or that you’ve uncovered, and are addressing limitations. Stakeholders, such as funders and community boards, want to know their investments are well spent. Showing evidence of intermediate outcomes (e.g. new programs and policies) and longer-term outcomes (e.g., improvements in education or health indicators) is becoming increasingly important to receiving – and retaining – funding.
• Their use shows that you’re serious about evaluation and about improving your work. Being a good trustee or steward of community investment includes regular review of data regarding progress and improvement.
• They can show the field what you’re learning, and thus pave the way for others to implement successful methods and approaches. In that way, you’ll be helping to improve community efforts and, ultimately, quality of life for people who benefit.

When and by whom should data be collected and analyzed?

As far as data collection goes, the “when” part of this question is relatively simple: data collection should start no later than when you begin your work – or before you begin in order to establish a baseline or starting point – and continue throughout.  Ideally, you should collect data for a period of time before you start your program or intervention in order to determine if there are any trends in the data before the onset of the intervention. Additionally, in order to gauge your program’s longer-term effects, you should collect follow-up data for a period of time following the conclusion of the program.

The timing of analysis can be looked at in at least two ways: One is that it’s best to analyze your information when you’ve collected all of it, so you can look at it as a whole. The other is that if you analyze it as you go along, you’ll be able to adjust your thinking about what information you actually need, and to adjust your program to respond to the information you’re getting. Which of these approaches you take depends on your research purposes.  If you’re more concerned with a summative evaluation – finding out whether your approach was effective, you might be more inclined toward the first.  If you’re oriented toward improvement – a formative evaluation – we recommend gathering information along the way. Both approaches are legitimate, but ongoing data collection and review can particularly lead to improvements in your work.

The “who” question can be more complex. If you’re reasonably familiar with statistics and statistical procedures, and you have the resources in time, money, and personnel, it’s likely that you’ll do a somewhat formal study, using standard statistical tests. (There’s a great deal of software – both for sale and free or open-source – available to help you.)

If that’s not the case, you have some choices:

• You can hire or find a volunteer outside evaluator, such as from a nearby college or university, to take care of data collection and/or analysis for you.
• You can conduct a less formal evaluation. Your results may not be as sophisticated as if you subjected them to rigorous statistical procedures, but they can still tell you a lot about your program.  Just the numbers – the number of dropouts (and when most dropped out), for instance, or the characteristics of the people you serve – can give you important and usable information.
• You can try to learn enough about statistics and statistical software to conduct a formal evaluation yourself. (Take a course, for example.)
• You can collect the data and then send it off to someone – a university program, a friendly statistician or researcher, or someone you hire – to process it for you.
• You can collect and rely largely on qualitative data.  Whether this is an option depends to a large extent on what your program is about. You wouldn’t want to conduct a formal evaluation of effectiveness of a new medication using only qualitative data, but you might be able to draw some reasonable conclusions about use or compliance patterns from qualitative information.
• If possible, use a randomized or closely matched control group for comparison. If your control is properly structured, you can draw some fairly reliable conclusions simply by comparing its results to those of your intervention group.  Again, these results won’t be as reliable as if the comparison were made using statistical procedures, but they can point you in the right direction.  It’s fairly easy to tell whether or not there’s a major difference between the numbers for the two or more groups.  If 95% of the students in your class passed the test, and only 60% of those in a similar but uninstructed control group did, you can be pretty sure that your class made a difference in some way, although you may not be able to tell exactly what it was that mattered. By the same token, if 72% of your students passed and 70% of the control group did as well, it seems pretty clear that your instruction had essentially no effect, if the groups were starting from approximately the same place.

Who should actually collect and analyze data also depends on the form of your evaluation.  If you’re doing a participatory evaluation, much of the data collection - and analyzing - will be done by community members or program participants themselves. If you’re conducting an evaluation in which the observation is specialized, the data collectors may be staff members, professionals, highly trained volunteers, or others with specific skills or training (graduate students, for example).  Analysis also could be accomplished by a participatory process. Even where complicated statistical procedures are necessary, participants and/or community members might be involved in sorting out what those results actually mean once the math is done and the results are in. Another way analysis can be accomplished is by professionals or other trained individuals, depending upon the nature of the data to be analyzed, the methods of analysis, and the level of sophistication aimed at in the conclusions.

How do you collect and analyze data?

Whether your evaluation includes formal or informal research procedures, you’ll still have to collect and analyze data, and there are some basic steps you can take to do so.

Implement your measurement system

We've previously discussed designing an observational system to gather information. Now it’s time to put that system in place.

• Clearly define and describe what measurements or observations are needed. The definition and description should be clear enough to enable observers to agree on what they’re observing and reliably record data in the same way.
• Select and train observers. Particularly if this is part of a participatory process, observers need training to know what to record; to recognize key behaviors, events, and conditions; and to reach an acceptable level of inter-rater reliability (agreement among observers).
• Conduct observations at the appropriate times for the appropriate period of time. This may include reviewing archival material; conducting interviews, surveys, or focus groups; engaging in direct observation; etc.
• Record data in the agreed-upon ways. These may include pencil and paper, computer (using a laptop or handheld device in the field, entering numbers into a program, etc.), audio or video, journals, etc.

Organize the data you’ve collected

How you do this depends on what you’re planning to do with it, and on what you’re interested in.

• Enter any necessary data into the computer. This may mean simply typing comments, descriptions, etc., into a word processing program, or entering various kinds of information (possibly including audio and video) into a database, spreadsheet, a GIS (Geographic Information Systems) program, or some other type of software or file.
• Transcribe any audio- or videotapes. This makes them easier to work with and copy, and allows the opportunity to clarify any hard-to-understand passages of speech.
• Score any tests and record the scores appropriately.
• Sort your information in ways appropriate to your interest. This may include sorting by category of observation, by event, by place, by individual, by group, by the time of observation, or by a combination or some other standard.
• When possible, necessary, and appropriate, transform qualitative into quantitative data. This might involve, for example, counting the number of times specific issues were mentioned in interviews, or how often certain behaviors were observed.

Conduct data graphing, visual inspection, statistical analysis, or other operations on the data as appropriate

We’ve referred several times to statistical procedures that you can apply to quantitative data. If you have the right numbers, you can find out a great deal about whether your program is causing or contributing to change and improvement, what that change is, whether there are any expected or unexpected connections among variables, how your group compares to another you’re measuring, etc.

There are other excellent possibilities for analysis besides statistical procedures, however. A few include:

• Simple counting, graphing and visual inspection of frequency or rates of behavior, events, etc., over time.
• Using visual inspection of patterns over time to identify discontinuities (marked increases, decreases) in the measures over time (sessions, weeks, months).
• Calculating the mean (average), median (midpoint), and/or mode (most frequent) of a series of measurements or observations. What was the average blood pressure, for instance, of people who exercised 30 minutes a day at least five days a week, as opposed to that of people who exercised two days a week or less?
• Using qualitative interviews, conversations, and participant observation to observe (and track changes in) the people or situation. Journals can be particularly revealing in this area because they record people’s experiences and reflections over time.
• Finding patterns in qualitative data.  If many people refer to similar problems or barriers, these may be important in understanding the issue, determining what works or doesn’t work and why, or more.
• Comparing actual results to previously determined goals or benchmarks. One measure of success might be meeting a goal for planning or program implementation, for example.

Take note of any significant or interesting results

Depending on the nature of your research, results may be statistically significant (the 95% or better certainty that we discussed earlier), or simply important or unusual.  They may or may not be socially significant (i.e., large enough to solve the problem).

There are a number of different kinds of results you might be looking for.

• Differences within people or groups. If you have repeated measurements for individuals/groups over time, we can see if there are marked increases/decreases in the (frequency, rate) of behavior (events, etc.) following introduction of the program or intervention. When the effects are seen when and only when the intervention is introduced – and if the intervention is staggered (delayed) across people or groups – this increases our confidence that the intervention, and not something else, is producing the observed effects.
• Differences between or among two or more groups.  If you have one or more randomized control groups in a formal study (groups that are drawn at random from the same population as the group in your program, but are not getting the same program or intervention, or are getting none at all), then the statistical significance of differences between or among the groups should tell you whether your program has any more influence on the dependent variable(s) than what’s experienced by the other groups.
• Results that show statistically significant changes. With or without a control or comparison group, many statistical procedures can tell you whether changes in dependent variables are truly significant (or not likely due to chance). These results may say nothing about the causes of the change (or they may, depending on how you’ve structured your evaluation), but they do tell you what’s happening, and give you a place to start.
• Correlations. Correlation means that there are connections between or among two or more variables. Correlations can sometimes point to important relationships you might not have predicted. Sometimes they can shed light on the issue itself, and sometimes on the effects of a group’s cultural practices. In some cases, they can highlight potential causes of an issue or condition, and thus pave the way for future interventions.

Correlation between variables doesn’t tell you that one necessarily causes the other, but simply that changes in one have a relationship to changes in the other.  Among American teenagers, for instance, there is probably a fairly high correlation between an increase in body size and an understanding of algebra.  This is not because one causes the other, but rather the result of the fact that American schools tend to begin teaching algebra in the seventh, eighth, or ninth grades, a time when many 12-, 13-, and 14-year-olds are naturally experiencing a growth spurt.

On the other hand, correlations can reveal important connections.  A very high correlation between, for instance, the use of a particular medication and the onset of depression might lead to the withdrawal of that medication, or at least a study of its side effects, and increased awareness and caution among doctors who prescribe it.  A very high correlation between gang membership and having a parent with a substance use problem may not reveal a direct cause-and-effect relationship, but may tell you something important about who is more at risk for substance use.

• Patterns. In both quantitative and qualitative information, patterns often emerge: certain health conditions seem to cluster in particular geographical areas; people from a particular group behave in similar ways; etc.  These patterns may not be specifically what you were looking for or expected to find, but they may either be important in themselves or shed light on the areas you’re interested in.  In some cases, you may need to subject them to statistical procedures (regression analysis, for example) to see if, in fact, they’re random, or if they constitute actual patterns.
• Obvious important findings.  Whether as a result of statistical analysis, or of examination of your data and application of logic, some findings may stand out.  If 70% of a group of overweight participants in a healthy eating and physical activity program lowered their weight and blood pressure significantly, compared to only 20% of a similar group not in the program, you can probably assume that program may have been effective.  If there’s no change whatsoever in education outcomes after two years of your education program, then you’re either running an ineffective program, or you’re simply not reaching those who are most likely to have poorer outcomes (which can also be interpreted to mean you’re running an ineffective program.)

Not all important findings will necessarily tell you whether your program worked, or what is the most effective method.  It might be obvious from your data collection, for instance, that, while violence or roadway injuries may not be seen as a problem citywide, they are much higher in one or more particular areas, or that the rates of diabetes are markedly higher for particular groups or those living in areas with greater disparities of income. If you have the resources, it’s wise to look at the results of your research in a number of different ways, both to find out how to improve your program, and to learn what else you might do to affect the issue.

Interpret the results

Once you’ve organized your results and run them through whatever statistical or other analysis you’ve planned for, it’s time to figure out what they mean for your evaluation. Probably the most common question that evaluation research is directed toward is whether the program being evaluated works or makes a difference. In research terms, that often translates to “What were the effects of the independent variable (the program, intervention, etc.) on the dependent variable(s) (the behavior, conditions, or other factors it was meant to change)?”  There are a number of possible answers to this question:

• Your program had exactly the effects on the dependent variable(s) you expected and hoped it would. Statistics or other analysis showed clear positive effects at a high level of significance for the people in your program and – if you used a multiple-group design – none, or far fewer, of the same effects for a similar control group and/or for a group that received a different intervention with the same purpose. Your early childhood education program, for instance, greatly increased development outcomes for children in the community, and also contributed to an increase in the percentage of children succeeding in school.
• Your program had no effect.  Your program produced no significant results on the dependent variable, whether alone or compared to other groups. This would mean no change as a result of your program or intervention.
• Your program had a negative effect. For instance, intimate partner violence increased (or at least appeared to) as a result of your intervention. (It is relatively common for reported events, such as violence or injury, to increase when the intervention results in improved surveillance and ease of reporting).
• Your program had the effects you hoped for and other effects as well.
  • These effects might be positive. Your youth violence prevention program, for instance, might have resulted in greatly reduced violence among teens, and might also have resulted in significantly improved academic performance for the kids involved.
  • These effects might be neutral. The same youth violence prevention program might somehow result in youth watching TV more often after school.
  • These effects might be negative. (These effects are usually called unintended consequences.) Youth violence might decrease significantly, but the incidence of teen pregnancies or alcohol consumption among youth in the program might increase significantly at the same time.
  • These effects might be multiple, or mixed.For instance, a program to reduce HIV/AIDS might lower rates of unprotected sex but might also increase conflict and instances of partner violence. Your program had no effect or a negative effect and other effects as well. As with programs with positive effects, these might be positive, neutral, or negative; single or multiple; or consistent or mixed.

If your analysis gives you a clear indication that what you’re doing is accomplishing your purposes, interpretation is relatively simple: You should keep doing it, while trying out ways to make it even more effective, or while aiming at other related issues as well.

As we discuss elsewhere in the Community Tool Box, good programs are dynamic -- constantly striving to improve, rather than assuming that what they’re doing is as good as it can be.

If your analysis shows that your program is ineffective or negative, however – or, for that matter, if a positive analysis leaves you wondering how to make your successful efforts still more successful – interpretation becomes more complex. Are you using an absolutely wrong approach? Are you using an approach that could be effective, but is poorly implement? Is there a particular contributing factor you’re failing to take into account?  Are there barriers to success – of culture, experience, personal characteristics, systematic discrimination – present in the population from which participants are drawn?  Are there particular components or elements you can change to make your program more effective, or should you start again from scratch?  What should you address to make a good program better?

Careful and insightful interpretation of your data may allow you to answer questions like these. You may be able to use correlations, for instance, to generate hypotheses about your results. If positive or negative changes in particular variables are consistently associated with positive or negative changes in other variables, the two may be connected. (The word “may” is important here. The two may be connected, but they may not, or both may be related to a third variable that you’re not aware of or that you consider trivial.) Such a connection can point the way toward a factor (e.g., access to support) that is causing the changes in both variables, and that must be addressed to make your program successful. Correlations may also indicate patterns in your data, or may lead to an unexpected way of looking at the issue you’re addressing.

You can often use qualitative data to understand the meaning of an intervention, and people’s reactions to the results.The observation that participants are continually suffering from a variety of health problems may be traced, through qualitative data, to nutrition problems (due either to poverty or ignorance) or to lack of access to health services, or to cultural restrictions (some Muslim women may be unwilling – or unable because of family prohibition – to accept care and treatment from male doctors, for example).

Once you have organized your data, both statistical results and anything that can’t be analyzed statistically need to be analyzed logically. This may not give you convincing information but it will almost undoubtedly give you some ideas to follow up on, and some indications of connections and avenues you might not yet have considered. It will also show you some additional results – people reacting differently than before to the program, for example. The numbers can tell you whether there is change, but they can’t always tell you what causes it or why (although they sometimes can), or why some people benefit while others don’t. Those are often matters for logical analysis, or critical thinking.
 
Analyzing and interpreting the data you’ve collected brings you, in a sense, back to the beginning. You can use the information you’ve gained to adjust and improve your program or intervention, evaluate it again, and use that information to adjust and improve it further, for as long as it runs. You have to keep up the process to ensure that you’re doing the best work you can and encouraging changes in individuals, systems, and policies that make for a better and healthier community.

You have to become a cultural detective to understand your initiative, and, in some ways, every evaluation is an anthropological study.

In Summary

The heart of evaluation research is gathering information about the program or intervention you’re evaluating and analyzing it to determine what it tells you about the effectiveness of what you’re doing, as well as about how you can maintain and improve that effectiveness.

Collecting quantitative data – information expressed in numbers – and subjecting it to a visual inspection or formal statistical analysis can tell you whether your work is having the desired effect, and may be able to tell you why or why not as well.  It can also highlight connections (correlations) among variables, and call attention to factors you may not have considered.

Collecting and analyzing qualitative data – interviews, descriptions of environmental factors, or events, and circumstances – can provide insight into how participants experience the issue you’re addressing, what barriers and advantages they experience, and what you might change or add to improve what you do.

Once you’ve gained the knowledge that your information provides, it’s time to start the process again. Use what you’ve learned to continue to evaluate what you do by collecting and analyzing data, and continually improve your program.