Tool: Framework for Evaluating Qualitative Studies
The Cabinet Office of the British government developed a framework for evaluating qualitative studies. The framework suggests 18 questions that can be applied to any piece of qualitative research to check its reliability. Not all 18 have to be answered positively, but the more that can be, the more likely the research is to be actually reliable, and accepted as such…at least according to the Cabinet Office. Here’s the list:
- How credible are the findings?
- How has knowledge or understanding been extended by the research?
- How well does the evaluation address its original aims and purpose?
- How well is the scope for drawing wider inference explained?
- How clear is the basis of evaluative appraisal?
- How defensible is the research design?
- How well defended are the sample design/target selection of cases/documents?
- How well is the eventual sample composition and coverage described?
- How well was the data collection carried out?
- How well has the approach to and formulation of analysis been conveyed?
- How well are the contexts of data sources retained and portrayed?
- How well has diversity of perspective and content been explored?
- How well has detail, depth and complexity (i.e. richness) of the data been conveyed?
- How clear are the links between data, interpretation and conclusions - i.e., how well can the route to any conclusions be seen?
- How clear and coherent is the reporting?
- How clear are the assumptions/theoretical perspectives/values that have shaped the form and output of the evaluation?
- What evidence is there of attention to ethical issues?
- How adequately has the research process been documented?
Contributor
Phil Rabinowitz