Thank you for the opportunity. I would like to know the methodology and procedures behind the development and evaluation of the toolbox. I have been unable to find any publications on this. I am a PhD candidate at Western University who is developing a toolkit for implementation in health care. Any response is greatly appreciated.
A quick Google Scholar search provides the following resources:
Fawcett, S. B., Francisco, V. T., Schultz, J. A., Berkowitz, B., Wolff, T. J., & Nagy, G. (2000). The Community Tool Box: a Web-based resource for building healthier communities. Public health reports, 115(2-3), 274.
Francisco, V. T., Fawcett, S. B., Schultz, J. A., Berkowitz, B., Wolff, T. J., & Nagy, G. (2001). Using Internet‐Based Resources to Build Community Capacity: The Community Tool Box [http://ctb. ukans. edu/]. American Journal of Community Psychology, 29(2), 293-300.
And I happen to know that the Global Journal of Community Psychology Practice has published an article about the Toolbox as well:
Holt, C. M., Fawcett, S. B., Schultz, J. A., Berkowitz, B., Wolff, T. J. Francisco, V. T. (2013). Building Community Practice Competencies Globally through the Community Tool Box. Global Journal of Community Psychology Practice, 4(4), 1-8. Retrieved Day/Month/Year, from (http://www.gjcpp.org/).
These will certainly get you started. I think if you have additional questions, you should feel free to email the folks at the Toolbox with additional questions: email@example.com
Hope this helps! Thanks for your interest in the Community Tool Box!
* * * * * * * * * * * * * * * * * * * * * *
Thank you for your very thoughtful questions about the Community Tool Box, and please excuse our belated reply.
Steve Fawcett and I have tried to respond to them briefly. Since there was a team that created the Community Tool Box, we are responding as founding team members who were there from the beginning, We have a sense of our origins and the thinking behind them.
Note that there are background readings that offer a fuller picture of the origins and design of the CTB and subsequent applications. Several reprints will be emailed to you following this updated response.
Was a framework/ model/ theory used to guide the development and evaluation of the toolbox/ toolkit? I don't mean the conceptual framework described in Francisco et al 2001, as this appears to refer to the content.
In our formation, theories, models, and frameworks were secondary considerations. Our main goal was and is to provide information on community health and development that readers can apply directly in their own community settings to improve community outcomes. In an effort to be ecumenical—to leave space for people to use their own frameworks for action—we did not tout a single framework for action. However, we did have a general framework in mind: http://ctb.ku.edu/en/get-started
Did the authors utilized an adult educational theory when developing the content?
We used elements of behavioral instruction to structure content for each section: intro, rationale, how-to steps, rationales, examples, checklist or task analyses.
Were any behaviour change theories used? i.e., the authors are trying to change audience behaviour. Was this measured?
Not explicitly. We were trying to change audience behavior, in the sense that we wanted readers to use information derived in practice. But we did not attempt to measure behavior change directly.
How were stakeholders chosen? Was a consensus method applied to choices made? Was there a Steering/ Advisory Group?
At the beginning, we formed an Advisory Group of perhaps 15 members. These were well-regarded professionals in the community health and development field who we knew personally, or at least knew of. We dialogued with them by e-mail and met with them in conference call quarterly for a few years.
Was feasibility, usability, etc evaluated?
Feasibility and usability were measured largely through measures of traffic on the site (e.g., unique sessions, bounce rate, time on site, use by individual section). For the past decade or so, we have been using Google Analytics to obtain more refined measures; we go over such feedback together on an (approximately) quarterly basis.
In addition, we have conducted and continue to conduct approximately-yearly surveys of the nearly 1000 users on our e-mail distribution list. The feedback on those surveys helps us to evaluate and sometimes modify our work. We are also guided by comments received in our Guest Book, a link to which has always been on our home page, as well as by comments directed to us at firstname.lastname@example.org
Was a process and/or outcome evaluation conducted?
See response to question above
There is some use of 'toolbox' and 'toolkit'. Are these seen as interchangeable? What definition is used?
The terms overlap, but are not interchangeable. “Tool Box” refers to the entire Community Tool Box site. “Toolkit” for us refers to one of 16 sets of specific instructional materials associated with a key community topic (assessment, planning, etc.) The “Toolkits” include task analyses and examples for practice routines that are common to multiple disciplines involved in community health and development (e.g., public health, community development, community psychology, international development).
When material / literature was collected, was systematic review methodology used? or something else.
I’m not sure I would say we used “systematic review methodology,” but in creating different content on the site, we looked for the best available sources at the time, and cited them as references.
Was there guidance from a specific dissemination theory?
Somewhat. Much of our training and experience as community development professionals does involve dissemination, and we have always attempted to apply that knowledge in publicizing our work. Ev Rogers’ diffusion theory and social marketing are two sources of influence: e.g., listen to audiences about their interests, design content that has relative advantage, make it easier and more rewarding to access and use the CTB, use examples to align to meet needs of those working on different issues, etc. See more on social marketing in the CTB: http://ctb.ku.edu/en/implement-social-marketing-effort
I hope these responses may be helpful to you. We surely appreciate your being in touch with us, and send best wishes for continued success in all the work you do.
Bill Berkowitz and Steve Fawcett for the Community Tool Box team