Evaluation is a process that "assesses the effectiveness of a program in achieving
its goals and/or objectives, uses methods to determine whether program outcomes
can be attributed to the program or other factors, and aims at program
improvement through modification of program operation and/or design."
(This definition comes from the Juvenile Justice Evaluation Center Online Glossary).
We hope these resources will help you evaluate and monitor your program and its activities.
Evaluation Briefs
Evaluation Briefs are concise summaries of information on evaluation created by NCMHPYVP staff.
Web sites
American Evaluation Association (AEA) (http://www.eval.org/) is an international professional association of evaluators. Resources available on their Web site include the following:
- Guiding Principles for Evaluators, which describe the professional practice of evaluators and what clients and the public can expect from professional evaluators with whom they work.
- Local affiliate organizations, which may be able to help you locate a qualified evaluator appropriate for your project and budget.
- Links to other resources, including independent consultants and evaluation firms, Web-resources on evaluation, and online evaluation handbooks and textbooks.
Bureau of Justice Assistance (BJA) Evaluation Web Site (http://www.ojp.usdoj.gov/BJA/evaluation/) provides state staff, criminal justice planners, researchers, evaluators, and local practitioners with a variety of resources for evaluating criminal justice programs. The site includes information the following topics:
- evaluation logic models
- performance measures
- program monitoring
- data collection and analysis
- process and impact evaluations
- evaluation planning
- how to choose an evaluation team
Community Tool Box (http://ctb.ku.edu/en) is a Web site created and maintained by the Work Group on Health Promotion and Community Development at the University of Kansas, in collaboration with AHEC/Community Partners in Amherst, Massachusetts. The Tool Box includes practical guidance for the tasks necessary to promote community health and development. Each section, including the sections on evaluation and assessment, includes the following:
- a description of the task
- advantages of performing this task
- step-by-step guidelines
- examples
- checklists of points to review
- training materials
In addition to sections on evaluation and assessment, the Tool Box contains sections on the following topics:
- leadership
- strategic planning
- grant writing
- other tasks and activities
Evaluation Center at Western Michigan University (http://www.wmich.edu/evalctr/index.html) provides a wide range of on-line evaluation resources. These include the following:
- Evaluation Checklists: a collection of refereed checklists for designing, budgeting, contracting, staffing, managing, and assessing evaluations of programs, personnel, students; collecting, analyzing, and reporting evaluation information; and determining merit, worth, and significance.
- Glossary Resources, including glossaries of evaluation terms and links to other evaluation glossaries.
- Evaluation bibliographies
- An international directory of evaluators that can be searched by country, state, region of the United States, name, and specialty.
- Evaluation E-Mail Discussion Lists
- A large number of useful publications, many of which are available for download at no-cost.
Juvenile Justice Evaluation Center Online (http://www.jrsa.org/jjec/index.html) is a project of the Justice Research & Statistics Association, funded by the Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice. This Web site includes the following resources:
- Publications about evaluating juvenile justice projects, including Evaluability Assessment: Examining the Readiness of a Program for Evaluation, Incorporating Evaluation Into the Request for Proposal (RFP) Process, and Strategies for Evaluating Small Juvenile Justice Programs
- Instruments and scales to measure psychological issues, substance abuse, and other issues.
- JJEC Logic Model, for designing evaluations
- Glossary of evaluation terms.
- Links to Web sites relating to evaluation and juvenile justice.
SAMHSA's Prevention Platform (http://prevention.samhsa.gov/about/spf.aspx) is a Web site created by the Substance Abuse and Mental Health Services Administration (SAMHSA) including resources and tools for those working on issues of substance abuse prevention. It includes tools and resources in the areas of assessment, capacity, planning, implementation, and evaluation. Evaluation resources found on this Web site include a comprehensive tool for designing a process or outcome evaluation, a measures and instruments repository, and a number of tools for managing evaluation data.
Web-Based Courses and Workshops
Are You Making Progress? Increasing Accountability Through Evaluation
(http://www.ed.gov/admins/lead/safety/training/evaluation/index.html)
This workshop provides participants with the knowledge and skills necessary to initiate the process of evaluating school prevention efforts. Although designed by the National Training and Technical Assistance Center for Drug Prevention and School Safety Coordinators for those working in middle schools, much of the information is useful for other school-based and community-based programs. In this workshop, participants:
- Review the benefits of conducting a thorough evaluation of prevention activities
- Make informed decisions about the best way to approach an evaluation
- Identify and select the right person to help evaluate a program
- Collaborate with an evaluator to develop a solid and practical evaluation plan
Center for Substance Abuse Prevention's (CSAP) Prevention Pathways: Online Courses (http://pathwayscourses.samhsa.gov/) offer a basic introduction of evaluation to prevention professionals and members of the public. They are designed for people not familiar with the basic concepts of program evaluation. Pathways currently offers three evaluation courses:
- Evaluation for the Unevaluated: Program Evaluation 101
- Evaluation for the Unevaluated: Program Evaluation 102
- Wading through the Data Swamp: Program Evaluation 201
Publications Available Online
Framework for Program Evaluation in Public Health (http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm). Morbidity and Mortality Weekly Reports, 48 (RR11). September 17, 1999.
This CDC framework is a practical, nonprescriptive tool designed to summarize and organize essential elements of program evaluation. The framework comprises steps in program evaluation practice and standards for effective program evaluation. Adhering to the steps and standards of this framework will allow an understanding of each program's context and will improve how program evaluations are conceived and conducted. Furthermore, the framework encourages an approach to evaluation that is integrated with routine program operations. The emphasis is on practical, ongoing evaluation strategies that involve all program stakeholders, not just evaluation experts. Understanding and applying the elements of this framework can be a driving force for planning effective public health strategies, improving existing programs, and demonstrating the results of resource investments.
Getting to Outcomes 2004: Promoting Accountability Through Methods and Tools for Planning, Implementation and Evaluation
(http://www.rand.org/pubs/technical_reports/TR101/) by M Chinman, P Imm, and A Wanderman. Santa Monica: Rand Health, 2004.
This manual presents a ten-step process that enhances practitioners’ prevention skills while empowering them to plan, implement, and evaluate their own programs. It was specifically designed to help agencies, schools, and community coalitions improve programs aimed at preventing or reducing drug and tobacco use among youth. The manual includes text and worksheets and can be applied to any type of prevention programming. It includes chapters on needs and resources assessment; goals and objectives; choosing best practice programs; ensuring program “fit;” capacity,
planning, process, and outcome evaluation; continuous quality improvement, and sustainability.
Guide to Project Evaluation: A Participatory Approach
(http://www.phac-aspc.gc.ca/ncfv-cnivf/familyviolence/html/fvprojevaluation_e.html). Ottawa: Health Canada, 1996.
This guide provides easy-to-use, comprehensive framework for project evaluation. The guide includes information on the defining key evaluation questions, delineating key evaluation steps, writing project goals and objectives, outlining success indicators, collecting and interpreting data, and using evaluation results.
W.K. Kellogg Foundation Evaluation Toolkit (http://www.wkkf.org) contains a number of online publications about evaluation. These include the following:
- W.K. Kellogg Foundation Logic Model Development Guide
- W.K. Kellogg Foundation Evaluation Handbook
- Fundación W.K. Kellogg Manual de Evaluación
- Guiding Program Direction with Logic Models
- Evaluation in Foundations: The Unrealized Potential
Program Manager's Guide to Evaluation (http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eval/index.html). Washington, DC. Administration for Children, Youth, and Families, U.S. Department of Health and Human Services, no date.
A basic guide to evaluation for program managers, which discusses why and how programs should be evaluated, as well as how to report evaluation results.
User Friendly Handbook for Mixed Method Evaluations
(http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf97153). Arlington, VA: National Science Foundation, 1997.
This handbook is based on the recognition that experienced evaluators have found that the best evaluation results are often achieved by using mixed method evaluations, which combine quantitative and qualitative techniques. It contains a discussion of the differences between quantitative and qualitative evaluation methods and how the two can be used together to provide a comprehensive perspective on a program’s success.
Writing@CSU Writing Guides: Empirical Research (http://writing.colostate.edu/
guides/index.cfm?guides_active=empirical&category1=18&category2=19) are a series of “online textbooks” created at the Writing Center at Colorado State University that provide detailed coverage of a number of important issues relevant to evaluation and other types of quantitative and qualitative research. Publications in this series include the following:
- Reliability and Validity
- Generalizability and Transferability
- Introduction to Statistics
- Experimental Methods and Design
- Ethnography, Observational Research, and Narrative Inquiry
- Case Studies
- Survey Research
- Content Analysis
- Rhetoric and the Presentation of Research in English Studies.
Center Resources
Center Resources are places on our Web site that contain information on evaluation including how to use evaluation as part of your sustainability plan, summaries from 13 1999 SS/HS evaluation monographs, examples of grantees that have used evaluation successfully, and where SS/HS grantees can go for evaluation TA.
Legacy Wheel:
Evaluation
http://www.promoteprevent.org/resources/legacy_wheel/evaluations.html
Grantees at Work:
Strategic Use of Evaluation
Fresno Unified School District - California
Safe Schools/Healthy Students Grantee
1999, 2000, and 2001 SS/HS Evaluation Monographs:
http://www.promoteprevent.org/Publications/monograph/index.html
SS/HS Evaluation Team:
http://www.promoteprevent.org/resources/national-evaluation/index.html
Center Conference Materials
Center Conference Materials include PowerPoint presentations and handouts from all evaluation related events including sessions at National Conferences, SS/HS evaluation meetings and workshops, and teleconferences.
