The SS/HS Evaluation Toolkit is designed to provide you with tools, information, and resources to help you design and implement your evaluation, fulfill your obligations to the cross-site National Evaluation, and meet other Federal reporting requirements (especially those of the Government Performance Results Act – commonly called GPRA). The contents of the toolkit are presented in approximately the same sequence as the information in the Evaluation section of the SS/HS Project Resource Guide (discussed below). However, the “heart” of the Toolkit are those tools which, together, will help a project design and/or refine its Logic Model and develop an Evaluation Plan consistent with that Logic Model.
Below is a detailed summary of the sections contained in the Evaluation Toolkit. The sections will guide you through the evaluation process, from hiring an evaluator and designing an evaluation, to communicating evaluation results to help sustain your program.
I. Introduction to SS/HS Evaluation
Every SS/HS grantee is required to conduct a local evaluation. The purpose of the local evaluation, in the words of the program announcement, is to “provide timely information for creating strategic plans, measuring progress, and keeping the project focused on the overall objective of the SS/HS initiative—promoting healthy childhood development and preventing violence and substance abuse.”
Local evaluation is the key to program success because it provides useful information for managing both the day-to-day and long-term aspects of your project by measuring your site’s progress toward set goals and objectives.
Local evaluation is valuable because it:
- demonstrates the importance of your initiative to partners and stakeholders
- assists staff who are implementing an activity in making changes that may impact the activity’s effectiveness
- offers information that will help you decide where resources can be used most efficiently
For a more in depth overview of evaluation, please see Introduction to SS/HS Evaluation.
II. Preparing for Your Evaluation
Most SS/HS grantees contract with a professional evaluator to plan and carry out the evaluation. Evaluators will often assist with writing the original application and developing the SS/HS evaluation plan that is required at six-months post-award.
Some sites may not have an evaluator at the beginning of their grant for a variety of reasons. If you still need to hire an evaluator, Hiring an Evaluator will provide some guidance on this process.
Once an evaluator is hired, the project director is always responsible for managing the evaluator. If a participatory evaluation is used, the project director must take on additional responsibilities, including the following:
- Working with the evaluator to ensure that the evaluation design is appropriate for the site’s needs and resources, follows the initiative’s timeline, and fulfills all federal requirements
- Identifying members for an Evaluation Team
- Using information from the process evaluation to monitor the initiative and make modifications when needed
- Communicating with partners and stakeholders about the evaluation and seeking their input and support
- Using evaluation data to fulfill Federal reporting requirements
Guidance on this relationship can be found in Managing An Evaluator.
Evaluation Design
It is very important to work with your evaluator to decide on an appropriate evaluation design for your project. Simple designs are easier to implement, but provide less information, while complex evaluation designs may lead to more accurate results, but can be more difficult to carry out. You must also decide whether to choose a quantitative, qualitative, or mixed approach. Quantitative data can be measured and reported in numerical form, while qualitative data are in narrative format, and are useful for creating documents such as case studies.
A discussion about how evaluation design affects an outcome evaluation is provided inEvaluation Designs and Approaches.
Performance Indicators
SS/HS projects are required to identify and provide annual data for at least one performance indicator for each of the elements of the SS/HS Initiative, (at least one performance indicator for the SS/HS Partnership for ’05 and ’06 cohorts), and the SS/HS GPRA measures.
For more information on performance indicators, please see:
- Sample SS/HS Performance Indicators (05-06 cohorts)
- Sample SS/HS Performance Indicators (07 cohort)
- Performance Indicator Reporting Materials: Ed 524-B Forms (Parts 1-3) and general instructions
- SS/HS Evaluation Timeline
Measures and Instruments
As part of the evaluation of your SS/HS initiative, you and your evaluator will be collecting data on the process and outcome measures you have identified in your logic model. For more information on this topic, please see Measures and Instruments and the Measures and Instruments Resource Page.
Government Performance and Results Act (GPRA)
SS/HS grantees are required to collect and report information every six months on Government Performance and Results Act (GPRA) indicators.
Your performance indicators may or may not also satisfy GPRA requirements. For sample measures that can be used to satisfy the SS/HS GPRA indicators and other information on GPRA, see:
- Government Performance and Results Act Summary
- Government Performance and Results Act
- GPRA Measures Chart (05-06 cohorts)
- GPRA Measures Chart (07 cohort)
- Recommended Items for GPRA Measures (05-06 cohorts)
- Recommended Items for GPRA Measures (07 cohort)
Protection of Pupil Rights Amendment and Family Educational Rights and Privacy Act
All SS/HS grantees must adhere to the provisions of the Protection of Pupil Rights Amendment and the Family Educational Rights and Privacy Act. The Protection of Pupil Rights Amendment (PPRA) and Family Educational Rights and Privacy Act (FERPA) overview and FERPA and PPRA Resource Page provide provide futher guidance on these regulations.
Protection of Human Subjects
Grantees conducting research activities involving human subjects that are not exempt must take the following steps to ensure all research is conducted legally and safely:
- Identify a registered Institutional Review Board (IRB) to review your project.
- Make sure that the IRB you have selected has a Federal Assurance.
- Have the IRB review and approve your research activities.
- Obtain informed consent and assent and/or parental permission from the subjects involved in your project unless this requirement is waived or altered by the IRB.
To learn more about what kinds of human subjects research are exempt and regulations related to Institutional Review Boards, please see:
- Protection of Human Subjects
- Sample SS/HS IRB Submission (Research Protocol: Safe and Caring Connections Project, XX School District)
- Sample Parent Consent Letter
- Protection of Human Subjects Resource Page (DOE)
The Federal Government has created a cross-site national evaluation designed to collect further information on the GPRA indicators for the SS/HS initiative, and to examine additional issues relevant to the initiative. These findings, collected by the National Evaluation Team (NET), will be used to demonstrate the success of the SS/HS initiative and provide support for the continuation of funding.
For more information on the national evaluation, please see the following resources:
- National Evaluation Web site
- National Evaluation Team Contact Information
- SS/HS Evaluation Timeline (including National Evaluation activities)
- Project Level Survey
- School Level Survey
- School Climate Survey for Staff
- Optional School Climate Survey for Students
- Optional School Climate Survey for Parents
- NET Data Collection Summary
III. The Project Logic Model and Evaluation Plan
Developing and/or revisiting your logic model is an important step in conceptualizing your evaluation. It should be done prior to drafting the Evaluation Plan (described below). SS/HS projects are required to submit a logic model with their first six month performance report.
Your logic model serves as a blueprint for the evaluation by demonstrating the perceived linkages between local needs, goals, objectives, activities, and process and outcome measures. Your logic model can assist in identifying evaluation procedures that can help you determine whether your activities are producing successful results as described in your logic model. Creating a strong logic model has been a hallmark of successful SS/HS sites.
For a Logic Model Worksheet and other resources on logic models, see the following documents:
’05 and ’06 Cohorts
’07 Cohort
An evaluation plan provides an overview of all evaluation activities and their relationships to one another and to other activities. A good evaluation plan can help form the basis of a work plan for your evaluator and can assist you in managing the evaluation process. Your evaluation plan should be based on the evaluation section of your original application, your SS/HS logic model, and the federal reporting requirements previously described. The evaluation plan describes the following:
- The evaluation design: What evaluation method will be used; what data will be collected; how these data will be collected; and how these data will be analyzed.
- How the evaluation will be implemented: Who will collect data; when; what data will be reported to you as project director and other stakeholders, at what interval, and in what form.
For resources on evaluation plans, see the following documents:
05-06 Cohorts
- Evaluation Plan Worksheet Instructions (05-06 cohorts)
- Evaluation Plan Sample (05-06 cohorts)
- Evaluation Plan Worksheet (05-06 cohorts)
07 Cohort
- Evaluation Plan Worksheet Instructions (07 cohort)
- Evaluation Plan Sample (07 cohort)
- Evaluation Plan Worksheet (07 cohort)
IV. Other Resources
In most cases, the evaluator is primarily responsible for supporting the implementation of the evaluation in the field. However, as a project director, you should stay fully informed about the progress of the evaluation, since you are responsible for ensuring that all aspects of the initiative work together smoothly.
Evaluation is an essential tool for project management. It is a powerful means to demonstrate the potential value of an initiative and gain the support needed to sustain effective activities and desired outcomes. Partners and stakeholders should understand how an initiative is impacting the community, and be able to articulate this to people who may be involved in sustaining the initiative’s efforts.
After evaluation results are gathered, it is important to present them so that they are understandable and can be easily communicated to others. Positive evaluation data can build institutional, political, and community support for sustaining your initiative’s components. People are more likely to support the dedication of resources to continue a program if they can see that it is making a real difference in their community.
For more resources on evaluation, please see the Evaluation Resource Page.
SS/HS Evaluation Team
Technical assistance resources available to SS/HS sites include their Technical Assistance Specialist as well as members of the Evaluation Team. For more information on how to receive comprehensive and targeted technical assistance regarding any evaluation issue, click here.
