National Evaluation

Communities In Schools has commissioned a multi-year external evaluation. This evaluation, conducted by ICF international, will ultimately reveal the most successful strategies for preventing students from dropping out of school and identify replicable evidence-based practices that can be adopted throughout the United States.

OVERVIEW

During the first year of the evaluation, detailed information was collected on the work Communities In Schools is doing in schools, providing a comprehensive picture of how the model is implemented in thousands of schools across the country. Key baseline information confirms that, while innovation and adaption result in vastly different programming and emphasis at each Communities In Schools site, the Communities In Schools model is the unifying framework.

The second and third years of the National Evaluationt focused on school-level studies, determining the difference Communities In Schools makes at the school level.

It also began the student-level studies that will continue in years four and five. These longitudinal student studies will compare Communities In Schools and non-Communities In Schools student behavioral and academic outcomes over a three-year period and will provide a "gold standard" of scientifically based research to demonstrate the effectiveness of the Communities In Schools model.

At the conclusion of the five-year evaluation, all of the findings will be compiled so that the overall impact of the Communities In Schools model of integrated student services can be analyzed and replicated.


SCHOOL-LEVEL STUDIES

The Communities In Schools national office has completed the school-level studies (represented in the mid-level of the evaluation pyramid), which are a major component of the National Evaluation designed to provide hard evidence that integrated student services provided by Communities In Schools result in positive school outcomes.

In January 2008, ICF International concluded a school-level analysis that features a comparison study of Communities In Schools and non-Communities In Schools schools on relevant outcomes such as dropout and graduation rates, and reading and math proficiency. Case studies will complete the school-level analysis.

The school-level results provide the first external, empirical affirmation of the relationship between integrated student services and school-wide outcomes. Further, results show that it is specifically CBISS – Communities In Schools’ unique approach to implementing coordinated services – that yields the strongest outcomes.

ICF surveyed 1,700 Communities In Schools schools to determine the level of Communities In Schools implementation of the Communities In Schools model taking place at each school. The schools were then given a score from 1-100, based on their degree of fidelity to components of the Communities In Schools model. When scores were correlated with school-level outcomes, the cohort of sites scoring 70 or higher showed the most positive outcomes. This established the relationship between outcomes and the level of implementation of the Communities In Schools model. The group (referred to as “high implementers”) represents 47 percent (710) of the sample sites. The remaining sites (808) are implementing the Communities In Schools model to a lesser degree and are referred to as “partial implementers.”

To determine whether the positive student outcomes could be attributed solely to implementation of the Communities In Schools model, 610 Communities In Schools schools were matched against 610 comparison schools. The results that follow reflect the outcomes achieved by the subset of high-implementing sites, and their comparison schools. This high-implementer group closely approximates the baseline of excellence that will be achieved when all affiliates in the Communities In Schools network are accredited through the Total Quality System process. Their results accurately reflect the outcomes anticipated by high fidelity to the Communities In Schools model.

Initial Findings

The initial findings build on a growing body of research that establishes the indispensable role that community-based integrated student services (CBISS) can play in helping schools, particularly (though not exclusively) those that are low-performing, with a high population of students at risk of dropping out.

The Communities In Schools study establishes three important facts:

  1. Compared to dropout prevention programs with scientifically-based evidence and listed in the Department of Education's What Works Clearinghouse, Communities In Schools is one of a small number to prove it keeps students in school and the only one in the country to prove that it increases graduation rates, graduating students on time with a regular diploma.
  2. When implemented with high fidelity, the Communities In Schools model results in a higher percentage of students reaching proficiency in fourth- and eighth-grade reading and math, than comparison schools during the same time period.
  3. Effective implementation of the Communities In Schools model of integrated student services correlates more strongly with positive school-level outcomes than does the uncoordinated provision of services alone.

Communities In Schools Model

The success of the Communities In Schools approach lies in the implementation of a preCommunities In Schoolse set of strategies designed to provide the most appropriate and effective prevention and intervention services to schools and communities. The model assumes integration of community and school resources in a coordinated manner.

The Communities In Schools model includes:

Evaluation Contractor
The evaluation contractor for this project is ICF International (formerly Caliber Associates). ICF International brings nearly 40 years of experience in evaluating social, environmental, security, defense, energy and transportation programs, using both qualitative and quantitative approaches. ICF is an established and respected partner of the U.S. Department of Education, and has a portfolio of clients that include state and federal government agencies, and domestic and international for-profit and nonprofit organizations. ICF is known for its high standards of rigor and comprehensive research designs.

 

STUDENT-LEVEL STUDIES
In years four and five of the evaluation, Communities In Schools will continue the work begun in year three with the most rigorous of all evaluation components--the experimental studies.

Experimental studies are widely considered the "gold standard" in research. These studies, often called randomized controlled trials by researchers, involve the random assignment of students to a treatment group or a control group. Through random assignment, researchers are able to make the two groups as similar in composition as possible. By minimizing differences between the groups, any difference in outcomes can be attributed to participation in the Communities In Schools program. The effectiveness of the Communities In Schools program can best be determined through this type of study.

Currently, randomized controlled trials have begun in Communities In Schools of Central Texas in Austin, Texas and in Communities In Schools of Jacksonville, Florida. Initial results from these studies are expected in 2009, with final results expected at the completion of the three-year period in 2010. 


EVALUATION PYRAMID
The framework for the evaluation can be best depicted as a three-level pyramid. The foundation of this pyramid is a comprehensive assessment of Communities In Schools services and operations at the more than 3,000 Communities In Schools school sites.

The mid-level of the pyramid includes the school-level studies which provide evidence regarding school-level outcomes and will determine where Communities In Schools is having the most substantial impact, what is being done at the successful sites, and how positive impact is achieved.

The top-level of the pyramid includes the student-level studies which provide evidence regarding student-level outcomes. It is the most rigorous component of the evaluation and will firmly establish the causality - the "value added" - of Communities In Schools strategies and processes with respect to specific student outcomes, such as academic progress and graduation rates.

Evaluation Pyramid

RESOURCE MATERIALS

The Communities In Schools national office crafted resource materials to address and promote the initial results of the national evaluation. Various documents are provided below for this purpose.

 

Print this page Print