The Comprehensive School Reform Demonstration Act provided $145 million to schools for “research-proven” schoolwide reform. To help schools understand their options, several organizations developed guides to approaching schoolwide reform. These guides, however, did not evaluate the research or the effects of the reforms on student achievement.
An Educators’ Guide to Schoolwide Reform
Recognizing this gap, five associations formed an unusual coalition to sponsor a critical evaluation of the research on the 24 most prominent comprehensive reform approaches, and awarded the project to AIR. The coalition included:
The American Association of School Administrators
The American Federation of Teachers
The National Association of Elementary School Principals
The National Association of Secondary School Principals
The National Education Association.
AIR evaluated the relative strengths of the approaches to reform in three areas:
Evidence of effects on students,
Developer support for implementation, and
Cost.
To determine evidence of effects on students, AIR evaluated the amount of rigorous research and the strength of the research findings. All studies on outcomes included unpublished studies, but not anecdotal data, and were reviewed against a set of standards for evaluating the quality of the study. These standards included the size and type of the sample, use of comparison groups, and use of statistical analysis. AIR reported findings only from studies that met minimum criteria for methodological rigor.
Based on the studies that met these criteria, the approaches to schoolwide reform with the strongest evidence of effect on student performance were Direct Instruction, High Schools that Work, and Success for All. Many of the newer approaches, including New American Schools approaches, showed promise, but had not been in schools long enough to build a substantial research base on student outcomes.
AIR also evaluated developer support for implementation. As reported in the study, all developers provided some support for implementation, and half provided strong support, including at least four technical assistance visits in the first year. AIR’s report also provided information on other aspects of implementation, including the expected steps necessary to develop the programs and the relative difficulty of implementing aspects of each reform.
In evaluating cost, AIR examined two estimates: a high estimate, which assumed that the school paid for all required components, and a lower estimate, which assumed that the school could reallocate staff. Costs to schools implementing these programs ranged from $12,000 to $825,000 in the first year, including time for professional development and salaries for required additional staff.
AIR’s evaluation, together with other descriptive information, provides a thorough overview of the strengths and weaknesses of the 24 approaches to schoolwide reform. The report does not, however, suggest that a single reform approach is best in all circumstances. Schools are advised to consider this information in the context of their own goals and needs before selecting an approach to reform.