- Email: [email protected]
Searching for the best
Evaluation and Program Planning, Vol. 16, pp. 259-261, Printed
in the USA. All rights
0149-7189/93 $6.00 + .OO 1993 Pergamon Press Ltd.
1993
reserved.
SEARCHING
FOR THE BEST
The Chapter 1 Experience in the U.S. Department
of Education
ELOIS M. SCOTT Office
of Policy and Planning,
U.S. Department
of Education,
Washington,
DC
ABSTRACT The Department of Education (ED) has a vested interest in ensuring that schools employ the most promising practices for educating children. In the Chapter I program, the largest federally supported elementary and secondary program, the Department conducts research activities and uses nominations from a variety of sources to identify promising practices and to document their success. The Department disseminates this information through its Technical Assistance Centers and through its awardprocess in the Chapter 1 School Recognition Program.
THE FEDERAL
(U.S. Department of Education, 1992b). The Department of Education wants to maximize the access that children in Chapter 1 programs have to the best instructional practice. Consequently, the Department has a vested interest in finding out what works, why, and how it can be replicated. To inform the Department and policy makers, the Department has been conducting a series of case studies of best/promising practices over the past 3 years. This article briefly describes two of the major methods the Department uses for identifying promising practices and how it disseminates these findings to the field. The identification methods include research efforts to study promising practices and multiple nomination methods.
In the United States the fiscal contribution the federal government provides to support elementary and secondary education is small. Only about 5% of all monies necessary to fund elementary and secondary schools comes from the federal government, whereas the remaining 95% comes from the individual states and localities in which the schools are located. The majority of the 5% provided by the federal government is allocated through its Chapter 1 program to school districts with high concentrations of poverty to supplement their education program for children who have been identified as being at risk of being unsuccessful in school. Since the Chapter 1 program began in 1965, Congress has appropriated over $70 billion; in 1992 alone, the Chapter 1 program received $6.1 billion and served over 5 million children
RESEARCH
ACTIVITIES
TO IDENTIFY
The Department is currently conducting two studies that employ case study methodology to identify and examine the use of promising practices and to address the following issues: l l
how and why schools selected a particular how the programs are implemented
ROLE
PROMISING
PRACTICES
what it takes to sustain them * what is required to replicate them
l
Selection of Promising Practices In both of the studies, the Department solicited recommendations of types of programs that hold promise for improving instruction for educationally disadvantaged
strategy
The author would like to thank Heather McCollum of Policy Studies Associates, Sam Stringfield and Linda Winfield of The Johns Hopkins University, and Mary Ann Millsap of Abt Associates Inc. for their significant contribution to this article. These individuals were the project officers for the case studies that are described in this article. This article was written in the author’s private capacity. The ideas expressed are the author’s, and no official support by the U.S. Department of Education should be assumed or inferred. Requests for reprints should be sent to Elois M. Scott, U.S. Department of Education, Room 3127, FOB 6, 400 Maryland Avenue, S.W., Washington, DC 20202.
259
260
ELOIS
children. These program types may be classified as curriculum-based reforms that introduce new curriculum or revamp existing curriculum and governance-based reforms that restructure school organizations. From these program types, exemplary locations were nominated by program developers, practitioners, and technical assistance center personnel. In one of the two studies, a founding site and a replication site were selected. In the other study, one site was located in a rural area and the other an urban area. The fact that one was a founding site was not a consideration.
What Have We Learned? The purpose of conducting these case studies was to provide information on how the selected promising compensatory education practices were implemented and sustained. One of the things that we have learned is that curriculum-based reforms and governance-based reforms work differently and impact student performance differently. Consequently, one should carefully examine the needs of one’s school and its long-range objectives prior to the selection of a particular strategy. Another finding from these case studies supports earlier research on the principal as the instructional leader. We find in each of the case studies the nearly universal importance of the principal to the success or failure of the special strategy. The three necessary leadership qualities that emerge from the case studies include: (a) the commitment to the success of the particular reform strategy, (b) the ability to motivate staff to support the change, and (c) the management skills necessary to make the required organizational changes. Without a strong leader, programs have little chance of succeeding. In addition to a strong leader, the following findings were also observed: The implementation of all strategies costs money. Sometimes the costs are hidden, as when teachers must spend much of their own time in either learning or implementing the strategy. All reforms require sustained professional development . Opportunities for professional development must occur both prior to the implementation of the program and during the implementation and should be periodic after the program is fully implemented. This type of professional development aids in the institutionali~ation of the program. Curriculum reforms are the easiest to implement. They do not require that the school change, only that the specific teachers involved change. Governance or philosophical approaches, on the other hand, are
CHAPTER
1 SCHOOL
The Chapter 1 School Recognition Program is sponsored by the Department of Education and serves three purposes: 1. to identify compensatory education have been judged to be successful
programs that in meeting the
M. SCOTT
harder to implement, primarily because they involve a change in the entire school culture. The impact on student learning is more predictable and immediate from successful implementation of curriculum-based reforms; in governance-based reform sites, expectations of increases in student achievement are less concrete in the initial years of the new system. The implementation of curriculum frameworks may be useful in keeping governancebased reforms focused. Schools experiencing the most difficulty initiating reform usually have other serious problems, such as severe fiscal problems, racial tensions, and inadequate school and/or district leadership. Involving parents in reform initiatives is often extremely challenging. Differences in race, ethnicity, and class between school staff and parents present frustrating barriers to collaborative work. Strategies involving the whole school day are more likely to result in more coordination and integration with students’ regular classroom work. Similarly, the more the reform requires teachers to engage in problem solving, the more likely they are to use problemsolving strategies in the classroom. For any of the programs to be implemented effectively, both schools and school districts must be committed to providing the following:
Adequate opportunities and time for collaborative planning, including the use of site-based management teams dents,
involving teachers, and parents.
administrative
staff,
stu-
Opportunities for ongoing professional development to ensure that staff acquire the skills to participate in problem solving and that teams are able to implement the programs as they were intended. Authority to make changes. To maintain commitment and avoid frustration, school personnel must be given the authority needed to act on their own decisions. Sufficient resources to support the innovation, including funds, materials, and facilities, and financial incentives to attract and retain qualified staff. In selecting a potentially effective practice, schools must additionally take into consideration contextual variables, including school culture, fiscal base, support for the program, and staff stability. Each of the programs we examined was exemplary in some manner; however, none of the programs should be considered to be a panacea.
RECOGNITION
PROGRAM
learning needs of disadvantaged students in preschool through 12th grade 2. to share this information to assist education agencies in improving the effectiveness of their Chapter 1 programs
261
The Chapter 1 Experience 3. to recognize and honor those education agencies that have achieved special success in compensatory education Nomination
and Selection
Process
Since 1984, the Department has requested nominations of outstanding Chapter 1 schools from state education agencies in the 50 states, the District of Columbia, Puerto Rico, and the Bureau of Indian Affairs. Each Chapter 1 school that is nominated must describe how the goals and objectives of its program reflect what the Department has identified as the 10 attributes of effective compensatory education. These attributes include the following: 1. appropriate instructional materials, methods, and approaches that result in maximum use of academic learning time 2. student progress closely monitored to provide regular feedback and reinforcement 3. recognition and reward for excellence by the district or other educational agency 4. coordination with the regular school program and other special programs 5. parent and community involvement
6. professional development and training 7. clear project goals and objectives that state high expectations for student learning and behavior 8. strong leadership 9. use of evaluation results in improving the program 10. positive school and classroom climate In addition to addressing each of the above attributes, each of the nominated schools must also present evidence of their effectiveness, as determined by: (a) gains in formal measures of student achievement, including standardized test scores; (b) improvement in informal measures of achievement and other project goals, including teacher assessments and student participation counts; and (c) evidence of gains in student achievement that are maintained for more than 1 school year. After projects are nominated, the evaluation components are reviewed by Technical Assistance Center (TAC) personnel. They are then reviewed by a panel appointed by ED that consists of researchers, representatives of educational organizations, and compensatory education teachers and administrators. Those that are found to be particularly successful are included in the Source book. During the past award time, (1989- 1990), 182 projects were selected.
TAC DISSEMINATION ACTIVITIES tended time. Examples of other types of workshops inThere are 6 Technical Assistance Centers (TACs) and 10 clude activities to integrate higher order thinking skills Rural Technical Assistance Centers (RTACs). These into the curriculum, coordination of services, and alterCenters are authorized in the Hawkins-Stafford Elemennative assessments. tary Secondary School Improvement Amendments of In addition, some TACs operate Specialty Options. 1988. Their purpose is to provide assistance to State EdThese Specialty Options provide a service to the other ucation Associations (SEAS), Local Education AssociTACs by synthesizing and developing information based ations (LEAS), teachers, and parents in implementing on research that the TACs can use with the states in their Chapter 1. Particular emphasis is to be paid to program region. There are three Specialty Options: Effective improvement, parental involvement, instruction, testing Practices, Curriculum and Instruction, and Parent Inand evaluation, and curriculum. Each TAG’RTAC devolvement and Family Literacy. velops a plan for technical assistance with SEAS on the The purpose of the Effective Practices Specialty Optype of services to be provided. Frequently these services tion is to identify effective practices and put them into include conducting workshops. The workshops are exa data base that will be on-line to educators throughout amples of translating our current research findings into the nation. The effective practices are selected based on practice. For example, based on studies conducted in five levels of effectiveness: referral by experts; prelimiPlanning and Evaluation, workshops have been pronary support data; substantiation by evaluation data; vided using information from our studies of neglected sustained effectiveness; and other agency recognition. or delinquent children, academic instruction, and exSUMMARY This article has described two procedures the Department of Education uses to identify effective practices and to disseminate what we know. One thing we have learned is that there is more variation within program types than there is among programs. implementation is difficult to sustain, and replication often results in vari-
ation that fits the school and the personnel. Although variation may be worthwhile, it makes rigorous evaluation difficult. We have some good ideas about what works and why, but we still have a way to go before we have rigorous outside evaluations that provide us with good information on what works best.
REFERENCES U.S. DEPARTMENT OF EDUCATION. (1992a). Effective compensatory education sourcebook, Vol. I (Revised): Effective educational practices in schools that serve disadvantaged students, Washington, DC: Author.
U.S. DEPARTMENT
OF EDUCATION.
(1992b).
Nutionai
ment of the Chapter I program - The interim report. Washington, Auihor.
msew DC:
in the USA. All rights
0149-7189/93 $6.00 + .OO 1993 Pergamon Press Ltd.
1993
reserved.
SEARCHING
FOR THE BEST
The Chapter 1 Experience in the U.S. Department
of Education
ELOIS M. SCOTT Office
of Policy and Planning,
U.S. Department
of Education,
Washington,
DC
ABSTRACT The Department of Education (ED) has a vested interest in ensuring that schools employ the most promising practices for educating children. In the Chapter I program, the largest federally supported elementary and secondary program, the Department conducts research activities and uses nominations from a variety of sources to identify promising practices and to document their success. The Department disseminates this information through its Technical Assistance Centers and through its awardprocess in the Chapter 1 School Recognition Program.
THE FEDERAL
(U.S. Department of Education, 1992b). The Department of Education wants to maximize the access that children in Chapter 1 programs have to the best instructional practice. Consequently, the Department has a vested interest in finding out what works, why, and how it can be replicated. To inform the Department and policy makers, the Department has been conducting a series of case studies of best/promising practices over the past 3 years. This article briefly describes two of the major methods the Department uses for identifying promising practices and how it disseminates these findings to the field. The identification methods include research efforts to study promising practices and multiple nomination methods.
In the United States the fiscal contribution the federal government provides to support elementary and secondary education is small. Only about 5% of all monies necessary to fund elementary and secondary schools comes from the federal government, whereas the remaining 95% comes from the individual states and localities in which the schools are located. The majority of the 5% provided by the federal government is allocated through its Chapter 1 program to school districts with high concentrations of poverty to supplement their education program for children who have been identified as being at risk of being unsuccessful in school. Since the Chapter 1 program began in 1965, Congress has appropriated over $70 billion; in 1992 alone, the Chapter 1 program received $6.1 billion and served over 5 million children
RESEARCH
ACTIVITIES
TO IDENTIFY
The Department is currently conducting two studies that employ case study methodology to identify and examine the use of promising practices and to address the following issues: l l
how and why schools selected a particular how the programs are implemented
ROLE
PROMISING
PRACTICES
what it takes to sustain them * what is required to replicate them
l
Selection of Promising Practices In both of the studies, the Department solicited recommendations of types of programs that hold promise for improving instruction for educationally disadvantaged
strategy
The author would like to thank Heather McCollum of Policy Studies Associates, Sam Stringfield and Linda Winfield of The Johns Hopkins University, and Mary Ann Millsap of Abt Associates Inc. for their significant contribution to this article. These individuals were the project officers for the case studies that are described in this article. This article was written in the author’s private capacity. The ideas expressed are the author’s, and no official support by the U.S. Department of Education should be assumed or inferred. Requests for reprints should be sent to Elois M. Scott, U.S. Department of Education, Room 3127, FOB 6, 400 Maryland Avenue, S.W., Washington, DC 20202.
259
260
ELOIS
children. These program types may be classified as curriculum-based reforms that introduce new curriculum or revamp existing curriculum and governance-based reforms that restructure school organizations. From these program types, exemplary locations were nominated by program developers, practitioners, and technical assistance center personnel. In one of the two studies, a founding site and a replication site were selected. In the other study, one site was located in a rural area and the other an urban area. The fact that one was a founding site was not a consideration.
What Have We Learned? The purpose of conducting these case studies was to provide information on how the selected promising compensatory education practices were implemented and sustained. One of the things that we have learned is that curriculum-based reforms and governance-based reforms work differently and impact student performance differently. Consequently, one should carefully examine the needs of one’s school and its long-range objectives prior to the selection of a particular strategy. Another finding from these case studies supports earlier research on the principal as the instructional leader. We find in each of the case studies the nearly universal importance of the principal to the success or failure of the special strategy. The three necessary leadership qualities that emerge from the case studies include: (a) the commitment to the success of the particular reform strategy, (b) the ability to motivate staff to support the change, and (c) the management skills necessary to make the required organizational changes. Without a strong leader, programs have little chance of succeeding. In addition to a strong leader, the following findings were also observed: The implementation of all strategies costs money. Sometimes the costs are hidden, as when teachers must spend much of their own time in either learning or implementing the strategy. All reforms require sustained professional development . Opportunities for professional development must occur both prior to the implementation of the program and during the implementation and should be periodic after the program is fully implemented. This type of professional development aids in the institutionali~ation of the program. Curriculum reforms are the easiest to implement. They do not require that the school change, only that the specific teachers involved change. Governance or philosophical approaches, on the other hand, are
CHAPTER
1 SCHOOL
The Chapter 1 School Recognition Program is sponsored by the Department of Education and serves three purposes: 1. to identify compensatory education have been judged to be successful
programs that in meeting the
M. SCOTT
harder to implement, primarily because they involve a change in the entire school culture. The impact on student learning is more predictable and immediate from successful implementation of curriculum-based reforms; in governance-based reform sites, expectations of increases in student achievement are less concrete in the initial years of the new system. The implementation of curriculum frameworks may be useful in keeping governancebased reforms focused. Schools experiencing the most difficulty initiating reform usually have other serious problems, such as severe fiscal problems, racial tensions, and inadequate school and/or district leadership. Involving parents in reform initiatives is often extremely challenging. Differences in race, ethnicity, and class between school staff and parents present frustrating barriers to collaborative work. Strategies involving the whole school day are more likely to result in more coordination and integration with students’ regular classroom work. Similarly, the more the reform requires teachers to engage in problem solving, the more likely they are to use problemsolving strategies in the classroom. For any of the programs to be implemented effectively, both schools and school districts must be committed to providing the following:
Adequate opportunities and time for collaborative planning, including the use of site-based management teams dents,
involving teachers, and parents.
administrative
staff,
stu-
Opportunities for ongoing professional development to ensure that staff acquire the skills to participate in problem solving and that teams are able to implement the programs as they were intended. Authority to make changes. To maintain commitment and avoid frustration, school personnel must be given the authority needed to act on their own decisions. Sufficient resources to support the innovation, including funds, materials, and facilities, and financial incentives to attract and retain qualified staff. In selecting a potentially effective practice, schools must additionally take into consideration contextual variables, including school culture, fiscal base, support for the program, and staff stability. Each of the programs we examined was exemplary in some manner; however, none of the programs should be considered to be a panacea.
RECOGNITION
PROGRAM
learning needs of disadvantaged students in preschool through 12th grade 2. to share this information to assist education agencies in improving the effectiveness of their Chapter 1 programs
261
The Chapter 1 Experience 3. to recognize and honor those education agencies that have achieved special success in compensatory education Nomination
and Selection
Process
Since 1984, the Department has requested nominations of outstanding Chapter 1 schools from state education agencies in the 50 states, the District of Columbia, Puerto Rico, and the Bureau of Indian Affairs. Each Chapter 1 school that is nominated must describe how the goals and objectives of its program reflect what the Department has identified as the 10 attributes of effective compensatory education. These attributes include the following: 1. appropriate instructional materials, methods, and approaches that result in maximum use of academic learning time 2. student progress closely monitored to provide regular feedback and reinforcement 3. recognition and reward for excellence by the district or other educational agency 4. coordination with the regular school program and other special programs 5. parent and community involvement
6. professional development and training 7. clear project goals and objectives that state high expectations for student learning and behavior 8. strong leadership 9. use of evaluation results in improving the program 10. positive school and classroom climate In addition to addressing each of the above attributes, each of the nominated schools must also present evidence of their effectiveness, as determined by: (a) gains in formal measures of student achievement, including standardized test scores; (b) improvement in informal measures of achievement and other project goals, including teacher assessments and student participation counts; and (c) evidence of gains in student achievement that are maintained for more than 1 school year. After projects are nominated, the evaluation components are reviewed by Technical Assistance Center (TAC) personnel. They are then reviewed by a panel appointed by ED that consists of researchers, representatives of educational organizations, and compensatory education teachers and administrators. Those that are found to be particularly successful are included in the Source book. During the past award time, (1989- 1990), 182 projects were selected.
TAC DISSEMINATION ACTIVITIES tended time. Examples of other types of workshops inThere are 6 Technical Assistance Centers (TACs) and 10 clude activities to integrate higher order thinking skills Rural Technical Assistance Centers (RTACs). These into the curriculum, coordination of services, and alterCenters are authorized in the Hawkins-Stafford Elemennative assessments. tary Secondary School Improvement Amendments of In addition, some TACs operate Specialty Options. 1988. Their purpose is to provide assistance to State EdThese Specialty Options provide a service to the other ucation Associations (SEAS), Local Education AssociTACs by synthesizing and developing information based ations (LEAS), teachers, and parents in implementing on research that the TACs can use with the states in their Chapter 1. Particular emphasis is to be paid to program region. There are three Specialty Options: Effective improvement, parental involvement, instruction, testing Practices, Curriculum and Instruction, and Parent Inand evaluation, and curriculum. Each TAG’RTAC devolvement and Family Literacy. velops a plan for technical assistance with SEAS on the The purpose of the Effective Practices Specialty Optype of services to be provided. Frequently these services tion is to identify effective practices and put them into include conducting workshops. The workshops are exa data base that will be on-line to educators throughout amples of translating our current research findings into the nation. The effective practices are selected based on practice. For example, based on studies conducted in five levels of effectiveness: referral by experts; prelimiPlanning and Evaluation, workshops have been pronary support data; substantiation by evaluation data; vided using information from our studies of neglected sustained effectiveness; and other agency recognition. or delinquent children, academic instruction, and exSUMMARY This article has described two procedures the Department of Education uses to identify effective practices and to disseminate what we know. One thing we have learned is that there is more variation within program types than there is among programs. implementation is difficult to sustain, and replication often results in vari-
ation that fits the school and the personnel. Although variation may be worthwhile, it makes rigorous evaluation difficult. We have some good ideas about what works and why, but we still have a way to go before we have rigorous outside evaluations that provide us with good information on what works best.
REFERENCES U.S. DEPARTMENT OF EDUCATION. (1992a). Effective compensatory education sourcebook, Vol. I (Revised): Effective educational practices in schools that serve disadvantaged students, Washington, DC: Author.
U.S. DEPARTMENT
OF EDUCATION.
(1992b).
Nutionai
ment of the Chapter I program - The interim report. Washington, Auihor.
msew DC: