., _ ,“W. 1 sp. United States General Accounting Office Briefing Report to the Chairman, - Committee on Education and Labor, House of Representatives 1989 EFFECTIVE SCHOOLSPROGRAMS Their Extent and Characteristics United States General Accounting Office Washington, D.C. 20548 Human Resources Division B-235384 September 13, 1989 The Honorable Augustus F. Hawkins Chairman, Committee on Education and Labor House of Representatives Dear Mr. Chairman: You requested that we conduct a national study of effective schools pro- grams to provide information on the extent and characteristics of these programs in the nation’s school districts and schools. In subsequent dis- cussions with your office, we agreed to (1) determine the number of school districts with effective schools programs, (2) identify common program characteristics and practices, (3) describe how school districts evaluate the effect of their programs on students’ academic achieve- ment, and (4) discuss federal requirements for evaluating these pro- grams. You were particularly interested in the ability of school districts to provide insight on whether effective schools programs improved the academic achievement of low-income and minority children. The Hawkins-Stafford Elementary and Secondary School Improvement Amendments of 1988 (P.L. 100-297) authorized Chapter 1 compensatory education funds and Chapter 2 educational improvement block grant funds to be used for funding effective schools programs. As you know, an effective schools program is a school-based program developed to improve the academic achievement of all children in a school, regardless of socioeconomic background or ethnicity, and in particular education- ally deprived children.’ Effective schools programs, like other school improvement efforts, have become increasingly common as the educa- tional reform movement has burgeoned during the 1980s. Effective schools programs seek to develop or improve on school charac- teristics that have been identified in research as associated with high student achievement. Public Law loo-297 specifies five such character- istics that these programs should seek to achieve. They are (1) strong and effective administrative and instructional leadership, (2) emphasis on the acquisition of basic and higher-order skills, (3) a safe and orderly school environment, (4) a climate of expectations that virtually all chil- dren can learn under appropriate conditions, and (5) continuous assess- ment of students and programs to evaluate the effects of instruction. ‘The Department of Education defines educatlonally deprived children as children whose educational attainment is below the level that is appropnate for their age Page 1 GAO/HRD-89132BR Effective Schools Programs B-236384 To collect the information in this report we mailed copies of a question- naire to a random sample of 1,685 school district superintendents. Our findings are representative of the approximately 16,000 local school dis- tricts in the nation. The questionnaire we used asked superintendents about effective schools programs operating in their schools during school year 1987-88. This report elaborates on the briefing we provided congressional staff on June 20, 1989. To measure the overall number of districts and schools with effective Many School Districts schools programs, we asked district officials whether any of their Operate Effective schools operated school improvement programs based on the findings of Schools Programs effective schools research. District responses to our questionnaire indi- cate that about 41 percent or 6,500 of the nation’s school districts had effective schools programs in operation in approximately 38,000 ele- mentary and secondary schools during school year 1987-88. Many dis- tricts reported that their programs had been established recently; over half had effective schools programs that were first implemented during school years 1986-87 or 1987-88. An additional 17 percent or about 2,600 of the nation’s districts have plans to implement effective schools programs during school years 1988-89 or 1989-90. To provide the Congress with information on the extent to which effec- tive schools programs include specific activities recommended by pro- gram experts, we measured the number of districts and schools with programs that met certain program criteria. About 27 percent or 4,300 of the nation’s districts had effective schools programs in approximately 27,000 schools; these schools used (1) school teams on which teachers and administrators work together to plan and monitor their programs and (2) written plans for improving school effectiveness. Program experts recommend school teams and written plans to help foster school-based planning and decisionmaking. When adding the criteria that districts also evaluate their effective schools programs by disaggregating (that is, analyzing separately) aca- demic achievement data by student socioeconomic status or ethnicity, about 13 percent or 2,100 of the nation’s school districts had such pro- grams in approximately 18.000 elementary and secondary schools. Pro- gram experts consider this evaluation method particularly important because it permits schools and districts to specifically identify academic achievTement gains made by low-income and minority children. Without disaggregating achievement data in this manner, a district could incor- rectly interpret districtwide achievement gains as improvement among Page 2 GAO,%RD89-132BR Effective Schools Programs -. B235384 all children, when, in fact, those gains were made by only some children, such as those from higher-income families. Which Schools Have Effective schools programs operate in elementary and secondary schools Effective Schoc11s at an average rate approximately commensurate with districts’ average total percentage of elementary and secondary schools. For example: An Programs? average of about 53 percent of effective schools programs operate in elementary schools; similarly, elementary schools make up, on average, 52 percent of the total number of schools in the districts with programs. An average of about 20 percent of effective schools programs operate in high schools; similarly, high schools make up, on average, 21 percent of the total number of schools in the districts with effective schools pro- grams. In addition, these programs commonly serve mixed enrollments of low-income and higher-income children. Which Districts H:ave Effective schools programs are somewhat more likely to operate in large Effective Schools school districts, but are about as common in urban and nonurban areas. For example, although districts with enrollments of 10,000 students and Programs? over make up 4 percent of all c’.S. districts, they account for about 8 percent of the districts with effective schools programs. These programs operate in urban school districts, including central city districts, at a rate nearly commensurate with the percentage of such districts in the nation. Many Districts Require Our questionnaire results indicate that in half of all school districts with Schools to Implement effective schools programs, some or all schools with the programs were required by their districts to implement them. In the other half of the Effective Schools districts with programs, schools voluntarily implemented them. Programs Programs Frequently School districts frequently receive external assistance in developing and Developed and implementing effective schools programs from state education agencies (SEAS), universities, federally funded regional educational laboratories, Implemented With and other agencies or individuals outside school districts. Our question- External Assistance naire results indicate that about 64 percent of the districts with effec- tive schools programs received such external assistance, most often from SEAS. We identified 31 SEAS that assisted districts and schools in developing or implementing effective schools programs. Page 3 GAO/HRD-89.132BR Effective Schools Program - B236334 School districts reported using a variety of student performance meas- Program Evaluation- ures, ranging from achievement tests to student dropout rates, to evalu- Districts Seldom ate their programs’ effect on academic achievement. When conducting Determine If Programs these evaluations, however, school districts seldom determine whether the academic achievement of low-income or minority children is Help All Children improved; this is because districts seldom disaggregate academic achievement data by student socioeconomic status or ethnicity. Approximately 83 percent of all school districts with effective schools programs evaluate their programs using achievement tests, which typi- cally measure basic language arts and mathematics skill levels or assess mastery of specific curriculum objectives. About 75 percent of these dis- tricts use nontest measures, which commonly include grades, dropout rates, attendance, and enrollment rates in advanced and remedial classes. Sixty-six percent of the districts with programs use both test and nontest measures. Only about 12 percent of the districts with effective schools programs regularly disaggregate achievement test results by student socioeco- nomic status. An even smaller percentage (about 9 percent) do so by ethnicity. Approximately 22 percent of the districts disaggregate nontest measures by student socioeconomic status, and about 29 percent do so by ethnicity. As requested by your office, we did not obtain official agency comments on this report. We did, however, discuss its contents with Department of Education officials and have incorporated their comments where appro- priate. As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report for 7 days from its issue date. At that time, copies will be sent to the Secretary of Education and other interested parties. The major contributors to this report are listed in appendix VIII. Sincerely yours, William J. Gainer Director, Education and Employment Issues Page 4 GAO MRD-8%132BR Effective Schools Program@ Page 5 GAO NRD-89-132BR Effective Schools Programs Contents Letter 1 Effective Schools 10 Programs: Their Background 10 Objectives, Scope, and Methodology 19 Extent and Many School Districts Operate Effective Schools 21 Characteristics Programs Which Schools and Districts Have Effective Schools 25 Programs? School Characteristics Emphasized by Programs 30 Many Districts Require Schools to Implement Effective 32 Schools Programs Districts Seldom Determine If Programs Help All Children 38 Federal Evaluation Requirements for Effective Schools 43 Programs Appendix I 46 Selected Characteristics of Effective Schools Cited in Research Literature Appendix II 47 Five School Characteristics in the Definition of Effective Schools Programs Included in Public Law 100-297 Appendix III 48 Chapter 2 Block Grant Program: Six Authorized Local Page 6 GAO/HRD-SS-132BR Effective Schools Programs - Contents Targeted Assistance Programs Appendix IV Estimates and Confidence Intervals for Selected Questionnaire Responses Appendix V Effective Schools Programs in the Fifty Largest Cities (School Year 1987-88) Appendix VI Examples of SEA Assistance to School Districts and Schools for Effective Schools Programs Appendix VII 56 Tables Supporting Figures in Report Text and Supplementary Tables Appendix VIII 60 Major Contributors to This Report Page 7 GAO/BRD89-132BR Effective Schools Programa - Contents Tables Table VII.l: Extent of ESPs Meeting Criterion 1 in U.S. 56 Districts (Data for Fig.9) Table VII.2: School Years ESPs First Implemented (Data 56 for Fig. 10) Table VII.3: Number of School Districts and Schools With 56 ESPs by Program Criteria (Data for Fig. 11) Table VII.4: Use of School Teams and Written Plans 56 Table VII.5: Grade Level of Schools With ESPs (Data for 56 Fig. 13) Table VII.6: Percent of Children on Free or Reduced-Price 57 Lunch Served by ESPs (Data for Fig.14) Table VII.7: Student Enrollment in Districts With ESPs 57 and All U.S. Districts (Data for Fig. 15) Table VII.8: Geographic Area of School Districts With 57 ESPs and All U.S. Districts (Data for Fig.16) Table VII.9: School Characteristics Cited by Districts by 58 Program Criteria (Data for Fig. 17) Table VII.10: School District Involvement in ESPs (Data 58 for Fig. 18) Table VII. 11: Source of External Assistance to Districts 58 With ESPs (Data for Fig.20) Table VII.12: Measures of Achievement Districts Used to 58 Evaluate Results of ESPs (Data for Fig.24 and Related Information) Table VII.13: Student Subgroups by Which Districts 59 Disaggregated Achievement Test Results (Data for Fig.25) Table VII. 14: Student Subgroups by Which Districts 59 Disaggregated Nontest Measures (Data for Fig.26) Table VII. 15: Percent of Children on Free or Reduced- 59 Price Lunch Served by ESPs in Districts That Do Not Disaggregate Achievement Data by Student Socioeconomic Status (Data for Fig. 27) Figures Figure 1: What Are ESPs? 10 Figure 2: How Schools Typically Implement ESPs 11 Figure 3: Characteristics Frequently Cited in Effective 13 Schools Figure 4: Federal Funds Authorized for ESPs 15 Figure 5: State Distribution of Chapter 2 Funds 17 Figure 6: Study Objectives 18 Page 8 GAO/HRD-89.132BR Effective Schools Programs -~ .-____ Contents Figure 7: Methodology 19 Figure 8: Criteria Used to Measure Extent of ESPs 21 Figure 9: Extent of ESPs in U.S. Districts Meeting 22 Criterion 1 Figure 10: Many ESPs Recently Implemented 23 Figure 11: Using Different Criteria, Extent of ESPs in U.S. 24 Districts Figure 12: Which Schools and Districts Have ESPs? 25 (Criterion 1) Figure 13: ESPs About as Likely at Each Grade Level 26 Figure 14: Socioeconomic Status of Children 27 Figure 15: Large Districts More Likely to Have ESPs Than 28 Small Districts Figure 16: ESPs About as Likely in Urban and Nonurban 29 School Districts Figure 17: School Characteristics Emphasized by ESPs 30 Figure 18: School District Involvement in ESPs 32 Figure 19: Examples of State Laws That Pertain to ESPs 33 Figure 20: Sources of External Assistance for ESPs 35 Figure 21: Examples of SEA Support for ESPs 36 Figure 22: Train-The-Trainers Staff Development 37 Training Figure 23: How Do Districts Evaluate ESPs? 38 Figure 24: Program Evaluation 39 Figure 25: District Evaluations Using Achievement Tests 40 Figure 26: District Evaluations Using Eontest Measures 41 Figure 27: In Districts Not Disaggregating by 42 Socioeconomic Status Figure 28: Chapter 1 and 2 Evaluation Requirements in 43 Public Law loo-297 Abbreviations CCD common core of data ESPS effective schools programs GAO General Accounting Office MSA metropolitan statistical area SEA st,ate education agency Page 9 GAO/HRD-89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Chamcteristics The Hawkins-Stafford Elementary and Secondary School Improvement Background Amendments of 1988 (P.L. 100-297) authorized Chapter 1 compensatory education funds and Chapter 2 educational improvement block grant funds to be expended for effective schools programs (ESPS). An effective schools program is a school-based program to improve the academic achievement of all children in a school, regardless of socioeconomic background or ethnicity, and in particular educationally deprived chil- dren (see fig. 1). These programs are also designed to promote school- level planning and decisionmaking and to become formal processes by which schools assess their needs and plan for their own improvement. Effective schools programs, like other school improvement efforts such as parental choice in selecting schools for their children to attend and increased graduation requirements, have become increasingly common as the educational reform movement has burgeoned during the 1980s. Figure 1 GAQ What are ESPs? Effective Schools Programs are school-based programs to improve the academic achievement of all children, regardless of socioeconomic status or ethnicity, and particularly educational deprived children. Page 10 GAO/HR.D-89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Fioure 2 GAO How Schools Typically Implement ESPs l Acquaint school staffs with research findings l Establish teams of teachers and administrators l Assess schoolwide and classroom needs l Formulate improvement plans Effective schools programs have been developed by individual schools! school districts, state education agencies (SEAS),federally funded regional educational laboratories, and universities. According to the research literature and experts we interviewed, effective schools pro- grams differ in (1) the amount of technical and financial assistance received from sources outside the school, (2) the amount of training given staff, (3) the emphasis on schoolwide versus classroom character- istics, and (4) the variety and use of school data that are obtained for evaluation purposes. Despite these differences, the literature shows that schools typically implement effective schools programs in a similar man- ner (see fig. 2). Page 11 GAO/HRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Effective Schools Research Effective schools programs seek to develop or improve on school charac- teristics identified by effective schools research as associated with high student achievement (see app. 1 for a list of 22 such characteristics). This research includes studies on school effectiveness, teacher effective- ness, organizational management, and program implementation, accord- ing to the literature we reviewed. Most typically, effective schools programs emphasize the findings of studies on school effectiveness.’ In these studies, researchers have iden- tified characteristics present in schools with unexpectedly high stan- dardized student achievement test scores given their students’ socioeconomic background. Most of these studies focused on urban ele- mentary schools serving low-income and minority students. These stud- ies challenged the assumption that schools could have little effect on improving student achievement-a popular interpretation of earlier studies on equality in education, which found a direct relationship between socioeconomic background and student achievement.? The findings of school effectiveness research were rapidly adopted by schools, districts, and states as models for school improvement pro- grams during the 1980s. The growth of effective schools programs was encouraged, in part, by (1) the belief among educators that schools could make a difference in improving student achievement and (2) school effectiveness researchers who advocated programs based on their research. The most influential researcher and advocate of these pro- grams was the late Ronald Edmonds, who, as a New York City school administrator, launched that city’s effective schools program in 1978. ‘See, for example, W.B. Brookover and L.W. Lezotte. Changes m School Charactenstrcs Coincident with Changes in Student Achievement (East Lansing: Institute for Research on Teaching, Michigan State University. 1979); M.M. Rutter and others, Fifteen-Thousand Hours: ,Secondary Schtxns and their Effects on Children (Cambridge. Mass.: Harvard I’niversity Press. 1979): G. Weber. Inner-Citv Children Can Be Taught to Read: Four Successful Schools (Washington. D.C.: Council for B-a- tion. 1971). ‘James Coleman and others, Equality of Educational Opportunity (Washmgton. D.C National Center for Education Statistics. 1966); Christopher Jenks and others, Inequahty: A Reassessment of the Effect of Family and Schooling in America (New York: Baser Books. 1972) Page 12 GAO,/HRD89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 3 GAQ Characteristics Frequently Cited in Effective Schools l Strong principal leadership l A pervasive and broadly understood instructional focus l Safe & orderly school climate l High teacher expectations for student achievement l Student achievement data used for evaluating program success J Edmonds defined an effective school as one in which equal percentages of students from varying socioeconomic backgrounds achieve a mini- mum level of mastery in basic skills. He selected five characteristics identified by school effectiveness research and popularized their use as a model for school improvement (see fig. 3). Literature describing existing effective schools programs indicates that they make extensive use of this five-characteristic model, although many variations exist. In addition, these five characteristics are incorpo- rated in the definition of an effective schools program used in the Haw- kins-Stafford Amendments of 1988 (see app. 2 for the list in P.L. lOO- 297). Page 13 GAO/HRD89-132BR Effective Schools Programs Effective Schools Pmgrams: Their Extent and Characteristics In some of the literature we reviewed, researchers were critical of the widespread growth of programs based on the findings of school effec- tiveness research.3 Criticism of the research is leveled against the (1) reduction of the findings to a set of specific characteristics, including the five just mentioned, to be used as a formula for school improvement; (2) generalization of the findings of studies conducted in urban elemen- tary schools to all levels of schools in urban and nonurban areas; (3) notion that once aware of a set of specific characteristics, schools can simply decide to adopt them; and (4) lack of causal evidence about what actions might bring about these specific characteristics in a school. Analyzing Academic When schools and districts analyze students’ academic achievement to Achievement to Evaluate evaluate the results of effective schools programs, both the experts we interviewed and the literature we reviewed indicated that measures of Effective Schools academic achievement should be disaggregated (that is, analyzed sepa- Programs rately) by student socioeconomic status and ethnicity.l Distinguishing the academic achievement of low-income and minority students permits schools and districts to determine whether children in these subgroups show improvement. Without separate analyses, achievement data would remain aggregated, and a school or district could incorrectly interpret achievement gains among only some children-for example, those who are nonminority or from higher-income families-as improvement among all children. How school enrollments are mixed demographically may influence which student subgroups are relevant for disaggregation, according to some experts we interviewed. For example, in districts where schools with the programs by and large serve only children from low-income families: it may not be relevant to disaggregate by student socioeco- nomic status. %ec. for example, R. Rowan. S. Bossert, D. Dryer, ” Research on Effectrve Schools: A Cautionary Note.” Educational Researcher, Vol. 12, No.4 (April 1983) pp.24-31; S.C. Purkey and M.S. Smith. “Effective Schools: A Review.” The Elementary School Journal. Vol. 83. No. 4 (1983). pp.427~51; M. Cohen, “Instructional Management and Social Conditions in Effective Schools,” in School Finance and School Improvement: Linkages in the 1980s ,4. Odden and L.D. Webb, eds. (Washington. DC.: Amen- can Educational Finance Association, 1983); J. D’Amico. “Each Effective School May Be One of A Kind.” Educatronal Leadership (Dec. 1982). pp.61~62. ‘Student socioeconomic status can be approximated with actual family income, level of parent educa- tron. student participation in the free or reduced-price lunch (or other pubhc assistance) program, and the language spoken at home Page 14 GAO/HRDW-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 4 GAQ Federal Funds Authorized for ESPs ESPs may be funded with l Chapter 1 funds for schoolwide projects , l Chapter 2 funds for local and state programs Some schools and districts with effective schools programs seek to nar- row the performance gap between students; thus, they may establish goals for raising the academic achievement of a certain percentage of lower-achieving students to a higher level. To determine whether such a goal is met, a school or district could analyze achievement measures for students grouped according to their prior achievement level, without disaggregating the data by student socioeconomic status or ethnicity. This method of evaluating program results, however, would not disclose whether subgroups of children (for example, low-income or minority) that were lower achievers in an earlier period show improved perform- ance in a later period. Page 15 GAO/HRD&W-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Federal Support for Public Law loo-297 includes the first Chapter 1 and Chapter 2 program Effective Schools authorizations for effective schools programs (see fig. 4). In fiscal year 1989, a total of $4.6 billion was appropriated for Chapter 1 and $462.8 Programs million for the Chapter 2 block grant program. Chapter 1 funds may be used for effective schools programs when dis- tricts fund these programs as schoolwide projects. Such projects may only operate in schools where at least 75 percent of the children enrolled are from low-income families. The purpose of schoolwide projects is to upgrade a school’s entire educational program rather than serve only certain students like other Chapter 1 programs. Chapter 2 funds may be used by districts for effective schools programs as innovative programs to carry out schoolwide improvements-one of six authorized local targeted assistance programs (see app. 3). Public Law loo-297 requires SEASto use a portion of the Chapter 2 funds they retain for effective schools programs (see fig. 5). SEASmust distribute at least 80 percent of their Chapter 2 funds to school districts for use among the authorized targeted assistance pro- grams. The other 20 percent is reserved for state programs that include (1) Chapter 2 program administration, (2) assistance to school districts and statewide activities to provide targeted assistance, and (3) assis- tance to school districts and statewide activities to support effective schools programs. Of Chapter 2 funds reserved for state programs, SEASare required to use at least 20 percent for effective schools programs, unless a waiver is granted by the Secretary of Education. The Secretary may waive this requirement if a state already spends twice the required amount from nonfederal sources on these programs. The total amount of state Chapter 2 funds available for effective schools programs in fiscal year 1989 was about $18.4 million. Individual state funds available for effec- tive schools programs ranged from about $92,000 in states such as Dela- ware and Vermont to $1.9 million in California. Page 16 GAO/HRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 5 GAQ State Distribution of Chapter 2 Funds SEAS distribute at least 80% of funds to districts -- ~TObe used for six local targeted assistance programs, including ESPs Remaining 20% of funds reserved for state use -- @Statesmust use a portion for ESPs Page 17 GAO/HRD-89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 6 GAQ Study Objectives Determine Number of U.S. districts with effective schools programs Identify Common ESP characteristics and practices Describe How districts evaluate program effect on achievement Discuss Federal requirements for evaluating ESPs Page 18 GAO/HRDW-132BR Effective Schools Programs EffectiveSchools Programs: Their Extent and Characteristics Figure 7 w Methodology Interview Federal and state officials Researchers / program experts Review Effective schools research Survey A nationally representative sample of school district superintendents To provide needed information on the extent and characteristics of Objectives, Scope, and effective schools programs in the nation’s school districts, the Chairman Methodology of the House Education and Labor Committee asked us to conduct a national study of these programs. In response to his request, we defined our objectives as shown in figure 6. The methodology used to accomplish our objectives is summarized in figure 7. To help us plan our study and develop a questionnaire for 1,685 school district superintendents! we (1) interviewed experts as well as Department of Education and SEA offi- cials and (2) reviewed the research literature on effective schools. The questionnaire, which was the primary means through which we Page 19 GAO/HRD89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics obtained information about effective schools programs nationwide, cov- ered topics relating to the development, implementation, and evaluation of programs operating in school year 1987-88. We mailed copies of the questionnaire to the superintendents of a strati- fied random sample of school districts across the country; we received responses from 82 percent of the school districts in our sample. Our findings are representative of the approximately 16,000 school districts in the nation.“, 6 In cases where information was incomplete or discrep- ancies occurred in districts’ responses, we spoke with the respondents and made appropriate changes. We did not visit districts to verify the accuracy of their responses. We used a stratified sample design to obtain national representativeness and minimize sample size. The strata for our sample consisted of (1) 3 10 school districts, selected randomly from 873 districts identified by experts as having effective schools programs-including school-based planning teams-in which student achievement was disaggregated by student socioeconomic status; (2) 223 school districts, selected randomly from 309 districts identified by experts as having effective schools pro- grams-including school teams-but with unknown evaluation prac- tices; (3) the 50 largest school districts of the nation’s 50 largest cities (see app. V for a list of cities);; and (4) 1,089 school districts, selected randomly from the 1986-87 public education agency universe of the Department of Education’s common core of data (CCD).s We discussed our questionnaire results with Department of Education officials and incorporated their comments where appropriate. We con- ducted our study during October 1988 through April 1989 in accordance with generally accepted government auditing standards. ‘Our sample only includes local school districts, which do not include regional educatron service agen- cies supetxison union administrative centers. state-operated agencies, and federally operated agen- cies. whrch together total approximately 1.300 districts.. (Source: t‘.S. Department of Education, Center for Education Statistics. ) “The confidence internals for our data are shown in appendix I\ ‘Ue selected the fifty largest cittes based on 1986 Bureau of the Census population estimates. sCCD 1s a universe suney of I’.% schools and districts and is not subJect to sampling error. It consists of data suitted annually by all SEAS m response to SIX sumey mstruments administered by the Center for Educatlon Statistics, whtch mamtams the database The 1986-87 CCD public education agency umverse had a response rate of 9 1 percent Page 20 GAO/HRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Fiaure 8 GAO Criteria Used to Measure Extent of ESPs Criterion 1 School improvement programs based on findings of effective schools research Criterion 2 Also use school teams and written improvement plans Criterion 3 Also evaluate ESPs by disaggregating achievement data by student socioeconomic status or ethnicity We used various criteria to measure the extent of effective schools pro- Many School Districts grams to provide the Congress with information on (1) the overall Operate Effective number of districts and schools with the programs and (2) the extent Schools Programs that these programs include specific activities recommended by program experts (see fig. 8). Page 2 1 GAO,/HRDSS-132BR Effective Schools IVograms Effective Schools Programs: Their Extent and Characteristics Figure 9 - GAQ Extent of ESPs in U.S. Districts Meeting Criterion 1 Majority have or plan to start programs Not Planning to Implement an ESP k 42% Had ESP in School Year 1987-88 Planning to Implement an ESP in School Years 1988-89 or 1989-90 Using the first criterion, which stipulates that the programs be based on the findings of effective schools research, about 41 percent or 6,500 of the nation’s school districts had effective schools programs in operation in approximately 38,000 elementary and secondary schools during school year 1987-88 (see fig. 9). An additional 17 percent or about 2,600 of the nation’s districts have plans to implement effective schools pro- grams during school years 1988-89 or 1989-90. Many of the programs have been recently established (see fig. 10). Over half of the district offi- cials reported that their effective schools programs were first imple- mented during school years 1986-87 or 1987-88. Page 22 GAO/HRJM9-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 10 GAQ Many ESPs Recently Implemented Over half began during school years 1986-88 Pwcm( d School Dbtrkta With ESPs Mnting CrHmdon 1 50 55 40 45 I 35 50 20 25 15 r’ 10 Bdem 1sso-52 105244 1054.55 loss-55 107S50 The second program criterion includes those programs that use (1) school-level teams on which teachers and administrators work together to plan and monitor their programs and (2) written plans for improving school effectiveness; about 2’7 percent or 4,300 of the nation’s districts had such effective schools programs in approximately 27,000 schools (see fig. 11 and see app. VII). Program experts encourage school teams and written plans to help foster school-level planning and decisionmak- ing by school staff, which (as discussed on p, 10) are objectives of effec- tive schools programs. Page 23 GAO/HRD-89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 11 w Using Different Criteria, Extent of ESPs in U.S. Districts 45 Pucmt of Schd Didrkts Wh ESR 40 55 30 r-l- J When using the third criterion-that districts also disaggregate aca- demic achievement data by student socioeconomic status or ethnicity when evaluating their programs-only about 13 percent or 2,100 of the nation’s school districts had effective schools programs in approxi- mately 18,000 elementary and secondary schools (see fig. 11)” As dis- cussed on pages 14 and 15, such analyses permit schools or districts to determine whether low-income and minority children show improvement. “For the purpose of cnterion 3. academic achievement data may be obtained using achievement tests or nontest measures such as student grades. attendance, and dropout rates. Page 24 GAOjHRJS89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 12 r MO Which Schools and Districts Have ESPs? (Criterion 1) l ESPs are about as likely in schools at each grade level l ESPs commonly serve mix of low-income and higher-income children l ESPs are more likely in large districts l ESPs are about as likely in urban and nonurban districts Which Schools and we collected data on the grade levels of schools with the programs, the Districts Have socioeconomic diversity and size of their enrollments, and their geo- Effective Schools graphic location (see fig 12). Programs? Programs About as Likely Effective schools programs meeting the first criterion operated in ele- at Each Grade Level mentary and secondary schools at an average rate approximately com- mensurate with districts’ average total percentage of elementary and Page 25 GAO/HRD89-132BR Effective Schools Program Effective Schools Programs: Their Extent and Characteristics Fiaure 13 G’GAQESPs About as Likely at Each Grade Level Pwmnt of Schxds in Districts With ESR 55 50 45 40 35 30 25 20 15 10 5 0 Gndm Lawl of Schools In Districts Wfih ESPs secondary schools, district officials reported (see fig. 13). For example, an average of about 53 percent of effective schools programs operate in elementary schools, with elementary schools making up, on average, 52 percent of the total number of schools in districts with programs. In addition, an average of 20 percent of effective schools programs operate in high schools, with high schools making up, on average, about 21 per- cent of the schools in districts with programs. Page 26 GAO/HRIM9-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 14 @ko Socioeconomic Status of Children ESPs serve mix of low-income and higher-income children Pumnt of School Districts With ESPs 20 15 10 0 l-10 11-20 21-30 31-w 41.50 5140 61-70 Percent of Children on Frm or Reduced-Price Lunch In Schools Wfih ESPs Programs Commonly Serve Effective schools programs meeting the first criterion commonly serve Mix of Low-Income and mixed enrollments of low-income and higher-income children, district officials reported (see fig. 14). To estimate the socioeconomic diversity Higher-Income Children of districts’ school enrollments, we asked district administrators to spec- ify the percentage of children who participated in the free or reduced- price lunch program in those schools with effective schools programs.l” “‘The percentage of children participating in the free or reduced-price lunch program may underestl- mate the percentage of low-Income children in a school because all eligible children do not participate. Children that receive free lunch have family Incomes of 130 percent or less of the official poverty threshold. which was 59.431 for a 3-person family in 1988; children that receive reduced-price lunch have family incomes of 185 percent or less of the official poverty thrt-;hold. Among all children that recewed free or reduced-price lunchrs during the first 6 months of fiscal year 1989, 85.5 percent received free lunches. (Source: 1’S Lkpartment of Commerce. Bureau of thr Census and U.S. Depan- ment of Agnculture. Child Sutntlon Dlvislon ) Page 27 GAOiHRD89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 15 w Large Districts More Likely to Have ESPs Than Small Districts Percent of School Districts 60 70 60 50 40 30 20 Programs More Likely in Effective schools programs are somewhat more likely to operate in large Larger School Districts school districts (enrollments of 2,500 or more). For example, although districts with enrollments of 10,000 students and over make up 4 per- cent of all U.S. districts, these districts account for about 8 percent of the districts with effective schools programs meeting the first criterion (see fig. 15). Likewise, districts with enrollments of 2,500 to 9,999 stu- dents make up 18 percent of all ITS. districts and about 29 percent of the districts with programs. Page 28 GAOiHRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 16 GAQ ESPs About as Likely in Urban and Nonurban School Districts 60 Porcenl of School Districts Primarily MSA but Not Outsid. of MSA centrai city of Prinmtfly MSA Central City Geographical Area Served by School Districta 1 1I School Districts With ESPs All U.S. School Districts Programs About as Likely Effective schools programs operate in urban school districts, including in Urban and Nonurban central city districts, at a rate nearly commensurate with the percentage of such districts in the nation. We defined urban districts as districts Areas that serve a metropolitan statistical area (MSA)and central city districts as districts that serve the central city of an MSA." For example, central city school districts make up 4 percent of all U.S. districts and about 5 percent of the districts with effective schools programs meeting the ’ I.411 MS.4 has one or more central counties contammg an urbanized area wth at least 50.000 inhabi- tants. Xn MS.4 may also mcludc outlymp counties that have close economic and social relationships with the central counties Page 29 GAO/HRDSS-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 17 r GAQ School Characteristics Emphasized by ESPs Many districts cited all nine of these characteristics l Strong instructional leadership l High achievement expectations l Pervasive instructional focus l Monitoring student achievement l Basic skills acquisition l Staff collaboration l Academic learning time l Parental support l Safe & orderly school climate first criterion (see fig. 16). We found that at least 39 of the nation’s 50 largest cities had effective schools programs in their districts during school year 1987-88 (see app. V). School Characteristics Effective schools programs seek to develop or improve on school charac- teristics identified by effective schools research as associated with high Emphasized by student achievement (see p. 12). We asked district administrators to Programs indicate which school‘characteristics their programs emphasized through specific activities such as needs assessments! staff development Page 30 GAO/HRD-89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics training, and changes in classroom instruction and school management proceduresL2 According to t,heir responses, in about 88 percent of the 6,500 districts, the programs sought to improve on at least six of nine school characteristics listed in our questionnaire (see fig. 17). The pro- grams in about 67 percent of the districts sought to improve on at least eight characteristics, and the programs in about 53 percent of the dis- tricts emphasized all nine (see app. VII for further details on fig. 17). A study that used a nationally representative survey of high schools provides another example of effective schools programs emphasizing many different characteristics. 13In this study, a majority of principals reported that effective schools programs in their schools gave major emphasis to at least 12 characteristics identified by effective schools research. The authors of this study questioned how the schools could devote major emphasis to so many characteristics simultaneously since each one was likely to require considerable effort. 1'011r wonnaire listed rune school charactenstics that were frequently wed in the literature on effectn Y schools research (see fig. 17 ). “‘SC. Purkey. R.A. Rutter, and FM. Newmann, ” I5 High School Improvement Programs: A Profile from the IIlgh School and Beyond Supplemental Suney.” Metropolitan Education, Vol.3 (Winter 1986-87). pp. 5Wl. The suney used was the 1984 High School and Beyond Supplemental Survey of Principals and ‘Teachers Page 31 GAO/HRWS132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 18 GAQ School District Involvement in ESPs Half of the districts required schools to implement programs Districts in Which All Schools With ESPs Voluntarily Chose to Have Them 46% Districts in Which All Schools With ESPs Were Required to Have Them Districts in Which Some Schools With ESPs Were Required to Have Them Our questionnaire results indicate that in half of the school districts Many Districts with effective schools programs meeting the first criterion, some or all Require Schools to schools with the programs were required by their districts to implement Implement Effective them (see fig. 18). In the other half of the districts with programs, schools implemented them on a voluntary basis. Some districts that Schools Programs require effective schools programs do so because of state laws that require (or encourage) school improvement programs (see fig. 19). Page 32 GAO/~89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Fiaure 19 @W Examples of State Laws that Pertain to ESPs States such as Maine, Indiana New York, and South Carolina, l Require or fund improvement programs in schools, or l Require schools to prepare improvement plans, or l Have accreditation standards that include effective schools objectives Some of the literature we reviewed discussed the strengths and weak- nesses of approaches in which districts become involved in schools’ deci- sions to implement an effective schools program.14 l Top-down approach: The school district requires schools to implement effective schools programs and determines program content, goals, and expected outcomes, usually focused on achievement tests. A strength ‘%e for example. SC. Purkev and M.S. Smith, “School Reform: The District Policy Implications of the Eifective Schools Literature.” The Elementary School Journal. Vol. 45. No. 3 (Jan. 1985), pp. 353- 63; L. Cuban, “Transforming the Frog into a Prince: Effective Schools Research, Policy, and Practice at the District Level.” Harvard Educational Review. Vol. 54. Ko. 2 (May 1984), pp. 129-37. Page 33 GAO/lXRD84132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics cited in this approach is that reform will be attempted where it is needed. But a weakness may be that the lack of staff choice in program implementation and management, as well as the focus on test scores, may diminish (1) school staffs’ sense of responsibility and commitment toward the program and (2) flexibility in curricula, classroom practices, and program management. l Bottom-up approach: The school district may provide incentives, such as program funds and teacher-release time from classes for program plan- ning, but each school voluntarily chooses to implement an effective schools program, determines its own agenda, and uses district funds in the manner that staff and parents choose. Choice and flexibility are strengths of this approach, but a weakness cited is that schools that could benefit from the program may choose not to participate or may drop out as staff enthusiasm declines. l Mixed approaches: This approach combines the strengths of the other two approaches. The district may require schools to implement effective schools programs and provide incentives that encourage staff planning and collaboration, such as funds for substitutes, but then relies on school staffs for program design, implementation, and management. Programs Frequently SEAS,universities, federally funded regional educational laboratories, Developed and and other agencies or individuals outside of the school district fre- quently assist districts and schools in developing and implementing Implemented With effective schools programs, according to our survey results and the liter- External Assistance ature we reviewed. To assist principals and teachers plan and implement their programs! agencies and individuals outside of the school district may develop program models and provide staff training and technical assistance; SEASmay provide financial support to districts and schools. Our questionnaire results indicate that about 64 percent of the districts with effective schools programs meeting the first criterion received external assistance in developing their programs. Many of these pro- grams were developed with assistance from an SEA(see fig. 20). Page 34 GAO NRD-89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 20 GM Sources of External Assistance for ESPs Many districts received assistance from SEAS Pwsnc of School Dhktr lhl Rocehd Exturul Assishnm r 65 60 \ 55 50 45 40 35 30 - 25 20 15 10 5 0 1 Page 35 GAO/HRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 21 GZAQExamples of SEA Support for ESPs l Connecticut and Minnesota SEAS have ESP model l New York, Michigan, and Kentucky SEAS provide training and technical assistance l California, New Jersey, and Wisconsin SEAS provide financial support to schools or districts Through interviews with state officials and program experts in all 50 states and the District of Columbia, we identified 31 SEASthat assisted schools and districts in developing or implementing effective schools programs (see fig. 21 and see app. VI for a listing of SEASand examples of the assistance they provide). We found that SEXstaff provided assis- tance through such means as school visits and staff development work- shops, including a train-the-trainers approach, in which SEA staff train district staff to train key school staff. For example, an assistant superin- tendent trained school principals in Marion School District 1, South Car- olina (see fig. 22). A trainer from SEAobserved and evaluated this workshop. Page 36 GAO/HRD-89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 22: Train-The-Trainers Staff Development Training Page 37 GAO/HRD89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Fipure 23 m How Do School Districts Evaluate ESPs? l Districts use achievement tests and nontest measures, but ... l Districts seldom disaggregate achievement data by student socioeconomic status or ethnicity; thus, they seldom determine if ESPs help all children School district administrators reported using various achievement tests Districts Seldom and nontest measures to evaluate the results of effective schools pro- Determine If Programs grams on student academic achievement (see fig. 23). When evaluating Help All Children their programs, however, school districts seldom determine if the aca- demic achievement of low-income and minority children is improved because the districts seldom disaggregate achievement data by student socioeconomic status or ethnicity. Page 38 GAO/HRD-89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 24 GCAQProgram Evaluation Districts use achievement tests and nontest measures 1 9% Used Nontest Measures Only Used Both Achievement Tests and Nontest Measures Used Achievement Tests Only Various Measures of The majority of school districts use both achievement tests and nontest Academic Achievement measures to evaluate their programs (see fig. 24). The achievement tests used by school districts typically measure basic language arts and math- Used ematics skill levels or assess mastery of specific curriculum objectives. Nontest measures used by school districts may include student grades, attendance, dropout rates, and enrollments in advanced and remedial classes. Page 39 GAO/KRD-89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 25 GAQ District Evaluations Using Achievement Tests Data seldom disaggregated PwcantdSchdDtdrktsWithESF’s Academic Achievement of When evaluating their programs, districts seldom disaggregate achieve- Student Subgroups Seldom ment test results or nontest measures by student socioeconomic status, ethnicity, or sex (see figs. 25 and 26). But districts frequently disaggre- Analyzed gate these measures by students’ prior achievement level. Analyzing achievement measures by prior achievement level may allow a school or district to determine whether an increased percentage of students is per- forming at a higher achievement level (see p. 15). Conducting such anal- yses, however. without disaggregating data by student socioeconomic status or ethnicity may fail to disclose whether relevant subgroups of children show improved performance among those students that were lower achievers in an earlier period. Page 40 GAOjHRD89-132BR Effective Schools Progmma Effective Schools Programs: Their Extent and Characteristics Fiaure 26 GAQ District Evaluations Using Nontest Measures Data seldom disaggregated P-of School Districts With ESPs 100 90 80 70 60 so 44 30 Page 41 GAO/BRDM-132BR Effective Schools F’rograms Effective Schools Programs: Their Extent and Characteristics Figure 27 GAQ In Districts Not Disaggregating By Socioeconomic Status ESPs serve mix of low-income and higher-income children 25 Porcmt of DIrtrids That Do Not Disaggmgnta Data By Socioeconomic Status 20 15 10 5 0 a 0 l-10 11-20 21-30 31-40 41-50 51a 61-70 71-80 81-90 91-100 Pareont of Chlldmn on Free or Reduced-Prlca Lunch in Schools With ESPs J Since the demographic mix of school enrollments may influence which student subgroups are relevant for separate analyses (see p. 14) we examined the socioeconomic diversity of the children served by the pro- grams in those districts that did not disaggregate achievement measures by student socioeconomic status. In such districts (see fig. 27) effective schools programs commonly served mixed enrollments of low-income and higher-income children. Because these districts do not distinguish the academic achievement of their low-income children, the districts could incorrectly interpret achievement gains among only some previ- ously lower-achieving children-for example, those from higher-income families-for improvement among all children. Page 42 GAO/HRD-89-132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Figure 26 GAQ Chapter 1 and 2 Evaluation Requirements in P.L. loo-297 @The Department must conduct a national study of ESPs funded under Chapter 2 l States must evaluate Chapter 1 and 2 programs and submit evaluations to the Department l The Department must summarize these state evaluations and periodically report to Congress Public Law loo-297 requires the Department of Education to contract to Federal Evaluation conduct a national study on effective schools programs funded under Requirements for Chapter 2 (see fig. 28). The law also requires SEASto evaluate programs Effective Schools supported with Chapter 1 and Chapter 2 funds, using data collected from school districts; the law. however, requires no specific state or Programs local evaluations of effective schools programs. SEASmust submit their evaluations to the Department biennially for Chapter 1 programs and in fiscal year 1992 for Chapter 2 programs. The Department in turn must provide the Congress with evaluations of Chapter 1 and 2 programs through reports summarizing SEAevaluations Page 43 GAO/m89.132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics of local programs. The Department must submit these reports biennially under Chapter 1 and by October 1, 1992, under Chapter 2. Public Law loo-297 requires the Department to develop (1) national standards for state and local evaluation and reporting purposes under Chapter 1 and (2) a model data collection and reporting system that defines specific data categories to be used by states, at their discretion, for required state evaluations and reports under Chapter 2. Chapter 1 Evaluation and Under Chapter 1, effective schools programs may be funded as school- Reporting Requirements wide projects, which may only operate in schools where at least 75 per- cent of the children enrolled are from low-income families. Public Law loo-297 and Chapter 1 program regulations require districts to annually evaluate schoolwide projects operating in their schools and to make the results, including achievement and other assessment data, available to their SEASand the public. According to an official in the Department of Education’s planning and evaluation service, the Department plans to examine effective schools programs and their results as part of an implementation study of the Chapter 1 program. The study will include information on (1) the frequency and nature of schoolwide projects and (2) their inclusion of features of effective schools programs. The Depart- ment expects the final report of this study to be available by December 1991.‘” In addition, Public Law loo-297 and Chapter 1 program regulations require districts to evaluate their Chapter 1 program as a whole at least once every 3 years and to suit the results to their SEAS.SEAS,in turn, must use this information for state-level evaluations that are suitted to the Department. SEASmust also collect data on the race, age, sex, number of children with handicapping conditions, and number of chil- dren by grade level served by Chapter 1 programs; SEAS must annually suit these data to the Department. Neither Public Law loo-297 nor Chapter 1 program regulations require districts or states to collect information on, or evaluate, effective schools programs separately from other Chapter 1 programs. Public Law lOO- 297 also does not require the Department to separately evaluate effec- tive schools programs funded under Chapter 1. According to the deputy director of compensatory education programs, because districts and states are not required to operate effective schools programs under 15According to the Department. it will also examine effective schools programs and their results in a longitudinal evaluation. begmning dunng school year 199<)-91. of the Chapter 1 program. Page 44 GAO/HRD89132BR Effective Schools Programs Effective Schools Programs: Their Extent and Characteristics Chapter 1, the Department believes that federally required data collec- tion for these programs would be burdensome to SEASand school districts. Chapter 2 Evaluation and Under Chapter 2, districts may fund effective schools programs as inno- Reporting Requirements vative programs to carry out schoolwide improvements-one of six authorized Chapter 2 local targeted assistance programs. Because the Department of Education has not made final decisions on the specific data categories that the Chapter 2 model data collection and reporting system will include, it is uncertain whether any data specifically on effective schools programs will be collected under the state and local evaluation provisions of Chapter 2. Data for these programs will be col- lected, however, for state and local Chapter 2 reporting purposes, according to a draft data collection form we obtained from the Department. By January 1990, the Department of Education expects to define the model system’s data categories for evaluation purposes. The Depart- ment informed us that it is considering collecting specific data on effec- tive schools programs, but wishes to avoid undue burden on the states. Proposed data categories for state and local reporting purposes were specified during spring 1989. These include (1) the number of children, by grade level, served by each authorized purpose for local targeted assistance, including effective schools programs, (2) the number of schools receiving Chapter 2 funds, and (3) the total amount of Chapter 2 funds used for each targeted assistance purpose. Concerning the required national study of effective schools programs funded under Chapter 2! the Congress intends data collection for this study to include student achievement outcomes disaggregated by stu- dent socioeconomic status and ethnicity.‘” The Department informed us that it plans to award the contract for the study, if funds are available, during fiscal year 1990 and, if so, expects the study to be completed no later than fiscal year 1992.” ‘“l:.S. Congress House. Elementaq and Secondary Education Conference Report to Accompany H.R. 5 (Rept. 100-567. Apr 13. 1988, p.423) “The Department said that it also Intends to examine effective schools programs and their results in an ongoing study of effective mstructional practices and curricula in elementary schools with high concentrations of disadvantaged students. Page 4.5 GAO/HID89132BR Effective Schools F’rograms Appendix I SelectedCharacteristics of Effective Schools Cited in ResearchLiterature Strong instructional leadership A safe, orderly, and disciplined school climate High expectations for student achievement Monitoring student achievement frequently to evaluate progress Staff consensus on explicit instructional goals and values Collaborative, collegial instructional planning Ongoing, schoolwide staff development training Teacher accountability and acceptance of responsibility for student per- formance A focus on basic-skills acquisition An emphasis on higher-order cognitive skills Increased academic learning time Cooperative activity and group interaction in the classroom Teacher responsibility for instructional and classroom management decisions Clearly recognized principles and guidelines for student behavior and performance Individual school autonomy and flexibility Staff stability and continuity High levels of parental involvement and support District-level support for school improvement High teacher morale and sense of community in school Schoolwide recognition of academic success Teacher empathy, rapport, and personal interaction with students Strategies to avoid retaining students in grade Source: A compilation of school characteristics that appeared in D.E. Mackenzie, “Research for School Improvement: An Appraisal of Some Recent Trends,” Educational Researcher (Vol.12, No.4, Apr. 1983, pp.5- 17); school characteristics cited in research reviewed in S.C. Purkey and MS. Smith, “Effective Schools: A Review,” The Elementary School Jour- nal (Vol.83, X0.4, March 1983, pp.427-452). Page 46 GAO/HRD89-132BR Effective Schools Programs Five School Characteristics in the Definition of Effective Schools Programs Included in Public Law loo-297 l Strong and effective administrative and instructional leadership that creates consensus on instructional goals and organizational capacity for instructional problem solving . Emphasis on the acquisition of basic and higher-order skills l A safe and orderly school environment that allows teachers and pupils to focus their energies on academic achievement l A climate of expectation that virtually all children can learn under appropriate conditions l Continuous assessment of students and programs to evaluate the effects of instruction Source: U.S. House of Representatives, Elementary and Secondary Edu- cation Conference Report to Accompany H.R. 5 (Report 100-567, April 13, 1988, p.85). Page 47 GAOiHRD-89.132BR Effective Schools F’rograms Appendix 111 Chapter 2 Block Grant Program: Six Authorized Local Targeted Assistance Programs l Programs to meet the educational needs of students at risk of dropping out and students for whom providing an education entails higher-than- average costs l Programs for the acquisition and use of instructional and educational materials, including library books, reference materials, computer soft- ware and hardware for instructional use, and other curricular materials that would be used to improve the quality of instruction l Innovative programs designed to carry out schoolwide improvements, including effective schools programs l Programs of training and professional development to enhance the knowledge and skills of educational personnel, including teachers, librarians, school counselors and other pupil services personnel, and administrators and school board members l Programs designed to enhance personal excellence of students and stu- dent achievement, including instruction in ethics, performing and crea- tive arts, humanities, activities in physical fitness and comprehensive health education, and participation in community service projects l Other innovative projects which would enhance the educational pro- gram and climate of the school, including programs for gifted and tal- ented students, technology education programs, early childhood education programs, community education, and programs for youth sui- cide prevention Source: U.S. House of Representatives, Elementary and Secondary Edu- cation Conference Report to Accompany H.R. 5 (Report 100-567, April 13, 1988, p.83). Page 48 GAO/HRD-89132BR EfTective Schools Programs Estimates and Confidence Intervak for Selected Questionnaire Responses Universe or Estimated range0 school Lower “1%; Variable districts Estimate limit Number of districts wtth ESPs meetrng criterion 1 15,700 Had ESP in school vear 1987-88 6.509 6.053 6,965 Planmng to Implement an ESP tn school year 1988-89 or 198990 2,597 2,247 2,947 Not plannrng to implement an ESP 6,594 6,132 7,056 School years ESPs frrst Implemented (percent) 6,509 1979-80 and before 5 3 8 1980-82 4 3 6 1982-84 8 6 10 1984-86 25 21 29 1986-88 57 53 62 Number of school districts with ESPs by program criteria Criterion 1 15,700 6,509 6,053 6,965 Criterion 2 6.509 4,304 4,022 4,585 Cnterion 3 6,509 2,105 1,834 2,377 Number of schools with ESPs by program criteria Critenon 1 6,472 38,285 34,807 42,087 Cntenon 2 4.300 27.431 24.291 30.790 Crtterion 3 2.102 17,794 15,103 20.612 Use of school teams and written plans (percent) 6,488 Used school teams 89 86 93 Used written plans 71 67 75 Used both 66 62 70 Grade level of schools with ESPs (oercent) 6.339 Elementarv 53 50 56 lntermedrate --____ 18 16 20 Hrgh school ____-. 20 18 22 Other 10 7 12 Grade level of all schools In drstncts with ESPs (percent) 6,339 Elementary ---___- -. 52 50 55 Intermediate 18 16 19 High school 21 19 23 Other 10 7 13 Page 49 GAO/HRDW-132BR Effective Schools Programs Appendix N Estimates and Confidence Intervals for !Selected Questionnaire Responses Universe or Estimated range0 school Lower “yg Variable districts Estimate limit Percent of children on free or reduced- price lunch served bv ESPs (percent) 6,410 0 2 1 3 l-10 17 14 21 11-20 21 18 25 21-30 18 1.5 22 31-40 17 14 21 41-50 8 5 10 51-60 6 4 8 61-70 4 2 6 71-80 3 1 4 81-90 2 1 4 91-100 1 0 2 Student enrollment tn drstncts wrth ESPs (percent) 6,452 10.000 + 8 6 11 -___ 2,500 - 9,999 29 24 32 1 2,499 62 58 67 Geographic area of school drstncts w;h ESPs (percent) 6,452 Pnmanly central city of MSA 5 3 7 MSA but not onmanlv central crtv 41 37 46 Outside of MSA 54 50 59 Number of school characterrstrcs cited by drstncts (percent) _______- 6,509 At least SIX of 9 88 85 91 At least eight of 9 67 63 72 All 9 -___-______ 53 48 57 School charactenstrcs cited by drstncts (crltena 1 ESPs) Instructtonal leadership 6,490 92 89 94 Staff expectations 6 490 92 89 94 Instructional focus 6,490 90 87 93 Monrtonng student achievement 6,489 88 86 91 Basrc-skulls acqursltron 6,474 89 86 91 Staff collaboration 6,490 86 83 89 Academrc learning trme 6,490 83 80 87 Parental support 6.490 81 77 84 Safe and orderly climate 6 490 76 72 80 (continued) Page 50 GAO/HRD+4132BR Effective Schools Programs Appendix IV E&mates and Confidence Intervals for selected Questionnaire Respomes Universe or Estimated range’ school Lower yi$; Variable districts Estimate limit School characteristics cited by districts (criteria 2 ESPs) (percent) Instructional leadership 4,285 92 89 96 Staff expectations 4,285 92 89 95 lnstructronal focus 4,285 93 90 96 Monitonna student achievement 4,284 90 87 93 Basrc-skulls acquisition 4,269 92 89 95 Staff collaboration 4,285 89 85 92 Academtc learnrng trme 4,285 84 80 88 Parental support 4.285 85 81 89 Safe and orderlv climate 4,285 80 75 84 School characteristrcs cited by districts (criteria 3 ESPs) (percent) 2,087 Instructional leadership 96 93 99 Staff expectations 94 90 98 Instructional focus 97 94 100 Monitoring student achrevement 91 87 96 Basic-skills acauisrtron 96 92 99 Staff collaboration 94 90 97 Academic learning time 89 84 94 Parental support 92 88 96 Safe and orderlv climate 88 82 93 School dtstnct involvement In ESPs (percent) 6,468 All schools with ESPs were required to have them 46 42 51 Some schools with ESPs were required to have them 4 2 6 All schools with ESPs voluntanly chose to have them 50 45 54 Whether distnct developed program with external assistance (percent) 6,505 Yes 64 60- 68 No 36 32 40 Source of external assistance (percent) SEA ---___ 4.161 64 58- 69 University 4.142 -31 26 36 Educational laboratory 4,127 36 30 41 Other 4 149 31 26 36 (continued) Page 51 GAO/BRD-S9132BR Effective Schools F’rograms Appendix IV Estlmat.es and Confidence Intervala for selected QuestioNuLre Reaponsea Universe or Estimated range. school Lower “;;; Variable districts Estimate limit Measures of achievement districts used to evaluate results of ESPs (percent) Achievement tests 6,505 83 80 87 Nontest measures 6,509 75 70 78 Both achievement tests and nontest measures 6,505 66 62 70 Achievement tests only 6,505 17 14 21 Nontest measures only 6,505 9 6 11 No achievement tests or nontest measures used 6,505 8 6 11 Student subgroups by which districts disaggregated achievement test results (percent) Socioeconomic status 5,386 Regularly 12 9 15 Occasionally 21 17 25 Ethnicity 5,400 Regularly 9 6 12 Occasionally 18 14 21 Sex 5,385 Regularly 9 6 12 Occasionally 18 14 21 Prior achievement level 5,401 Regularly 73 69 77 Occasionally 20 16 24 Student subgroups by which districts dtsaggregated nontest measures (percent) 4,850 Socioeconomic status 22 18 26 Ethnrcrty 29 24 34 Sex 33 28 38 Prior achievement level 76 72 81 (continued) Page 62 GAO/HBD84132BR Effective Schools Programa Appendix IV Estimatea and Gmfldence Intervals for selected Questionnaire Responws Universe or Estimated range’ school Lower Variable districts Estimate limit ?I% Percent of children on free or reduced- price lunch served by ESPs in districts that do not disaggregate achievement data by student socioeconomic status (oercent) 4,637 0 3 0 5 l-10 20 16 25 11-20 23 18 27 21-30 19 15 24 31-40 14 10 18 41-50 7 5 10 51-60 5 3 7 61-70 3 1 5 71-80 3 1 4 81-90 2 0 3 b c c 91-100 %omputed at the 95.percent level of statistical conftdence; e,g., we are 95 percent confident that the true proportlon of school dlstrlcts IS between the ranges specified bLess than one percent ‘Not applicable Page 53 GAO/BRDB9-132BR Effective Schools Programs Amendix 1’ Effective Schools Programs in the Fifty Largest Cities (School Year 1987-88) Criteria’ Criteria Cityb 1 2 3 1 2 3 New York, NY Yes Yes Yes Austin, TX Yes Yes Yes Gs .4naeles. CA Yes Yes Yes Oklahoma Cltv. OK Yes No Noe Chicago, IL Yes Yes Yes Kansas City, MO Yes Yes Yes Houston. TX Yes Yes Yes Fort Worth, TX Yes Yes Yes Philadelphia PA Yes Yes No St Louis, MO Yes Yes Yes Detroit Ml Yes Yes Yes Atlanta. GA d d d San Dleqo, CA Yes Yes Yes Lona Beach, CA Yes Yes No Dallas, TX Yes Yes No Portland, OR Yes Yes Yes San Antonlo, CA c c c Pittsburgh, PA Yes Yes Yes Phoenix. AZ Yes Yes Yes Miami. FL Yes Yes Yes Baltimore. MD Yes Yes Yes Tulsa, OK Yes Yes No San Francisco, CA Yes Yes Yes Honolu%. HI Yes Yes Yes lndlanapolts, IN Yes Yes Yes Clnclnnatl, OH No f f San Jose. CA d d d Albuquerque, NM Yes Yes No Memphis. TN d d a Tucson, AZ d d d WashIngton. D.C Yes Yes Yes Oakland, CA Yes Yes No Jacksonville, FL Yes Yes Yes Mlnneapolls, MN d d d Milwaukee WI I f Yes Yes Yes Charlotte, NC No Boston MA c c c Omaha, NE Yes Yes Yes Columbus. OH -.____ Yes No Noe Toledo, OH Yes No Noe New Orleans, LA Yes Yes Yes VIralma Beach, VA C C C Cleveland, OH Yes Yes Yes Buffalo, NY Yes Yes No ~_...___ Denver, CO Yes Yes Yes Sacramento. CA Yes Yes Yes El Paso TX Yes -___Yes No Newark, NJ Yes Yes No Seattle. WA d d d ____ -- Nashville TN Yes Yes Yes aThese crlterla are shown In figure 10 bCltles are presented tn order of we “School dlstrlct Intends to Implement program during school year 1988-89 or 1989-90 “No response to questionnaire received eDlstrlct dlsaggregates academic achievement data by student socloeconomlc status or ethnlclty ‘Not applicable Page 54 GAO/HID-89-132BR Effective Schools Programs Appendix \‘I Examples of SEA Assistance to School ‘Districts and Schools for Effective SchoolsPugyams -- SEA Kind of Assistance Alaska Staff training Arkansas Staff trarning California Developed ESP model: provrded staff training, technical assistance, an’ d financial support Colorado Technical assistance Connecticut Developed ESP model: provided staff training and technical assrstancc ? Delaware Technical assrstance and financial support lllinors Technical assistance lndrana Staff training, technical assrstance, and financial support Kentucky Staff training, technical assrstance. and financial support Louislana Financial support Massachusetts Financial support - Maryland Developed ESP model; provided staff training and financial __. support Mrchrgan Staff training and technical assistance Minnesota ___-- Developed ESP model: provided staff training MIssour Staff training New Hampshire - Staff training and technical assrstance New Jersey Staff training, technrcal assistance, and financial support Nevada Developed ESP model; provided staff trarnrng and technrcal assistance New York Staff training and technical assistance 6hro Staff training, technical assrstance, and frnancral support Oregon - Staff training and financial support Pennsylvanra Technrcal assistance Rhode Island Staff training and techntcal assistance South Carolrna Developed ESP model. provided staff trarnrng and technrcal assiistance South Dakota Staff trarnrng and technrcal assistance Texas Techntcal assrstance Utah Staff training and technical assrstance Vrrginia Technical assrstance and financial support ._____ -__ Vermont Technical assistance Wisconsin Financral support West Virginia Staff trarnrng and technical assstance Page 55 GAO/HRD89-132BR Effwtive Schools Programs Appendix 1’11 Tables SuepynrtingFigures in Report Text and Supplementary Tables Table VII.1: Extent of ESPs Mese!ting Criterion 1 in U.S. Districts (Da ta for Fig.9) School districts Number Percent Had ESP In school year 1987-88 6,509 41 Planning to implement an ESP tn school year 1988-89 or 1989-90 2,597 17 Not olannlna to lmolement an ESP 6.594 42 Total 15,700 100 Table Vll.2: School Years ESPs Fi rst Implemented (Data for Fig 10) Percent of school School vears districts 1979-80 and before 5 1980-82 4 1982-84 8 1984-86 25 1986-88 57 Table VII.3 Numlaer of School Distric:ts and Schools With ESPs by Program Criteria Districts Schools - Criteria (Data for t-ig 11) 1 6,509 38,285 2 4,304 27,431 3 2,105 17.794 Table Vll.4: Use of School Teams and Written Plans Percent of school districts with ESPs Used School Teams 89 Used Written Plans 71 Used Both Teams and Plans 66 Table Vll.5: Grade Llrvel of Schools With ESPs (Data for Fig 13) Average percent of schools in districts with ESPs Schools with Grade level ESPs Schools Elementary 53 52 Intermediate 18 18 High School 20 ..~.~ 21 Other Grade Levels 10 10 l’age 56 GAO/HRD89-132BR Effective Schools Programs Appendix VII Tables Supporting Fires in Report Text and Supplementary Tables Table Vll.6: Percent of Children on Free --- or Reduced-Price Lunch Sewed by ESPs Percent of school (Data for Fig 14) Percent of children districts with ESPS 0 -~ 2 l-10 17 11-20 -7 21-30 18 31-40 17 41-50 8 -- 51-60 6 61-70 4 71-80 3 81-90 2 91-100 1 Table Vll.7: Student Enrollment in -m Districts With ESPs and All U.S. Districts Percerrt of school districts _- (Data for Fig 15) Al I 1I.S. Enrollment With ESPs disitric:ts 10,000 + 8 4 ..___- 2,500 9,999 ~___- 29 lL5 1 - 2.499 6i! 76‘ Table Vll.8: Geographic Area of School Districts With ESPs and All U.S. Districts Perccrnt of school districts (Data for Fig 16) All U.S. Geographic area .~.- With ESPs districts Primarily central city of MSA 5- 4 MSA -- but not prlmarlly central city 41 36 Outside of MSA 54 60 Page 57 GAO4IRD89-132BR Effective Schools Programs Appendix VII Tables Supporting Figures in Report Text and Supplementary Tables - Table Vll.9: School Characteristics Cited by District13 by Program Criteria (Data for Percent of districts by Frg 17) criteria School characteristic cited 1 2 3 Providing strong instructional leadership 92 92 96 Raising staff expectations for student achrevement 92 92 94 Attarnrng a broadly understood instructional focus 90 93 97 Monitonng student achrevement to evaluate program success 88 90 91 Emohasizina basic-skills acaursition 89 92 96 Promoting staff collaboration and collegialrty 86 89 94 Improving academic learning trme 83 84 89 Improving parental support 81 85 92 Attainina a safe and orderlv school climate 76 80 88 Tab1 e Vll.10: School District Involvement in E SPs (Data for Frg.18) Percent of school districts with ESPs Drstncts in which all schools wrth ESPs were required to have them 46 Districts In which some schools with ESPs were required to have them 4 Districts In which all schools wrth ESPs voluntarily chose to have them 50 Table VII.11: Source of External Assistance to Districts With ESPs (Data Percent of school for Fig 20) districts that received Source external assistance SEA 64 University 31 Educational Laboratory 36 Other 31 Table Vll.12: Measures of rlchievement Districts Used to Evaluate Results of Percent of school ESPs (Data for Fig.24 and Related Achievement measure districts with ESPs Informatron) Achievement tests 83 Nontest measures 75 Both achievement test and nontest measures 66 Achrevement tests only 17 Nontest measures only 9 Rio achrevement tests or nontest measures used 8 Page 58 GAO/HRKM9-132BR Effective Schools Programs Appendix VII Tables SupportIng Figures in Report Text and Supplementary Tables Table Vll.13: Student Subgroups by Which Districts Disaggregated Percent of School Districts with ESPs Achievement Test Results (Data for Analyzed Analyzed Flg.25) Student SubarouD reaularlv occasionally Socioeconomic status 12 21 Ethnicity 9 18 Sex 9 18 Prior achievement level 73 20 Table Vll.14: Student Subgroups by Which Districts Disaggregated Nontest Percent of school Measures (Data for Flg.26) Student subgroup districts with ESPs Socioeconomic status 22 Ethnlcity 29 Pnor achievement level 76 Table VII.15 Percent of Children on Free or Reduced-Price Lunch Sewed by ESPs Percent of school in Districts That Do Not Disaggregate Percent of children districts with ESPs Achievement Data by Student 0 3 Socioeconomic Status (Data for Fig. 27) .~ l-10 20 21-30 19 31-40 14 41-50 7 51-60 - _____~. ___~... 5 61-70 ____. 3 71-80 3 81-90 2 91-100 a aLess than 1 percent Page 59 GAO/HRlM9-132BR Effective Schools Programs Appendix VIII Major Contributors to This Report William J. Gainer, Director, Education and Employment Issues, Human Resources (202) 275-5365 Division, Fred E. Yohey, Jr, Assistant Director Washington, DC. Deborah R. Eisenberg, Assignment Manager Richard J. Wenning, Evaluator-in-Charge Joanne R. Frankel, Technical Advisor C. Robert Deroy, Evaluator (Computer Science) Michael J. O’Dell, Social Science Analyst (206096) Page 60 GAO ‘-89.132BR Effective Schools Programs Requests for copies of GAO reports should be sent to: U.S. General Accounting Office Post Office Box 6015 Gaithersburg, Maryland 20877 Telephone 202-275-6241 The first five copies of each report are free. Additional copies are $2.00 each. There is a 25% discount on orders for 100 or more copies mailed to a single address. Orders must be prepaid by cash or by check or money order made out to the Superintendent of Documents. uuited states General Accounting Office ‘aid Wmhington, D.C. 20548 I Uffkial Business Pen&y for Private Use $300 ,
Effective Schools Programs: Their Extent and Characteristics
Published by the Government Accountability Office on 1989-09-13.
Below is a raw (and likely hideous) rendition of the original report. (PDF)