United States General Accounting Office GAO Testimony Before the Education Task Force, Committee on the Budget, United States Senate For Release on Delivery Expected at 9:30 a.m. Thursday, November 6, 1997 FEDERAL EDUCATION FUNDING Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Statement of Carlotta C. Joyner, Director Education and Employment Issues Health, Education, and Human Services Division GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Mr. Chairman and Members of the Education Task Force: I am pleased to be here today to discuss federal support for education, concerns raised by the multiple programs dispersed through over 30 agencies, and the lack of data about the programs and their impact. Hundreds of federal education programs—the specific number differs depending on how education is defined and who is counting the programs—are administered by more than 30 federal agencies. Although billions of federal dollars support education, state and local governments and the private sector spend even more. For example, federal spending for public elementary and secondary education is only about 7 percent of all funding for kindergarten through high school (K-12) education. State and local sources provide 47 and 46 percent of the funding, respectively. Figure 1: Federal Share of Public K-12 Education Costs Is Small School Year 1994-1995 Local and Intermediate Sources 46.4% Federal 6.8% State 46.8% My testimony today is based on work we have done over several years and on a recent analysis by the Department of Education’s National Center for Education Statistics (NCES).1 (A list of related GAO products appears at the 1 Federal Support for Education: Fiscal Years 1980 to 1997, Department of Education, NCES (Washington, D.C.: Sept. 1997). Page 1 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns end of this testimony.) I will focus on (1) the amount and complexity of federal support for education; (2) additional planning, implementation, and evaluative information needed by agencies and the Congress on federal education programs; and (3) some of the challenges of obtaining more and better information. In summary, billions of federal education dollars are distributed through hundreds of programs and more than 30 agencies. Agencies and the Congress need information to plan, implement, and evaluate these programs. Moreover, to gauge and ensure the success of these programs, the Congress and agencies need several kinds of information. First, they need to know which specific program approaches or models are most effective and the circumstances in which they are effective. They also need to know if individual programs are working nationwide. In addition, they need to be able to look across all programs that are designed to help a given target group to see if individual programs are working efficiently together and whether the federal effort is working effectively overall. We believe closely examining these multiple education programs is definitely needed. The current situation has created the potential for inefficient service and reduced overall effectiveness. Basic information about programs and program results is lacking, and we face many challenges in obtaining this important information. The Government Performance and Results Act of 1993 (GPRA) holds promise as a tool to help agencies manage for results, coordinate their efforts with other agencies, and obtain the information they need to plan and implement programs and evaluate program results. Overall federal support for education includes not only federal funds, but Billions of Federal also nonfederal funds associated with federal legislation. More than 30 Education Dollars departments or agencies administer federal education dollars, although Distributed Through the Department of Education administers the most, accounting for about 43 percent. Of the about $73 billion appropriated to support education, Hundreds of half supports elementary and secondary education. Overall, six program Programs and areas account for almost two-thirds of all budgeted education funding. Many departments and programs may target funds to the same target Multiple Agencies groups, such as poor children. Although some coordination takes place and some programs have been consolidated, much more needs to be done to coordinate the multiple education programs scattered throughout the federal government. Page 2 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Distribution of Federal NCES estimates federal support for education, excluding tax expenditures, Support for Education as approximately $100.5 billion in fiscal year 1997. This figure is an estimate of the value of the assistance to the recipients—not the cost to the government. NCES describes this support as falling into two main categories: funds appropriated by the Congress (on-budget) and a combination of what NCES calls “off-budget”2 funds and nonfederal funds generated by federal legislation. Appropriated funds include items such as grants, federal matching funds, and the administration and subsidy costs for direct and guaranteed student loans. Off-budget funds are the portion of direct federal loans anticipated to be repaid. Nonfederal funds generated by federal legislation include nonfederal (generally state or local) funds provided to obtain federal matching funds and capital provided by private lenders for education loans. According to NCES, in fiscal year 1997, appropriated funds constituted approximately three- quarters of the total: $73.1 billion. Multiple Departments and To ensure that all Americans have equal access to educational Programs opportunities, the federal government often targets its education funds to groups, such as poor children, that for various reasons have not had equal access to educational opportunities. The government may also target funds to ensure that all children have access to vital resources—such as well-trained teachers and technology. These concerns have helped disperse federal education programs to over 30 departments or agencies. The Department of Education spends the most, accounting for about 43 percent of appropriations or an estimated $31 billion in fiscal year 1997. (See fig. 2.) The Department of Health and Human Services (HHS) spends the next largest amount, with about 18 percent or an estimated $13 billion. Over half of this amount ($7.1 billion) funded research; another $4 billion funded the Head Start program. Other Departments with federal education dollars include the Departments of Agriculture, Labor, and Defense, with 13, 6, and 5 percent, respectively. The remaining 15 percent is spent by more than 30 additional departments or agencies. 2 The term “off-budget” as used by NCES differs from the technical definition of the term. Off-budget actually refers to transactions that belong on budget because they are a government cost but that are required by law to be excluded from the budget. The transactions associated with direct loans that NCES describes as off budget would more appropriately be described as nonbudgetary. Page 3 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Figure 2: Many Federal Departments Fund Education Programs Fiscal Year 1997 HHS ($13.1 Billion) 17.9% Agriculture ($9.7 Billion) 13.2% Labor ($4.6 Billion) 6.3% Defense ($3.7 Billion) 5.1% Over 30 Other Departments or Agencies ($10.9 Billion) 15.0% Education ($31.1 Billion) 42.6% Six program areas account for almost two-thirds of all on-budget education funding. As figure 3 shows, the child nutrition programs at the Department of Agriculture account for the largest percentage of funding—16 percent. The two programs accounting for the next largest percentages of funding are Department of Education programs: title I for disadvantaged preschool, elementary, and secondary children, with about 14 percent, and Pell Grants for postsecondary education for low-income students, with about 12 percent. The Department of Labor runs other major programs (Job Corps and other job training programs) as do HHS and the Department of Defense. Page 4 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Figure 3: Six Program Areas Account for Almost Two-Thirds of All Budgeted Education Dollars Fiscal Year 1997 Child Nutrition Programs (Agriculture) 16% Title I (Education) 13.8% Pell Grants (Education) 11.9% Training Programs (Labor) 8.7% Head Start (HHS) 7.7% Special Education (Education) 6.5% Other 35.4% Figure 4 shows the fiscal year 1997 funding for the six largest program activities. Funding ranges from $8.3 billion for child nutrition programs to $3.4 billion for special education activities. Page 5 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Figure 4: Six Largest Education Program Areas, Fiscal Year 1997 Fiscal Year 1997 9 Billions of Dollars 8.3 8 7.2 7 6.2 6 5 4.5 4.0 4 3.4 3 2 1 0 (A rog tion ltu s re) n) n) (La ms r) ) ca n n) HS gr ram du tio bo tio tio tio ra (H P tri (E uca ca ca og icu Nu art du du Ed Pr St ild I (E (E ng ial Ch ad ts ini ec le ran He Tit Tra Sp ll G Pe Elementary and secondary education programs account for half of all budgeted federal education dollars. (See fig. 5.) In addition, the federal government provides funds for postsecondary education (generally as grants and loan guarantees), research (through such Departments as HHS, Energy, and Defense, along with the National Science Foundation), and other activities such as rehabilitative services. Page 6 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Figure 5: Elementary and Secondary Education: Half of On-Budget Education Dollars Fiscal Year 1997 Other ($5.1 Billion) 7.0% Research ($15.9 Billion) 21.8% Elementary and Secondary ($36.6 Billion) 50.1% Postsecondary ($15.4 Billion) 21.1% Many Agencies and Federal funds are generally targeted to specific groups. However, many Programs Target Specific education programs administered by separate agencies may target any Groups single group. Although we have no comprehensive figures on the number of programs targeted to different groups, figure 6 shows the number of programs in various agencies targeted to three specific groups—young children, at-risk and delinquent youth, and teachers. Page 7 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Figure 6: Three Target Groups Served by Multiple Programs and Agencies Department of Department of Department of Department of Department of National Department of Agriculture Education Health and Interior Treasury Endowment Justice Human Services for the Arts 1 34 59 28 4 4 2 1 7 60 22 Department of Department of 6 9 Labor Housing and 6 8 Urban Development 4 6 7 2 ACTION 1 Department of Energy 3 Teachers At-Risk and Young Children State Justice (FY1993) Delinquent Youth (FY1992 & 1993) 1 Institute 1 (FY1996) Environmental 2 Protection Agency Corporation for 6 National and 3 Community Service 1 National Aeronautic and Space Appalachian 1 Administration Regional 1 3 Commission 9 4 1 1 1 3 7 National President's National Science General Services Department of Small Business Department of Foundation for Arts Crime Prevention Foundation Administration Transportation Administration Defense and Humanities Council Note: Circled numbers indicate number of programs. In September 1997, we reported that the federal government funded a wide array of programs dedicated to at-risk and delinquent youth.3 More specifically, 15 federal departments and agencies administered 127 at-risk and delinquent youth programs in fiscal year 1996. HHS and the Departments of Justice, Labor, and Education administered 98 programs— about 77 percent of all programs. HHS and Justice administered the most—59 and 22, respectively—but Department of Labor programs received the most money—$2.2 billion. About 43 percent of the funded programs received at least $15 million each. Many programs, often located in different federal departments and agencies, appear to fund similar 3 See At-Risk and Delinquent Youth: Fiscal Year 1996 Programs (GAO/HEHS-97-211R, Sept. 2, 1997). Page 8 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns services. For example, in 1996, 47 federal programs provided substance abuse prevention, 20 provided substance abuse treatment, and 57 provided violence prevention. Thirteen federal departments and agencies administered these programs and received about $2.3 billion.4 In addition, the same department or agency administered many programs providing similar services. Justice, for example, had nine programs providing substance abuse prevention services to youth in 1996. Furthermore, many individual programs funded multiple services: about 63 percent of the programs funded four or more services each in 1996, according to our review. We also examined programs that provide teacher training. For this target group, multiple federal programs exist in a number of federal agencies. For example, the federal government funded at least 86 teacher training programs in fiscal year 1993 in nine federal agencies and offices. For the 42 programs for which data were available, Department officials reported that over $280 million was obligated in fiscal year 1993.5 Similarly, in fiscal years 1992 and 1993, the government funded over 90 early childhood programs in 11 federal agencies and 20 offices, according to our review. Our analysis showed that one disadvantaged child could have possibly been eligible for as many as 13 programs. Many programs, however, reported serving only a portion of their target population and maintained long waiting lists.6 Secretary of Education Riley testified recently before this Task Force that the Department of Education has made progress in both eliminating low-priority programs and consolidating similar programs. He noted, for example, that the reauthorization of the Individuals With Disabilities Education Act reduced the number of programs from 14 to 6. In addition, the Department has proposed eliminating or consolidating over 40 programs as part of the reauthorization of the Higher Education Act. The multiple education programs scattered throughout the federal government have created the potential for inefficient service as well as 4 This does not include programs in the armed services in the Department of Defense. The services— Air Force, Army, Marine Corps, National Guard, and Navy—administered seven programs totaling $48.8 million. Of those, four programs provided substance abuse prevention, and all of them provided violence prevention services. None of the programs provided substance abuse treatment. 5 Multiple Teacher Training Programs: Information on Budgets, Services, and Target Groups (GAO/HEHS-95-71FS, Feb. 22, 1995). 6 Early Childhood Programs: Multiple Programs and Overlapping Target Groups (GAO/HEHS-95-4FS, Oct. 31, 1994). Page 9 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns difficulty for those trying to access the most appropriate services and funding sources. Federal programs that contribute to similar results should be closely coordinated to ensure that goals are consistent and, as appropriate, program efforts are mutually reinforcing.7 Uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort.8 The large numbers of programs and agencies supporting education More Information Is activities and target groups make management and evaluation information Needed About the critical to the Congress and agency officials. Information about the federal Federal education effort is needed by many different decisionmakers, for different reasons, at different times, and at different levels of detail. Much of that Pre-Kindergarten-12 information, however, is not currently available. Education Effort and Its Impact Kinds of Information To efficiently and effectively operate, manage, and oversee programs and Needed activities, agencies need reliable, timely program performance and cost information and the analytic capacity to use that information. For example, agencies need to have reliable data during their planning efforts to set realistic goals and later, as programs are being implemented, to gauge their progress toward reaching those goals. In addition, in combination with an agency’s performance measurement system, a strong program evaluation capacity is needed to provide feedback on how well an agency’s activities and programs contributed to reaching agency goals. Systematically evaluating a program’s implementation can also provide important information about the program’s success or lack thereof and suggest ways to improve it.9 Moreover, to gauge and maximize success of federal efforts, congressional and agency officials and decisionmakers need many kinds of information. First, agencies and the Congress need to know specific models that work and the circumstances in which they are effective, such as specific classroom activities for preventing substance abuse. Second, the Congress and agencies also need to know if the program, such as the activities funded by the Safe and Drug-Free Schools and Community Act, is working 7 Managing for Results: Using the Results Act to Address Mission Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997). 8 The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 9 Managing for Results: Building on Agencies’ Strategic Plans to Improve Federal Management (GAO/T-GGD/AIMD-98-29, Oct. 30, 1997). Page 10 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns nationwide. Finally, the Congress needs the ability to look across all programs designed to help a given target group to assess how the programs are working together and whether the overall federal effort is accomplishing a mission such as preventing substance abuse among youths. In addition, for specific oversight purposes, congressional decisionmakers sometimes want specific kinds of information. For example, this Task Force has indicated that two types of information would be particularly useful to its mission: knowing which federal education programs target which groups and knowing what characterizes successful programs. Areas in Which Some Some information is available about preK-12 programs that do not appear to Information Exists be achieving the desired results and others that appear to be successful. Secretary Riley, for example, has testified that the Department will be doing more to disseminate the latest information on what works in education. Our clearest evidence about a lack of positive effect from federal expenditures comes from one of the largest programs: title I. Title I of the Elementary and Secondary Education Act is the largest federal elementary and secondary education grant program. It has received much attention recently because of an Education Department report showing that, overall, title I programs do not ultimately reduce the effect of poverty on a student’s achievement.10 For example, children in high-poverty schools began school academically behind their peers in low-poverty schools and could not close this gap as they progressed through school. In addition, when assessed according to high academic standards, most title I students failed to exhibit the reading and mathematics skills expected for their respective grade levels. The study concluded that students in high-poverty schools were the least able to demonstrate the expected levels of academic proficiency. Many of our studies sought to determine what is working and identify information available or needed to make better determinations about program results. For example, we have looked at promising practices or 10 Prospects: Final Report on Student Outcomes, Abt Associates, Inc. (Cambridge, Mass.: Apr. 1997). The study notes, however, that although it was not able to discern a compensatory effect of chapter 1, this is not really an indication of program failure. Without chapter 1 services, students may have fallen further behind. (The program now called title I was called chapter 1 from the early 1980s through 1994.) Page 11 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns strategies in several areas: school violence,11 substance abuse prevention,12 and school-to-work transition.13 In 1995, we also prepared an overview of successful and unsuccessful practices in schools and workplaces.14 Our reviews identified several important program characteristics: strong program leadership, linkages between the program and the community, and a clear and comprehensive approach. The Department of Education also has contracts for evaluating what works. For example, the Prospects study—in addition to providing the data on the overall limited effect of title I—analyzed the five high- performing, high-poverty schools in its sample of 400 schools. Although the number of schools is too small for conclusive generalizations, the study described the characteristics of these schools as “food for thought” for future research on successful programs. These schools had an experienced principal; low teacher and pupil turnover; an emphasis on schoolwide efforts that seek to raise the achievement of every student; a greater use of tracking by student ability; a balanced emphasis on remedial and higher order thinking in classroom involvement; and higher parent support and expectations than low-performing, high-poverty schools. Similarly, the Department’s Special Strategies study of 10 promising alternatives for title I practices, found two—the Success for All and Comer School Development programs—to be especially successful approaches. Overall, the study concluded that “some programs, well implemented, appear to help students make dramatic academic progress; that pursuing schoolwide change may be well worth the effort; that intensive early intervention may yet be the best bet; and that after a third of a century of research on school change, we still have not provided adequate human and fiscal resources, appropriately targeted, to make large-scale program 11 School Safety: Promising Initiatives for Addressing School Violence (GAO/HEHS-95-106, Apr. 25, 1995). 12 Drug Control: Observations on Elements of the Federal Drug Control Strategy (GAO/GGD-97-42, Mar. 14, 1997). 13 Transition From School to Work: States Are Developing New Strategies to Prepare Students for Jobs (GAO/HRD-93-139, Sept. 7, 1993). 14 GAO/PEMD-95-28, Aug. 31, 1995. Page 12 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns improvements a reliably consistent reality in schools serving students placed at risk.”15, 16 Significant Information Significant information gaps exist, however, about both programs and Gaps their outcomes. Currently, no central source of information exists about all the programs providing services to the same target groups among different agencies or about those providing a similar service to several target groups. Instead, we have had to conduct the specific analyses previously described for at-risk and delinquent youth, young children, and teachers—as well as others—to obtain this information. Moreover, in our evaluations of specific programs—some of which get billions of federal dollars each year—the most basic information is lacking. For example, our study of the Safe and Drug-Free Schools Program revealed that the program has no centralized information about what specific services the funds pay for—much less whether the money is being spent effectively. In our ongoing work on Head Start, we found that no list of Head Start classrooms and their locations existed. A second important theme from our report on successful and unsuccessful practices was that few evaluations of successful strategies exist, and many of the existing evaluations lack the methodological rigor needed to determine effectiveness.17 We further documented this theme in our review of Head Start evaluations.18 Since Head Start’s inception in 1965, federal funding for the program has increased significantly. Since 1990, Head Start funding has more than doubled—increasing from $1.5 billion in fiscal year 1990 to almost $4 billion in fiscal year 1997. During this period, Head Start also received additional federal funds to, among other things, increase participation and improve program quality. Yet, little research has focused on program impact, and the body of Head Start research available is inadequate for use in drawing conclusions about the impact of the current Head Start program. We do not know what is working and what is not in today’s programs—the early research on Head Start, conducted over 20 years ago, may no longer apply to today’s program because of program 15 Urban and Suburban/Rural Special Strategies for Educating Disadvantaged Children: Findings and Policy Implications of a Longitudinal Study, Department of Education (Washington, D.C.: Apr. 1997). 16 The latter point has recently been reinforced by an analysis of education productivity. See David W. Grissmer, Education Productivity, Washington, D.C., Council for Educational Development and Research (Washington, D.C.: 1997). 17 GAO/PEMD-95-28, Aug. 31, 1995. 18 Head Start: Research Provides Little Information on Impact of Current Program (GAO/HEHS-97-59, Apr. 15, 1997). Page 13 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns changes and changes in the population served. We have recommended that HHS include in its research plan an assessment of the impact of regular Head Start programs. Although the Department believes that clear evidence exists of the positive impacts of Head Start services, it does have plans to evaluate the feasibility of conducting such studies. More promising, but still incomplete, is the information available for Safe and Drug-Free Schools programs.19 Information on effectiveness and impact has not been collected, although overall evaluations of the Safe and Drug-Free Schools program have not been completed. However, Education’s evaluative activities focus on broader aspects of program implementation and not the effectiveness of all Safe and Drug-Free Schools programs nationwide. Moreover, the lack of uniform information requirements on program activities and effectiveness may create a problem for federal oversight. If (1) process information is critical for program, agency, and interagency Challenges in management of federal elementary and secondary programs, and Obtaining Important (2) outcome and impact information is needed to assess results and focus Information efforts on what works, why is information not readily available? The challenges to collecting that information include • competing priorities—such as reducing paperwork and regulatory burden and promoting flexibility in program implementation—that restrict data collection and evaluation activities; • the cost of data collection; • the secondary role of education in many programs; • the difficulty of obtaining impact evaluation information (under any circumstances); • the special challenge to assessing overall effects on federal efforts involving multiple federal programs in multiple agencies; and • until recently, a lack of focus on results and accountability. Competing Priorities Limit The goal of having enough information for accountability and program Data Collection Activities management is always competing with the goals of providing local agencies with the flexibility they need to meet local needs and lessening regulatory burden. For example, the Paperwork Reduction Act has given the Office of Management and Budget (OMB) the responsibility of 19 Safe and Drug-Free Schools: Balancing Accountability With State and Local Flexibility (GAO/HEHS-98-3, Oct. 10, 1997). Page 14 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns approving collections of information done by the federal government, whether through questions, surveys, or studies. This can limit the burden on state and local governments and others; however, it can also limit the amount of information collected by the Department of Education. Similarly, the challenge of balancing flexibility and accountability is apparent in efforts to provide certain federal education funds as block grants. Agencies face the challenge of balancing the flexibility block grants afford states to set priorities on the basis of local need with their own need to hold states accountable for achieving federal goals.20 For example, the Safe and Drug-Free Schools program allows a wide range of activities and permits states to define the information they collect on program activities and effectiveness. With no requirement that states use consistent measures, the Department faces a difficult challenge in assembling the triennial state reports to develop a nationwide picture of the program’s effectiveness. One promising strategy as an alternative to traditional block grants is the use of Performance Partnership Grants (PPG). Under PPGs, the states and the federal government negotiate an arrangement that identifies specific objectives and performance measures regarding outcomes and processes. This approach gives the states more control over their funding decisions, while encouraging them to accept greater accountability for results.21 Information Is Costly Obtaining and analyzing information to manage and evaluate programs requires significant resources. For example, the Department of Education’s strategic plan cites the need to improve the quality of performance data on programs and operations and to promote the integration of federal programs with one another as well as with state and local programs. Towards this end, in fiscal year 1997, the Department of Education was appropriated about $400 million for educational research and improvement. Education estimates an additional $367 million was obligated by the Department for information technology for Department operations. 20 For more information on ensuring accountability in block grants, see Block Grants: Issues in Designing Accountability Provisions (GAO/AIMD-95-226, Sept. 1, 1995). 21 See, for example, a description of use of this approach in grants for substance abuse prevention and treatment services in Substance Abuse and Mental Health: Reauthorization Issues Facing the Substance Abuse and Mental Health Services Administration (GAO/T-HEHS-97-135, May 22, 1997). Page 15 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns In addition, evaluation research is costly. For example, in fiscal year 1993, the Department awarded 38 contracts totaling more than $20 million for evaluating elementary and secondary education programs. Contract amounts ranged from $38,000 to fund a program improvement conference to $6.8 million for implementing the chapter 1 longitudinal study (Prospects). But this accounted for only 1 year of this multiyear study: this longitudinal study to assess the impact of significant participation in title I programs on student and young adult outcomes cost about $25 million over a 4-year period.22 The median cost for an evaluation contract was about $180,000 in fiscal year 1993. In our testimony last spring on challenges facing the Department of Education, we noted that the Department needed more information to determine how its programs are working and that additional departmental resources may be needed to manage funds and provide information and technical assistance.23 For example, title I is intended to promote access to and equity in education for low-income students. The Congress modified the program in 1994, strengthening its accountability provisions and encouraging the concentration of funds to serve more disadvantaged children. At this time, however, the Department does not have the information it needs to determine whether the funding is being targeted as intended. Although the Department has asked for $10 million in its fiscal year 1998 budget request to evaluate the impact of title I, it has only just begun a small study of selected school districts to examine targeting to identify any necessary mid-course modifications. The ultimate impact of the 1994 program modifications could be diminished if the funding changes are not implemented as intended. Many Programs Involving Many federal programs involving education have other primary purposes. Education Have Other For example, the Department of Agriculture’s child nutrition program Primary Purposes provides school breakfast and school lunch programs. The Head Start program also emphasizes health and nutrition as well as parenting skills; cognitive development is only one of six program goals. In addition, Safe and Drug-Free Schools Act money can be used to provide comprehensive health education, whose major goals and objectives are broader than just drug and violence prevention. 22 Biennial Evaluation Report, Fiscal Years 1993-1994, Department of Education (Washington D.C.: 1995). 23 Department of Education: Challenges in Promoting Access and Excellence in Education (GAO/T-HEHS-97-99, Mar. 20, 1997). Page 16 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns Impact Evaluation Good evaluative information about program effects is difficult to obtain. Information Is Difficult to Each of the tasks involved—measuring outcomes, ensuring the Obtain consistency and quality of data collected at various sites, establishing the causal connection between outcomes and program activities, and distinguishing the influence of extraneous factors—raises formidable technical or logistical problems. Thus, evaluating program impact generally requires a planned study and, often, considerable time and expense. Program features affect the relative difficulty of getting reliable impact information. The more varied the program activities and the less direct the connection between the provider and the federal agency, the greater the difficulty of getting comparable, reliable data on clients and services. For example, a federal agency whose own employees deliver a specified service can probably obtain impact data more easily than one that administers grants that states then pass on to several local entities to be used different ways. Also, due to the absence of contrasting comparison groups, it is extremely difficult to estimate the impact of a long-standing program that covers all eligible participants.24 Multiple Federal Programs The sheer number of departments and agencies that spend federal Managed by Different education dollars makes it hard to aggregate existing information among Agencies federal programs for certain issues or target groups. Each program may have its own measures on the federal, state, and local levels. Even for a single program, each state may use different measures (as mentioned earlier regarding the Safe and Drug-Free Schools and Communities Act programs), creating difficult challenges to developing a nationwide picture of the program’s effectiveness. Yet this is just 1 of the 127 programs administered by 15 agencies that target at-risk and delinquent youth. If the Congress wanted to know the overall effectiveness of the federal effort in helping at-risk and delinquent youth, the task would be even more daunting than that the Department of Education faces in developing a nationwide picture of one flexibly administered program. Past Lack of Emphasis on Federally funded programs have historically placed a low priority on Results and Accountability results and accountability. Therefore, until recently, the statutory framework has not been in place to bring a more disciplined approach to federal management and to provide the Congress and agency decisionmakers with vital information for assessing the performance and costs of federal programs. 24 Program Evaluation: Improving the Flow of Information to the Congress (GAO/PEMD-95-1, Jan. 30, 1995). Page 17 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns In recent years, however, governments around the world, including ours, have faced a citizenry that is demanding that governments become more effective and less costly.25 These two demands are driving the move to a performance-based approach to managing public-sector organizations. GPRA is the centerpiece of a statutory framework provided by recent legislation to bring needed discipline to federal agencies’ management activities. Other elements are the expanded Chief Financial Officers Act, the Paperwork Reduction Act of 1995, and the Clinger-Cohen Act of 1996. These laws each responded to a need for accurate, reliable information for executive branch and congressional decision-making. In combination, they provide a framework for developing (1) fully integrated information about an agency’s mission and strategic priorities, (2) performance data for evaluating the achievement of these goals, (3) the relationship of information technology investments to meeting performance goals, and (4) accurate and audited financial information about the costs of meeting the goals. GPRA requires that agencies clearly define their missions, establish long-term strategic goals as well as annual goals linked to them, measure their performance according to the goals they have set, and report on their progress. In addition to ongoing performance monitoring, agencies are also expected to perform discrete evaluations of their programs and to use information obtained from these evaluations to improve their programs. Agencies are also expected to closely coordinate with other federal agencies whose programs contribute to similar results to ensure that goals are consistent and, as appropriate, that program efforts are mutually reinforcing. Each agency was required to submit to OMB and the Congress a strategic plan explaining its mission, long-term goals, and strategies for meeting these goals by September 30, 1997, and the Department of Education did so. Beginning in fiscal year 1999, agencies are to use their strategic plans to prepare annual performance plans. These performance plans are to include annual goals linked to the activities noted in their budget presentations as well as the indicators the agency will use to measure performance according to results-oriented goals. Agencies are subsequently to report each year on the extent to which goals are met, explain why these goals are not met, if needed, and discuss the actions 25 See, for example, Managing for Results: Experiences Abroad Suggest Insights for Federal Management Reform (GAO/GGD-95-120, May 2, 1995); Managing for Results: State Experiences Provide Insights for Federal Management Reforms (GAO/GGD-95-22, Dec. 21, 1994); and Government Reform: Goal-Setting and Performance (GAO/AIMD/GGD-95-130R, Mar. 27, 1995). Page 18 GAO/T-HEHS-98-46 Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns needed to meet any unmet goals. In addition, by early 1998, OMB must submit to the Congress governmentwide performance plans based on agencies’ plans as part of the president’s fiscal 1999 budget. For federal education programs, this shift to a focus on results can help inform decisionmakers about effective program models and the actual activities and characteristics of individual federal programs. GPRA provides an incentive for agency and program personnel to systematically assess their programs and identify and adapt successful practices of similar programs. The act also provides an early warning system for identifying goals and objectives that are not being met so that agency and program staff can replace ineffective practices with effective ones. The act’s emphasis on coordination among similar programs and linking results to funding also provides a way to better understand the overall effect of federal activities and to identify programs that might be abolished, expanded, or consolidated with others. If agencies and OMB use the annual planning process to highlight crosscutting program issues, the individual agency performance plans and the governmentwide performance plan should provide the Congress with the information needed to identify agencies and programs addressing similar missions. Once these programs are identified, the Congress can consider the associated policy, management, and performance implications of crosscutting program issues. This information should also help identify the performance and cost consequences of program fragmentation and the implications of alternative policy and service delivery options. These options, in turn, can lead to decisions about department and agency missions and allocating resources among those missions.26 Achieving the full potential of GPRA is a particularly difficult challenge because of the multiple programs and many departments involved in the federal effort to improve public K-12 education. Meanwhile, this challenge—combined with the current limited data available about the programs and their effectiveness—is precisely why GPRA is needed. It is also why we believe it holds promise to help improve the information available to decisionmakers and, thus, the federal effort in this important area. Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions you or members of the Task Force may have. 26 GAO/T-GGD/AIMD-98-29, Oct. 30, 1997. Page 19 GAO/T-HEHS-98-46 Related GAO Products Managing for Results: Building on Agencies’ Strategic Plans to Improve Federal Management (GAO/T-GGD/AIMD-98-29, Oct. 30, 1997). Safe and Drug-Free Schools: Balancing Accountability With State and Local Flexibility (GAO/HEHS-98-3, Oct. 10, 1997). Education Programs: Information on Major Preschool, Elementary, and Secondary Education Programs (GAO/HEHS-97-210R, Sept. 15, 1997). Education Programs: Information on Major Postsecondary Education, School-to-Work, and Youth Employment Programs (GAO/HEHS-97-212R, Sept. 15, 1997). At-Risk and Delinquent Youth: Fiscal Year 1996 Programs (GAO/HEHS-97-211R, Sept. 2, 1997). Managing for Results: Using the Results Act to Address Mission Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997). Substance Abuse and Violence Prevention: Multiple Youth Programs Raise Questions of Efficiency and Effectiveness (GAO/T-HEHS-97-166, June 24, 1997). The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). Head Start: Research Provides Little Information on Impact of Current Program (GAO/HEHS-97-59, Apr. 15, 1997). Department of Education: Challenges in Promoting Access and Excellence in Education (GAO/T-HEHS-97-99, Mar. 20, 1997). Schools and Workplaces: An Overview of Successful and Unsuccessful Practices (GAO/PEMD-95-28, Aug. 31, 1995). Block Grants: Issues in Designing Accountability Provisions (GAO/AIMD-95-226, Sept. 1, 1995). Multiple Teacher Training Programs: Information on Budgets, Services, and Target Groups (GAO/HEHS-95-71FS, Feb. 22, 1995). (104909) Page 20 GAO/T-HEHS-98-46 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns
Published by the Government Accountability Office on 1997-11-06.
Below is a raw (and likely hideous) rendition of the original report. (PDF)