United States General Accounting Office GAO Report to Congressional Requesters September 1999 ENVIRONMENTAL INFORMATION EPA Is Taking Steps to Improve Information Management, but Challenges Remain GAO/RCED-99-261 United States GAO General Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Development Division B-283313 September 17, 1999 The Honorable Christopher S. Bond Chairman The Honorable Barbara A. Mikulski Ranking Minority Member Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations United States Senate The Honorable James T. Walsh Chairman The Honorable Alan B. Mollohan Ranking Minority Member Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations House of Representatives The need to manage its programs for environmental results substantially increases the Environmental Protection Agency’s (EPA) demand for high-quality environmental information. Such information is also needed to identify and respond to emerging problems before significant damage is done to the environment. Various studies have shown that although much scientific and environmental data have already been collected, many gaps exist, and the data are often difficult to compile because different collection methods have been used. Likewise, much effort is still needed to identify, develop, and reach agreement on a comprehensive set of environmental measures to link EPA’s activities to changes in human health and environmental conditions. Recognizing the long-standing and serious shortcomings in the environmental information needed to manage for results, the EPA Administrator announced plans in October 1998 to create an office with central responsibility for information management, policy, and technology. The efforts to improve information management that preceded the new office, and that are to be absorbed by it, include several agencywide initiatives directed at improving the quality of EPA’s data and the agency’s ability to share data internally and externally. Reflecting congressional interest in EPA’s data management activities, the conference report accompanying the VA, HUD, and Independent Agencies fiscal year 1997 Page 1 GAO/RCED-99-261 EPA Information Management B-283313 appropriations act1 and subsequent discussions with your offices directed us to review (1) recent initiatives designed to help EPA improve the accuracy, completeness, and compatibility of its data; (2) the impact of data gaps and inconsistencies on EPA’s ability to evaluate and report on the results of its programs under the Government Performance and Results Act (the Results Act); and (3) the major management challenges facing EPA’s new central information office. EPA has several data improvement initiatives to obtain the environmental Results in Brief information needed to effectively set priorities, assess progress in achieving goals and objectives, and report on accomplishments in a credible way. These initiatives are specifically aimed at identifying critical gaps in EPA’s environmental data, developing data standards to enable separately designed databases to operate compatibly with one another, and identifying and correcting inaccuracies. While these initiatives are steps in the right direction, they are limited in scope and do not provide the overall strategy needed to ensure the completeness, compatibility, and accuracy of EPA’s environmental data. For example, EPA has not yet identified or evaluated options for filling the agency’s data gaps, has not yet developed a plan detailing how it will standardize the data in many of the agency’s key databases, and has not yet identified the specific actions that the agency and its state partners need to take to ensure the accuracy of environmental data. EPA’s ability to evaluate the outcomes of its programs in terms of changes in the environment is limited by gaps and inconsistencies in the quality of its data. Of the 357 measures of performance that EPA has developed for use during fiscal year 2000 to report its accomplishments under the Results Act, the agency reports that only 71 will reflect environmental outcomes; the other measures will reflect program activities, such as the number of actions taken to enforce environmental laws. EPA program managers acknowledge that additional measures of environmental outcomes are needed and that the agency’s forthcoming information plan will encourage such measures in all program offices and establish milestones for creating them. To meet these milestones, EPA’s program offices will have to overcome (1) difficulties in establishing cause-and-effect relationships between program activities and environmental outcomes, (2) a lack of reliable baseline data against which to measure progress and a more generalized lack of reliable data about the 1 H.R. Rep. No. 104-812, at 70-71 (1996). Page 2 GAO/RCED-99-261 EPA Information Management B-283313 environment, and (3) constraints on the resources for gathering and analyzing the data. Creating a successful central information office from disparate parts of EPA will help the agency to address obstacles to obtaining the data it needs to manage for results. However, establishing a successful office will require appropriate resources and the commitment of senior management. One of the office’s most pressing challenges will be to develop a plan that identifies clear priorities for the office and the resources it will need to successfully lead the agency’s efforts to make significant improvements in information management. Other key challenges for the new office, which have thwarted EPA’s earlier efforts to improve its information management activities, will be (1) obtaining sufficient authority and resources within EPA to address the complex information management issues facing the agency and (2) working more effectively with the states and regulated industries to balance the demand for more data with efforts to reduce the reporting burden. We are recommending that EPA develop an action plan that details the strategy, milestones, and resources that the new information office will require to ensure that EPA’s environmental and regulatory data are sufficiently complete, compatible, and accurate to meet its needs. EPA and the states collect a wealth of environmental data under various Background statutory and regulatory requirements, including reports on air emissions under the Clean Air Act, wastewater discharges under the Clean Water Act, and pollutant levels in drinking water under the Safe Drinking Water Act. However, EPA’s existing approach to data management is outmoded in many ways. It continues to rely heavily on paper-based reporting, and its many separately designed databases are generally not compatible with each other. Consequently, EPA has not been able to aggregate much of the data from the many different databases to present comprehensive information on chemicals, industrial sectors, localities, and environmental conditions because basic data elements are not standardized across these databases. Moreover, important gaps in the data exist. Data obtained from the detailed monitoring of environmental conditions and of human exposures to toxic pollutants are limited, and the human health and ecological effects of many chemical pollutants are not well understood. For example, EPA’s Integrated Risk Information System, which is a database of the potential health effects from chronic exposure to various substances found in the environment, has toxicological data on only one-third of the known hazardous air pollutants. Page 3 GAO/RCED-99-261 EPA Information Management B-283313 Since the announcement of its regulatory reinvention program in March 1995 and the issuance of its strategic plan in September 1997, both of which recognized the need for improvement in the management of information, EPA has begun several initiatives to improve how it collects, manages, and disseminates environmental information. In addition, in October 1998, the EPA Administrator announced an organizational change to create a new office responsible for information management, information policy, and technology stewardship. This office would be responsible for developing and implementing goals, standards, and accountability systems to manage and improve the quality of data used both within the agency and by the public. The office would also have the authority to set and oversee agencywide standards and policies for managing information resources. The new office, which is scheduled to become operational in early October 1999, will consolidate all or parts of the existing Office of Information Resources Management, the Center for Environmental Information and Statistics, and other organizational components, such as the Toxic Release Inventory program. EPA’s recent major initiatives to improve data quality address Recent Initiatives (1) long-standing problems involving gaps in the agency’s environmental Highlight Obstacles data; (2) the need for core, or common, data standards so that data from Facing EPA as It various information systems can be pulled together to present comprehensive information on geographical locations, chemicals, Seeks to Improve industrial sectors, and environmental conditions; and (3) the need for an Data Quality agencywide approach to ensuring the accuracy of EPA’s data—particularly a process for correcting errors in the agency’s databases. While EPA has made progress, each initiative has encountered obstacles that highlight the difficulties facing the agency as it attempts to improve its information management activities. EPA Faces Extensive Gaps For more than 25 years, EPA, the states, and others have collected data on in Information About the the health and environmental effects of a variety of pollutants, Environment, Health environmental conditions, and the compliance of, and enforcement actions taken against, the regulated community. Despite the vast array of Risks, and the Agency’s data in EPA’s information systems, we, the states, regulated entities, and Effectiveness EPA itself have pointed out that the agency does not have much of the information it needs pertaining to environmental conditions and trends (i.e., environmental indicators), the potential human health risks of various pollutants, and the environmental results of EPA’s activities. Page 4 GAO/RCED-99-261 EPA Information Management B-283313 These extensive data gaps are a result both of a lack of fundamental scientific knowledge and of inadequate data collection, according to EPA and others. EPA evaluations have recognized that the agency has numerous and significant gaps in its risk and environmental data. For example, one EPA review concluded that complete data on health effects exist for only 7 percent of the 3,000 most widely used chemicals. Similarly, EPA found that it lacks basic toxicity data for more than one-third of the chemicals produced in large volumes as well as for about two-thirds of the known hazardous air pollutants. Moreover, the environmental data that EPA does have are often fragmented because they were collected under various laws, such as the Clean Air Act or the Safe Drinking Water Act. Consequently, these data are not easily integrated, if they can be integrated at all, to provide environmental information about specific locations or the nation as a whole. One expert outside EPA recently expressed his concern about environmental data gaps: “We don’t have enough information to tell us where we are or where the trends are going. We don’t really know whether air quality, and especially water quality, are really improving or not under current law. As for solid waste, the situation is hopeless. We don’t even know where it is, much less whether it’s getting better or worse.”2 EPA does not yet have a strategy in place for prioritizing its needs for additional data and filling key data gaps. The agency has, however, made some initial efforts to assess its data needs and how it may obtain the needed data. For example, officials from EPA’s Office of Planning, Analysis, and Accountability, which has responsibility for coordinating the agency’s compliance under the Results Act, said they are conducting an analysis to determine whether there are gaps in the data needed to measure the agency’s progress in meeting its strategic objectives. Another effort dealing with the agency’s need for additional data has been led by EPA’s Center for Environmental Information and Statistics, which will become a part of the new information office. In early 1999, the Center completed the first phase of a strategy to identify and address EPA’s data gaps. During this initial screening analysis, staff from the Center and EPA’s program offices identified 26 key environmental problem areas that the agency has committed itself to address through its strategic plan, goals, and objectives. The Center then screened these problem areas for major gaps in data concerning health and ecological toxicity, ambient environmental conditions, and sources of pollution or other environmental 2 Terry Davies, Director, Center for Risk Management, Resources for the Future, quoted in Mary H. Cooper, “The Cleanup’s Next Phase: Setting Environmental Priorities for the 21st Century,” CQ Outlook (June 5, 1999). Page 5 GAO/RCED-99-261 EPA Information Management B-283313 stressors. The eight environmental problem areas in which EPA has the most significant data gaps, according to the Center’s draft screening analysis, are • aquatic ecosystem health—biological stressors (such as the impact of nonnative species), • aquatic ecosystem health—physical alterations (such as erosion), • indoor air, • pesticides—nondietary human health risks (such as exposure of the skin to agricultural pesticides), • air toxics, • pesticides—dietary human health risks, • aquatic ecosystem health—toxics, and • climate change. The second phase of the Center’s effort was to initially focus on two or three environmental areas identified in the first phase and develop recommendations for addressing the highest-priority data gaps in these areas. EPA realized, however, that a strategy for filling data gaps would need to be coordinated with other activities within the agency, such as efforts to reduce the reporting burden. As a result, the Center has not continued its effort to develop this strategy pending its move into the new information office. The creation of a strategy for prioritizing and filling key data gaps will be a part of the development of EPA’s comprehensive information plan, according to officials responsible for the plan. Efforts are under way in EPA to develop the data needed to fill at least some of these gaps. One such effort is the Environmental Monitoring and Assessment Program, which is working with other federal agencies to develop information that the public, scientists, and the Congress can use to evaluate the overall health of the nation’s ecological resources. Another effort, called the High Production Volume Challenge Program, was developed by EPA—in partnership with industry and environmental groups—to make publicly available a complete set of baseline data on the health and environmental effects of each chemical manufactured in, or imported into, the United States in amounts of 1 million or more pounds per year. Companies participating in this program pledge to evaluate the adequacy of existing data for these chemicals and to conduct tests where needed to fill data gaps. The new information office will be responsible for encouraging EPA’s program offices to reach out to other federal agencies, as well as to Page 6 GAO/RCED-99-261 EPA Information Management B-283313 universities, research institutes, and other sources of environmental information, for data that EPA does not collect but that may exist elsewhere. To date, EPA’s efforts to share and obtain data from other agencies, other than the state environmental agencies with which it shares responsibility for implementing federal environmental laws, have been limited. Such efforts have been hampered by technological limitations imposed by the myriad of incompatible information systems in use across the government. The new information office will be responsible for promoting EPA’s efforts to exchange data with other federal agencies. EPA also faces concerns about the reporting burden it may place on states and the regulated community as it seeks to fill data gaps. There is concern within the regulated community and among EPA’s state partners about the possibility that EPA will expand its reporting requirements—which it may have to do to obtain the data it needs. The states are calling for any such expansion to be balanced with a reduction in existing reporting requirements. Moreover, much of the data needed, such as environmental monitoring information, will be expensive to obtain. It will thus be important for EPA to prioritize its needs for additional data, to work with the states and industry to reduce the reporting burden, and to encourage efforts to use data that may already have been collected by other federal agencies or other entities. EPA and the States Have Because most of the information systems that EPA has developed over the Taken Initial Steps to years are not compatible with each other, the agency has not been able to Increase Data integrate important data it has collected about the environment and regulated facilities. EPA has been criticized from both outside and within Compatibility the agency for having developed and maintained “stovepiped” data systems that cannot share the enormous amounts of data gathered. In recent years, EPA has tried to make its data more compatible, and in 1998, the Administrator committed the agency to begin working toward full data integration across programs and across environmental media. Essential to the effort to integrate data from various EPA databases are common data definitions and formats, known as data standards. EPA has indicated that it needs such standards not only to make its data compatible, but also to facilitate some of its other information initiatives. For example, EPA considers data standards as key to reducing the reporting burden on industry and the states because it would permit integrated reporting of information to EPA. Page 7 GAO/RCED-99-261 EPA Information Management B-283313 In recent years, EPA has undertaken several efforts to develop standards for some of the data items in its information systems, most recently as part of the 1998 action plan for the Reinventing Environmental Information (REI) initiative. As part of this initiative, EPA and the states are developing six data standards to be used in 13 of EPA’s major databases. (See app. II for a list of these databases.) The standards being developed will apply common definitions and formats for describing the following items in each database: (1) date, (2) facility identification, (3) industrial classification, (4) location (latitude/longitude), (5) identification of chemical names, and (6) biological taxonomy (i.e., categories for describing plants and animals, such as class, family, and species). EPA considers these standards as a key step in moving to full integration of EPA’s data across its major systems. According to the REI action plan, these six standards will be developed, approved by EPA in partnership with the states, and in use in the 13 designated databases by the end of fiscal year 2003. The current effort to develop data standards is attempting to avoid the limitations of previous efforts, which encountered both technical difficulties in determining what appropriate standards should be and the unwillingness of program offices to adopt the standards. To overcome such problems, the REI effort involves a cooperative interchange among the EPA officials responsible for developing the standards, the EPA program offices that will use the data, and the state environmental agencies to reach consensus. While responsible officials believe that getting “buy-in” from the key collectors and users of the data is essential for the eventual successful adoption of the standards, it has contributed to the time required to launch the standards. According to EPA, as of August 1999, two of the six standards—the common definitions and formats for describing the date and the industrial classification—were ready for implementation pending review by the states. Definitions and formats have been proposed for the other standards, which are in varying stages of development and approval by EPA and review by the states. After agreeing on each standard, the states and EPA will need to agree on the rules for how each standard will be used. Only then will each standard be ready to be used for data being entered in the databases. To meet the implementation date of fiscal year 2003, EPA has set milestones that call for the approval of the definitions and formats for all six standards by the end of 1999. Both EPA officials and representatives Page 8 GAO/RCED-99-261 EPA Information Management B-283313 from the Environmental Council of the States (ECOS),3 which is spearheading the states’ involvement in this effort, have indicated that the fiscal year 2003 milestone for implementing the six standards may be met despite schedule slippages and the complexities of the task. ECOS officials also believe that as long as the states remain full partners in the development of the standards, there is a much greater likelihood that the states as well as EPA will find the standards useful and appropriate and that the initiative will ultimately be successful. The current initiative is limited in terms of the number of standards being developed (six), the number of EPA databases in which the standards will initially be used (13), and the amount of data in those databases (only the new data being entered) that will incorporate the standards. EPA recognizes that its current effort is only a first step toward its goal of full data integration. ECOS officials believe that EPA’s focusing on the six data standards and 13 databases is an appropriate way to begin. ECOS officials also believe, based on the states’ experiences in this area, that the effort should be limited to new data being entered into the databases. According to ECOS officials, previous efforts by states to conform existing data in their systems to new requirements such as data standards have required a prohibitive expenditure of time, expertise, and other resources. The focus of the current standards development effort is primarily on the compatibility of data among EPA’s information systems and those of state environmental agencies, rather than also emphasizing the compatibility of EPA’s data with the data of other federal agencies and nongovernmental sources. EPA’s Science Advisory Board has recommended that EPA do more to link the agency’s databases with federal or other external databases, noting that “answering many health-related questions frequently requires linking environmental data with census, cancer or birth registry data, or other data systems (such as water distribution maps) to determine whether there is a relationship between the environmental measures and health.”4 While EPA officials told us that they recognize the importance of linking EPA’s databases with those of other federal agencies, resource constraints and the lack of statutory coordination requirements have limited their actions in this area to participation in interagency forums and coalitions. They said that they have no specific plans or target dates to initiate more formal efforts. For at least one standard, however, the 3 ECOS is a national nonpartisan, nonprofit association of state and territorial environmental commissioners. 4 The EPA Science Advisory Board provides advice to EPA from scientists outside the agency. Science Advisory Board, Review of the Agency-Wide Quality Management Program, EPA-SAB-EEC-LTR-98-003 (Washington, D.C.: EPA, July 24, 1998). Page 9 GAO/RCED-99-261 EPA Information Management B-283313 current initiative is providing EPA with an opportunity to encourage compatibility with outside data sources. In developing a standard for describing the biological categories of plants and animals, EPA has participated in a joint effort by U.S. and Canadian federal agencies, the states, academic institutions, museums, and nongovernmental organizations to develop and maintain standard information—known as the Integrated Taxonomic Information System. Data accuracy, or the extent to which data are free from significant error, Long-Standing has long been a serious challenge facing EPA. Various reviews that we, EPA, Concerns Persist and others have done have revealed persistent concerns about the About the Accuracy of accuracy of data in many of EPA’s information systems. EPA has not conducted an agencywide assessment of the accuracy of its information EPA’s Data systems. However, in 1998, agency staff reviewed numerous studies of EPA’s data systems and found that these studies suggested variable error rates, with some systems and types of data more prone to error than others. Data used for some purposes may need to be more accurate than in other instances, which is, in part, why EPA has not set an across-the-board target rate for data accuracy. While EPA acknowledges that data errors exist in many of its systems, the agency believes that, in the aggregate, its data are of sufficient quality to support its programmatic and regulatory decisions. Preventing data errors and correcting errors once they have been identified, essential to data accuracy, have proved to be daunting tasks for EPA. For example, in January 1998, an EPA advisory council on information management issues described the difficulty of correcting errors in EPA’s databases: “Once an error is stored in one or more of the Agency’s systems, making corrections to all those systems is an exercise in frustration and futility. There is no simple way to ensure corrections are made to all possible systems.”5 Moreover, efforts to improve the accuracy of EPA’s information systems will need the cooperation of a complicated network of individuals and entities who provide, collect, manage, or use the data. Among such individuals and entities are EPA employees, staff from state—and, in some instances, local—governments, the regulated community, contractors, and private citizens who use data from these systems. 5 National Advisory Council for Environmental Policy and Technology, Managing Information as a Strategic Resource: Final Report and Recommendations of the Information Impacts Committee, EPA 100-R-98-002 (Washington, D.C.: EPA, Jan. 1998). Page 10 GAO/RCED-99-261 EPA Information Management B-283313 Preventing data errors is an issue of both whether accurate data are being generated and whether the integrity of the reported data is maintained as the data flow through the states’ and EPA’s information systems. EPA has an agencywide quality system that was revised in 1998 to expand and clarify requirements for how environmental data are collected and managed.6 All EPA organizational units involved in the use of environmental data are subject to the quality requirements, which include provisions to define and meet data quality needs. EPA’s quality program requires that each entity document its quality system in a quality management plan. Among other things, the quality management plans must discuss the criteria for measuring data quality, describe how the acquired data will be validated and verified, and identify any constraints on data collection. Currently, about 40 EPA organizational units are required to develop and implement quality management plans. In addition, the quality program’s requirements, including the quality management plan, apply to others outside of EPA, such as states, local and tribal governments, and contractors who generate and collect environmental data on behalf of EPA. Although EPA’s Science Advisory Board recently commended the agency for its development of a quality system and for the efforts of EPA’s quality assurance division to champion the need for quality assurance and quality control, the Board has also found that the system’s implementation has been uneven within the agency, increasing the likelihood of problems with data quality and the decisions made based on the data.7 Moreover, the Board reported that more than 75 percent of the states authorized to implement EPA’s environmental programs lack approved quality management plans for all or some of these programs and thus are likely to be generating data of unknown quality. The Board found the situation worrisome as it implied that compliance with EPA’s quality system is unimportant to these states. According to the Board, “Such a state is exposing itself, the reliability of its decisions, and its credibility, to criticisms due to its reliance upon data of unknown quality. The same is true for those agency programs that depend upon those data.” The Board recommended that EPA place its quality system at a higher level within the 6 EPA defines environmental data as any measurements or information that describe environmental processes or conditions or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, such as environmental monitoring; produced from models; and compiled from other sources such as databases or the scientific literature. 7 Science Advisory Board, Review of the Implementation of the Agency-Wide Quality System, EPA-SAB-EEC-LTR-99-002 (Washington, D.C.: EPA, Feb. 25, 1999). Page 11 GAO/RCED-99-261 EPA Information Management B-283313 agency structure to bring more attention and priority to quality assurance and quality control issues. EPA’s Inspector General has also examined EPA’s quality assurance program in terms of its ability to ensure data quality. In a 1998 review of the quality system developed by EPA’s Superfund program, the Inspector General found that EPA managers had not always fully developed and effectively implemented their quality assurance programs to the extent that they could ensure that they obtained data of known and adequate quality.8 The Inspector General raised concerns that similar weaknesses may exist in other EPA program offices. As the Science Advisory Board did, the Inspector General recommended that EPA elevate the responsibility for quality assurance for the agency as a whole to a level at which its manager could be an effective and independent advocate for quality assurance. In addition, the Inspector General recommended that the agency develop a strategy to institutionalize the quality assurance program, improve oversight of the program, develop minimum quality assurance requirements, and report annually on the program’s effectiveness. In April 1998, EPA’s Deputy Administrator called for a strategic action plan for implementing an agencywide approach to ensuring data quality. This plan contained recommendations for preventing and correcting errors. To encourage error prevention, the plan recommended that EPA develop a baseline review of each of its major data systems and establish milestones for improving accuracy over time. It also recommended that EPA develop data standards that would be used both by the states and EPA, arguing that the states would be more diligent in maintaining high-quality data for EPA if they used the same data in the course of running their own programs. To encourage error correction, the plan’s recommendations called for EPA to rely on data users to alert the agency to inaccurate data. Specifically, the plan called for EPA to establish an easy-to-understand guidance system so that users who noticed errors could report discrepancies to EPA. Although the data quality strategic plan was submitted to EPA’s Acting Deputy Administrator in December 1998, the plan has not been adopted, and thus its recommendations have not been implemented. However, EPA’s Acting Deputy Administrator has indicated that the plan will be useful to the new information office. For example, he said that developing an error correction system would be an “important early effort” of the new information office and that this system would build on some of the 8 EPA, Superfund: EPA Had Not Effectively Implemented Its Superfund Quality Assurance Program, Office of Inspector General, E1SKF7-08-0011-8100240 (Washington, D.C.: EPA, Sept. 30, 1998). Page 12 GAO/RCED-99-261 EPA Information Management B-283313 thinking that went into the 1998 plan. The system would rely on data providers and others to bring errors to EPA’s attention and then would ensure the prompt correction of the reported errors. The agency has allocated funds to develop an error correction process, and staff have developed a proposal; however, the proposal has not yet been discussed with program and regional offices and has not been approved by management of the new information office. The Acting Deputy Administrator has also indicated that data quality covers a broader array of issues than those addressed in the 1998 data quality strategic plan and that the new information office will be tasked with addressing these issues. As we pointed out in April 1999, EPA does not yet have a common understanding of what data quality means and how the agency and its state partners can most effectively ensure that the data used for decision-making or disseminated to the public are of high quality.9 To address this, EPA plans to elevate data quality issues in the new information office and to address both how data quality should be defined and how the quality of the agency’s data can be improved. As part of this effort, the agency plans to create a Quality and Information Council, comprising senior executives from across the agency, that will provide strategic direction and advice to the director of the new information office on data quality, information technology investments, and other issues. The information office will also include a “quality staff” that will focus on the design, policy development, and oversight of the agency’s quality program. Much of the current effort by EPA and its state partners to improve EPA’s Success in information management is tied to their initiatives to assess the results of Developing their programs. Spurred by the requirements of the Results Act, EPA has Environmental begun to set goals and measures of its performance that are intended to help the agency, as well as the Congress and the public, assess the Measures Will Be environmental results of the agency’s activities. Under the Results Act, EPA Dependent on Data is required to set long-term and annual goals as well as to measure the results of its programs in an annual report to the Congress. The first such Improvements report, due by the end of March 2000, will cover the agency’s performance in fiscal year 1999. The states and EPA have also been working together to develop mutually agreeable environmental goals and a set of results-oriented “core performance measures” to use in measuring the effectiveness and success 9 Environmental Protection: Status of EPA’s Efforts to Create a Central Information Office (GAO/T-RCED-99-147, Apr. 13, 1999). Page 13 GAO/RCED-99-261 EPA Information Management B-283313 of the states’ implementation of national environmental programs. The core performance measures are a central component of an initiative called the National Environmental Performance Partnership System (NEPPS).10 The goals and measures are to be used to evaluate state programs with a focus on outcomes, such as improvements in water quality, rather than activity measures, such as the number of inspections performed at manufacturing facilities. To date, EPA and the states have made limited progress in developing outcome-oriented performance measures under both the Results Act and NEPPS. EPA has relatively few environmental outcome measures among the annual performance measures it developed for fiscal year 2000 reporting under the Results Act.11 A large majority of the performance measures for fiscal year 2000 reflect the level of program activities, or outputs, such as the number of regulations issued or enforcement actions taken. According to EPA’s categorization, of 357 performance measures associated with these goals, only 71, or about 20 percent, measure program outcomes. According to EPA program managers, the agency has long used activity, or output, measures to manage its programs and will continue to need some such measures even as it incorporates more results-based goals and measures. For example, EPA program managers said that some program activity measures are required by statute and others are necessary for managing the programs. However, in other cases, EPA officials said that they chose their annual goals and measures based on what data they had available. 10 NEPPS was intended, among other things, to give states with strong environmental performance greater flexibility and autonomy in running environmental programs that EPA has delegated to the states. We recently reported on the experiences to date in developing these measures and in implementing the “performance partnership agreements.” Environmental Protection: Collaborative EPA-State Effort Needed to Improve New Performance Partnership System (GAO/RCED-99-171, June 21, 1999). 11 Outcome measures are those expressed in terms of program impacts or human health and environmental changes, rather than the projects to be completed or the number of activities to be performed, which are referred to as outputs. Page 14 GAO/RCED-99-261 EPA Information Management B-283313 Table 1: EPA’s Analysis of the Number and Type of Annual Performance Number of annual Measures for Its Strategic Goals for performance measures Fiscal Year 2000 EPA’s strategic goal Output Outcome Total Goal 1: Clean air 16 17 33 Goal 2: Clean and safe water 74 11 85 Goal 3: Safe food 16 1 17 Goal 4: Preventing pollution and reducing risk in communities, homes, workplaces, and ecosystems 29 13 42 Goal 5: Better waste management, restoration of contaminated sites, and emergency response 39 3 42 Goal 6: Reduction of global and cross-border environmental risks 27 9 36 Goal 7: Expansion of Americans’ right to know about their environment 25 6 31 Goal 8: Sound science, improved understanding of environmental risk and greater innovation to address environmental problems 29 3 32 Goal 9: A credible deterrent to pollution and greater compliance with the law 10 8 18 Goal 10: Effective management 21 0 21 Total 286 71 357 Source: EPA, Office of Planning, Analysis, and Accountability. As indicated by the relatively few environmental outcome measures developed to date, the development of performance measures is proving to be a difficult task for EPA. The challenges of developing outcome measures include (1) the difficulty of linking program activities to environmental outcomes, (2) a lack of baseline data against which to measure progress and a more generalized lack of reliable data about the environment, and (3) resource constraints for gathering and analyzing environmental data.12 Linking program activities to environmental outcomes presents a major technical challenge, as we noted in a 1997 report on the analytical challenges in measuring performance.13 Changes in environmental 12 Such difficulties are similar to those we identified in a recent report on the efforts of EPA and the states to develop core performance measures under NEPPS. Environmental Protection: Collaborative EPA-State Effort Needed to Improve New Performance Partnership System (GAO/RCED-99-171, June 21, 1999). 13 Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997). Page 15 GAO/RCED-99-261 EPA Information Management B-283313 conditions come about as a result of a complex web of factors, including such variables as the weather or economic activity, many of which are out of the control of EPA and its state partners. Likewise, it may take years for some environmental outcomes to occur. Officials from EPA’s Office of Air and Radiation told us, for example, that the outcomes, or environmental results, of air quality programs may not be seen for many years, making it particularly difficult to develop measures that assess annual performance results. Some EPA officials raised concerns about being held accountable under the Results Act for environmental outcomes that are largely out of their control and indicated that they would rather be held accountable for program outputs, over which they have more control. The lack of appropriate environmental data for developing outcome measures is tied to resource limitations and reporting burden. Environmental measures can be costly to develop and use. Moreover, some state officials mentioned that it is difficult to commit resources to the development and implementation of new results-oriented performance measures while still being held responsible for meeting other program requirements. Also, the states have indicated to EPA that they are not willing to collect more data for EPA’s needs if the data are not also needed for managing their own programs and assessing environmental conditions in their states. The states and EPA recently committed themselves to a joint effort to look for opportunities to reduce reporting burden. They also agreed to the principle that the data collected should support their ability to measure programs’ success in a manner that increasingly is based on environmental results. The development of meaningful measures of environmental results is intrinsically linked to the rest of EPA’s key information management concerns—both the data quality issues (such as the need to improve data accuracy, to increase data compatibility, and to reduce data gaps) and the overarching issues facing the new information office (such as the call for a reduction in the reporting burden and the need for effective partnerships with the states). To date, EPA has made only limited progress toward developing outcome measures. However, agency officials responsible for designing EPA’s new information office have stated that developing information on environmental results will be a part of the agency’s major initiative to overhaul how it collects, manages, and disseminates information. EPA has indicated that its forthcoming information plan will articulate the central role that measures of results will play in helping the agency meet its strategic goals and how efforts to develop such measures will be fostered and encouraged. The magnitude of the difficulties involved Page 16 GAO/RCED-99-261 EPA Information Management B-283313 in developing environmental outcome measures makes it likely that EPA will continue to struggle with this issue for a long time and will need to devote a great deal of attention to this effort if it is to make major progress. EPA’s information office will be responsible for improving the quality of EPA’s New data used within EPA and provided to the public and for developing and Information Office implementing the goals, standards, and accountability systems needed to Faces Significant bring about these improvements. To this end, the information office would (1) ensure that the quality of data collected and used by EPA is known and Management appropriate for its intended uses, (2) reduce the burden on the states and Challenges regulated industries of collecting and reporting data, (3) fill significant data gaps, and (4) provide the public with integrated information and statistics on environmental and public health issues. The office will also have the authority to set and oversee agencywide standards and policies for managing information resources, including those governing the purchase and operation of information technology systems. EPA estimates that the new office will be operational in early October 1999. Although the establishment of this office is an important step in improving how EPA collects, manages, and disseminates information, the office will face many challenges. As we reported in April 1999, developing a plan to show how the agency intends to achieve its vision and goals is a pressing need for the new office.14 Among other challenges facing the office are two that have thwarted previous efforts by EPA to improve its information management activities: (1) obtaining sufficient authority and resources to address the complex information management issues facing the agency and (2) working effectively with the states and regulated industries to balance the demand for more data with the efforts to reduce the reporting burden. EPA Is Beginning to EPA has begun work to develop an information strategic plan that it hopes Develop an Information will guide the agency’s information technology management and Strategic Plan investments on a multiyear basis. The new information office will have leadership responsibility for creating this plan. As we, EPA’s Inspector General, and others have pointed out, EPA has long needed—and long lacked—such a plan for guiding its use of and investments in information technology. Moreover, the Clinger-Cohen Act requires that agencies set 14 Environmental Protection: Status of EPA’s Efforts to Create a Central Information Office (GAO/T-RCED-99-147, Apr. 13, 1999). Page 17 GAO/RCED-99-261 EPA Information Management B-283313 goals for improving the effectiveness of their operations through the use of information technology and establish performance indicators to measure how well information technology supports their programs. As we recently reported, despite the agency’s having acknowledged the need to improve data management as a mission-critical problem in its fiscal year 2000 performance plan, EPA did not set goals or timeframes for implementing the information technology management requirements of the Clinger-Cohen Act.15 The primary purpose of EPA’s information strategic plan will be to ensure that the agency’s information technology will support the agency’s efforts to meet its strategic goals as articulated in the strategic plan prepared under the Results Act. As part of this effort, EPA will conduct various analyses to determine its information needs and corresponding investments in the information technology essential to carrying out its mission and achieving its strategic goals. EPA officials responsible for coordinating the development of the information plan recently set preliminary milestones calling for “startup planning and team organization” to begin in August 1999, with the bulk of fiscal year 2000 (beginning in October 1999) being used for assessing the current state of information in the agency and deciding on information priorities for future action. In addition to or as part of this long-range plan to guide EPA’s information priorities and investments, the information office will need a strategy and an action plan that articulates the office’s priorities and the resources needed to accomplish them. In particular, the office will need to articulate its strategy to address the data quality problems—accuracy concerns, lack of data compatibility, and data gaps—discussed in this report. Although such a strategy or action plan is not yet in place, the Administrator has identified 13 projects for the office that will receive early attention. Among these projects, some of which are new while others are ongoing initiatives, are the development of the data standards and the development of the strategic information plan. EPA’s New Information The EPA Administrator and the senior-level officials charged with creating Office Will Need Sufficient the information office acknowledge that the reorganization will raise a Authority and Resources variety of complex information policy and technology issues. To date, the focus has largely been on determining which organizational components 15 Observations on the Environmental Protection Agency’s Fiscal Year 2000 Performance Plan (GAO/RCED-99-237R, July 31, 1999). Page 18 GAO/RCED-99-261 EPA Information Management B-283313 and staff members should be transferred into the office. While such decisions are clearly important, EPA will also need to ensure that the office has sufficient authority and resources to overcome organizational obstacles that hindered previous attempts to adopt agencywide information policies and a strategy for information resources management. As EPA’s Chief Information Officer, the head of the information office will be expected to provide accountability at a senior management level for information technology issues agencywide and to ensure greater accountability for delivering effective information technology systems and services. As we reported in September 1998, EPA has not developed agencywide policies and procedures to govern key aspects of its projects to disseminate information, nor has it developed standards to assess the information’s accuracy and mechanisms to identify and correct errors.16 EPA recognizes the need for such agencywide policies governing information collection, management, and dissemination, and the new office will be responsible for developing them. However, EPA has a nearly 30-year history of operating in a decentralized fashion, with strong program offices that are responsible for implementing different statutes, such as the Clean Air Act and the Clean Water Act, with differing sets of reporting requirements. Moreover, EPA’s program offices have historically developed and managed their own information systems and made their own decisions about disseminating information. For example, program offices have been making their own, sometimes conflicting, decisions about the types of information to be released and the extent of the explanations needed about how information should be interpreted. The new office will need to have the clear authority to develop such procedures and policies and ensure that they are adhered to by the program offices. The office will also need significant resources and expertise to address the information management challenges facing EPA. While the new organizational structure will offer EPA an opportunity to better coordinate and prioritize its information initiatives, the agency will also need to determine whether its current information management resources, including staff expertise, are sufficient for the office to achieve its goals. 16 Environmental Information: Agencywide Policies and Procedures Are Needed for EPA’s Information Dissemination (GAO/RCED-98-245, Sept. 24, 1998). Page 19 GAO/RCED-99-261 EPA Information Management B-283313 EPA Has Begun Working In implementing environmental programs, EPA and the states have With the States and collected a wealth of environmental data under various statutory and Regulated Industries to regulatory authorities. However, EPA needs additional information on environmental conditions and changes over time if it is to identify problem Balance the Need for More areas that are emerging or that need additional regulatory action or other Data With the Efforts to attention. In contrast to the need for more and better data is a call from Reduce the Burden of Data states and regulated industries to reduce the paperwork burden associated Management and with managing and reporting data. Reporting Overall reductions in the reporting burden have typically proven difficult for EPA to achieve. For example, in March 1996, we reported that while EPA was pursuing a paperwork reduction of 20 million hours, its overall paperwork burden was actually increasing because of changes in programs and other factors.17 EPA has continued to undertake a number of activities to reduce the reporting and record keeping burden on regulated entities. However, the Office of Management and Budget has reported that these reductions have not yet been able to offset the growth in the reporting burden resulting from new collections, which are frequently associated with new rules needed to meet EPA’s statutory requirements.18 For fiscal year 1999, EPA has reported that the information collection burden it imposed on the public would be about 120 million hours, a slight increase over fiscal year 1998. The states and regulated industries have indicated that they will look to EPA’s new information office to reduce the burden of reporting requirements. EPA has recently initiated some efforts to reduce the reporting burden. For example, an EPA-state information management work group looking into this issue has proposed an approach to assess environmental data and reporting requirements based on the data’s value compared with the cost of collecting, managing, and reporting it. EPA has announced that in the coming months, its regional offices and the states will be exploring possibilities for reducing paperwork requirements, testing initiatives in consultation with EPA’s program offices, and establishing a clearinghouse of successful initiatives and pilot projects. Given that EPA depends on state regulatory agencies to collect much of the data it needs and to help ensure the quality of that data, EPA acknowledges 17 Environmental Protection: Assessing EPA’s Progress in Paperwork Reduction (GAO/T-RCED-96-107, Mar. 21, 1996). 18 Office of Information and Regulatory Affairs, Information Collection Budget of the United States Government, Fiscal Year 1999, Office of Management and Budget (Washington, D.C.: OMB, Apr. 16, 1999), pp. 4, 14, and 147. Page 20 GAO/RCED-99-261 EPA Information Management B-283313 the need to work in a close partnership with the states on a wide variety of information management activities, including the creation of its information office. Some partnerships have already been created, such as the effort discussed previously on reducing the reporting burden. Similarly, EPA and its state partners in the NEPPS program are determining what the appropriate data would be for developing environmental goals and measures. Representatives of state environmental agencies and ECOS have expressed their ideas and concerns about the role of the new information office and have frequently reminded EPA that they expect to share the responsibility for setting that office’s goals, priorities, and strategies. According to an ECOS official, the states have had more input in the development of the new EPA office than they typically have had in other major policy issues, and the states view this change as an improvement in their relationship with EPA. The EPA Administrator’s decision to implement fundamental improvements Conclusions in the agency’s environmental information is an important step in the right direction. However, additional actions are needed to resolve the complex information management problems that have beset EPA since its inception. To finish the job, EPA will need to follow up and expand its data improvement initiatives to fill what the agency considers to be the key gaps in its data, take advantage of opportunities to develop and implement data standards to achieve compatibility among environmental databases, and ensure the accuracy of its data. Such actions, which will require several years to complete, are complex and will involve considerable difficulties in obtaining the scientific and environmental data that are needed, reaching agreements with key internal and external stakeholders and EPA’s state regulatory partners on the appropriate environmental measures of EPA’s programs, and ensuring that key databases are accurate and compatible. Given these difficulties and the multiyear commitment that will be needed to overcome them, an action plan detailing the strategies, resources, benchmarks, and milestones for completing specific actions would be useful to ensure that EPA’s information improvement actions continue to receive appropriate attention within the agency and that these actions can be monitored by the Congress and others. To help EPA obtain the data it needs to effectively set priorities, assess Recommendation progress in achieving goals and objectives, and report on accomplishments in a credible way, we recommend that the EPA Administrator direct the program manager of the new information office to Page 21 GAO/RCED-99-261 EPA Information Management B-283313 develop an action plan that details the key steps that the agency needs to take to ensure that EPA’s environmental and regulatory data are sufficiently complete, compatible, and accurate to meet its needs. This action plan should include the office’s strategy, milestones, and resource needs to (1) fill key gaps that have been identified in the agency’s information on environmental conditions; (2) identify and develop all needed data standards and implement them in all major databases; (3) coordinate EPA’s data standardization efforts with the states, federal agencies, and other organizations that maintain major environmental databases; (4) improve the collection of accurate data by implementing its quality assurance program throughout the agency as well as in the states; and (5) identify procedures needed so that data errors detected in one EPA information system can be corrected agencywide. We provided copies of a draft of this report to EPA for review and Agency Comments comment. We also met with representatives of the EPA offices responsible for the activities discussed in this report and with the agency’s Associate Deputy Administrator. EPA said that the report accurately describes both the agency’s problems with and efforts to improve information management. EPA concurred with our recommendation and said that it is consistent with the agency’s intentions concerning information management. EPA also pointed out that its forthcoming information strategic plan should provide the overall strategy needed to ensure the completeness, compatibility, and accuracy of EPA’s environmental data. The agency also offered several technical comments and clarifications, which we incorporated as appropriate. The scope and methodology for our work are discussed in appendix I. We performed our work from December 1998 through August 1999 in accordance with generally accepted government auditing standards. Page 22 GAO/RCED-99-261 EPA Information Management B-283313 We will send copies of this report to the Honorable Carol Browner, EPA Administrator, and to other interested parties. We will also make copies available to others on request. Please call me at (202) 512-6111 if you or your staff have any questions. Key contributors to this report are listed in appendix III. Peter F. Guerrero Director, Environmental Protection Issues Page 23 GAO/RCED-99-261 EPA Information Management Contents Letter 1 Appendix I 26 Scope and Methodology Appendix II 28 Major Databases Covered by EPA’s REI Action Plan Appendix III 30 GAO Contact and Staff Acknowledgments Bibliography 31 Table Table 1: EPA’s Analysis of the Number and Type of Annual 15 Performance Measures for Its Strategic Goals for Fiscal Year 2000 Abbreviations ECOS Environmental Council of the States EPA Environmental Protection Agency HUD Department of Housing and Urban Development NEPPS National Environmental Performance Partnership System REI Reinventing Environmental Information VA Department of Veterans Affairs Page 24 GAO/RCED-99-261 EPA Information Management Page 25 GAO/RCED-99-261 EPA Information Management Appendix I Scope and Methodology To identify the Environmental Protection Agency’s (EPA) initiatives to improve information management within the agency and between EPA and its various stakeholders, we held discussions with officials of the following EPA offices: the Office of Information Transition and Organizational Planning, which has responsibility for establishing a central information office to improve EPA’s data management and information sharing activities; the Office of Information Resources Management; the Center for Environmental Information and Statistics; the Reinvention Office; the Office of Planning, Analysis, and Accountability; and the Office of Research and Development. To determine the status of these initiatives, we conducted interviews and reviewed documents obtained from officials responsible for carrying out the projects. We also reviewed reports and written comments on the projects and needed improvements by EPA’s Science Advisory Board, National Advisory Council on Environmental Policy and Technology, and Office of Inspector General; the National Academy of Public Administration; the Center for Strategic and International Studies; Resources for the Future; and by other public interest and environmental organizations. We also discussed the projects and EPA’s overall efforts to improve environmental information management with officials of the Environmental Council of the States, a national nonpartisan, nonprofit association of state and territorial environmental commissioners that monitors and provides input on EPA’s improvement efforts and is working in partnership with EPA to improve their data sharing. In addition, we attended EPA-sponsored workshops and stakeholder meetings used to share information and solicit input on the agency’s new information office and on various efforts to improve the accuracy, compatibility, and completeness of EPA’s environmental data. We also attended a May 1999 conference of environmental data users and held discussions with agency representatives involved in managing EPA’s data quality reforms and in making environmental information available to the public. To determine the status of EPA’s new central information office, including its organization, resources, duties, and responsibilities, we interviewed the director of the task force for the office’s information transition, operations, and planning, and task force members. We also interviewed the new office’s deputy national program manager designee; the director designees for its office of information technology services and office of planning, resources, and outreach; and the director and deputy director designees for its office of information collection. The new office’s national program manager had not been designated at the time of our review. Page 26 GAO/RCED-99-261 EPA Information Management Appendix I Scope and Methodology We conducted our work from December 1998 through August 1999 in accordance with generally accepted government auditing standards. Page 27 GAO/RCED-99-261 EPA Information Management Appendix II Major Databases Covered by EPA’s REI Action Plan Database Function Air programs Aerometric Information Retrieval System/Air Contains monitoring data on ambient air Quality Subsystem (AIRS/AQS) quality from about 10,000 stations around the country AIRS Facility Subsystem (AFS) Contains data on emissions, compliance, and permits for nearly 150,000 stationary air pollution sources Risk Management Plan Information System Scheduled to become operational in 1999; (RMP*Info) will contain all Risk Management Program data (excluding off-site consequence analysis) for facilities Water programs Permit Compliance System (PCS) Contains National Pollutant Discharge Elimination System data; tracks permits and monitoring data for more than 64,000 facilities Safe Drinking Water Information System Contains information on the nation’s (SDWIS) drinking water, including sampling data and information on noncompliance with regulatory standards Water Quality Information System Contains monitoring data on ambient water (STORET X) quality and biological samples from over 850,000 stations across the nation Land disposal programs RCRA Information System (RCRIS) Contains identification and location data for all hazardous waste handlers and tracks permits, site closure status, compliance, and cleanup activities Biennial Reporting System (BRS) Contains data on the generation of hazardous waste from large-quantity generators and data on waste management practices from treatment, storage, and disposal facilities CERCLIS Information System (CERCLIS 3) Contains information on hazardous waste sites, including data on site inspections, preliminary assessments, and remediation Public awareness programs Toxic Release Inventory System (TRIS) Contains data from industry on the release of over 300 toxic chemicals into the air, water, and land Envirofacts Data Warehouse (EF) Provides a single point of access to data from seven major EPA databases (continued) Page 28 GAO/RCED-99-261 EPA Information Management Appendix II Major Databases Covered by EPA’s REI Action Plan Database Function Enforcement programs OECA Docket (Docket) Tracks and reports information on civil judicial and administrative enforcement cases brought under the authority of environmental statutes National Compliance Database (NCDB) Tracks compliance monitoring and enforcement activities for the Pesticides and Toxic Substances Compliance and Enforcement Program Source: Based on “Burden Reduction and State Environmental Agencies,” Environmental Council of the States, 1999. Page 29 GAO/RCED-99-261 EPA Information Management Appendix III GAO Contact and Staff Acknowledgments Edward A. Kratzer, (202) 512-6553 Contact In addition to the individual named above, Susan E. Swearingen, J. Acknowledgments Kenneth McDowell, Donald E. Pless, and John A. Crossen made key contributions to this report. Page 30 GAO/RCED-99-261 EPA Information Management Bibliography National Advisory Council for Environmental Policy and Technology. Environmental Managing Information as a Strategic Resource: Final Report and Information Recommendations of the Information Impacts Committee. EPA 100-R-98-002. Washington, D.C.: Jan. 1998. _____. Using Information Strategically to Protect Human Health and the Environment: Recommendations for Comprehensive Information Resources Management. IRM Strategic Planning Task Force. Environmental Information and Assessment Committee. EPA 270-K-94-002. Washington, D.C.: Aug. 1994. Office of Information and Regulatory Affairs. Information Collection Budget of the United States Government, Fiscal Year 1999. Office of Management and Budget. Washington, D.C.: Apr. 16, 1999. Science Advisory Board. Review of the Center for Environmental Information and Statistics’ (CEIS) Draft Data Suitability Assessment of Major EPA Databases. EPA-SAB-EC-99-010. Washington, D.C.: Feb. 19, 1999. Cooper, Mary H. “The Cleanup’s Next Phase: Setting Environmental Priorities for the 21st Century.” CQ Outlook, June 5, 1999. U.S. Environmental Protection Agency. Information Resources Management: Office of Water Data Integration Efforts. Office of Inspector General. E1NWG6-15-0001-8100177. Washington, D.C.: June 22, 1998. _____. Special Review of EPA’s Information Systems Program. Vol. 1. Office of Inspector General. E1SKG3-15-0098-4400038. Washington, D.C.: Mar. 24, 1994. U.S. General Accounting Office. Environmental Enforcement: EPA Needs a Better Strategy to Manage Its Cross-Media Information. GAO/IMTEC-92-14. Apr. 2, 1992. _____. Environmental Information: Agencywide Policies and Procedures Are Needed for EPA’s Information Dissemination. GAO/RCED-98-245. Sept. 24, 1998. _____. Environmental Information: EPA Could Better Address Concerns About Disseminating Sensitive Business Information. GAO/RCED-99-156. June 25, 1999. Page 31 GAO/RCED-99-261 EPA Information Management Bibliography _____. Environmental Protection: EPA Faces Formidable Challenges Managing Water Quality Data. GAO/T-AIMD-93-2. Aug. 5, 1993. _____. Environmental Protection: EPA’s Actions to Improve Long-Standing Information Management Weaknesses. GAO/T-IMTEC-93-4. Mar. 29, 1993. _____. Environmental Protection: EPA’s Plans to Improve Long-Standing Information Resources Management Problems. GAO/AIMD-93-8. Sept. 16, 1993. _____. Environmental Protection: EPA’s Problems With Collection and Management of Scientific Data and Its Efforts to Address Them. GAO/T-RCED-95-174. May 12, 1995. _____. Environmental Protection: Status of EPA’s Efforts to Create a Central Information Office. GAO/T-RCED-99-147. Apr. 13, 1999. _____. EPA Toxic Substances Program: Long-Standing Information Planning Problems Must Be Addressed. GAO/AIMD-94-25. Nov. 17, 1993. _____. Hazardous Waste: Benefits of EPA’s Information System Are Limited. GAO/AIMD-95-167. Aug. 22, 1995. _____. Pesticides: EPA’s Efforts to Collect and Take Action on Exposure Incident Data. GAO/RCED-95-163. July 12, 1995. _____. Pesticides: Information Systems Improvements Essential for EPA’s Reregistration Efforts. GAO/IMTEC-93-5. Nov. 23, 1992. _____. Toxic Substances Control Act: Legislative Changes Could Make the Act More Effective. GAO/RCED-94-103. Sept. 26, 1994. _____. Toxic Substances: EPA Needs More Reliable Source Reduction Data and Progress Measures. GAO/RCED-94-93. Sept. 23, 1994. _____. Toxic Substances: EPA Should Focus Its Chemical Use Inventory on Suspected Harmful Substances. GAO/RCED-95-165. July 7, 1995. _____. Waste Minimization: Major Problems of Data Reliability and Validity Identified. GAO/PEMD-92-16. Mar. 23, 1992. Page 32 GAO/RCED-99-261 EPA Information Management Bibliography Center for Strategic and International Studies. The Environmental EPA Management Protection System in Transition: Toward a More Desirable Future. Final report of Enterprise for the Environment. Washington, D.C.: Jan. 1998. National Academy of Public Administration. Resolving the Paradox of Environmental Protection: An Agenda for Congress, EPA, and the States. Washington, D.C.: Sept. 1997. _____. Setting Priorities, Getting Results: A New Direction for EPA. Washington, D.C.: Apr. 1995. National Performance Review. Reinventing Environmental Regulation. Washington, D.C.: Mar. 16, 1995. Resources for the Future. Regulating Pollution: Does the U.S. System Work? Washington, D.C.: 1997. Science Advisory Board. Review of the Agency-Wide Quality Management Program. EPA-SAB-EEC-LTR-98-003. Washington, D.C.: July 24, 1998. _____. Review of the Implementation of the Agency-Wide Quality System. EPA-SAB-EEC-LTR-99-002. Washington, D.C.: Feb. 25, 1999. U.S. Environmental Protection Agency. Superfund: EPA Had Not Effectively Implemented Its Superfund Quality Assurance Program. Office of Inspector General. E1SKF7-08-0011-8100240. Washington, D.C.: Sept. 30, 1998. U.S. General Accounting Office. Environmental Protection Agency: Protecting Human Health and the Environment Through Improved Management. GAO/RCED-88-101. Aug. 16, 1988. _____. Environmental Protection: Assessing EPA’s Progress in Paperwork Reduction. GAO/T-RCED-96-107. Mar. 21, 1996. _____. Environmental Protection: Key Management Issues Facing EPA. GAO/RCED-98-153R. Apr. 23, 1998. _____. Major Management Challenges and Program Risks: Environmental Protection Agency. GAO/OCG-99-17. Jan. 1999. Page 33 GAO/RCED-99-261 EPA Information Management Bibliography _____. Managing for Results: Analytic Challenges in Measuring Performance. GAO/HEHS/GGD-97-138. May 30, 1997. _____. Managing for Results: EPA’s Efforts to Implement Needed Management Systems and Processes. GAO/T-RCED-97-116. Apr. 8, 1997. _____. Managing for Results: EPA’s Efforts to Implement Needed Management Systems and Processes. GAO/RCED-97-156. June 18, 1997. _____. Observations on the Environmental Protection Agency’s Fiscal Year 2000 Performance Plan. GAO/RCED-99-237R. July 31, 1999. _____. Paperwork Reduction: Burden Reduction Goal Unlikely to Be Met. GAO/T-GGD/RCED-96-186. June 5, 1996. _____. Paperwork Reduction: Government Goals Unlikely to Be Met. GAO/T-GGD-97-114. June 4, 1997. _____. Performance Plans: Selected Approaches for Verification and Validation of Agency Performance Information. GAO/GGD-99-139. July 30, 1999. _____. Results Act: Observations on EPA’s Draft Strategic Plan. GAO/RCED-97-209R. July 30, 1997. Environmental Council of the States. Burden Reduction and State EPA-State Working Environmental Agencies. Washington, D.C.: 1999. Relationship U.S. General Accounting Office. Environmental Protection: Collaborative EPA-StateEffort Needed to Improve New Performance Partnership System. GAO/RCED-99-171. June 21, 1999. _____. Environmental Protection: EPA’s and States’ Efforts to Focus State Enforcement Program on Results. GAO/RCED-98-113. May 27, 1998. _____. Environmental Protection: EPA’s and States’ Efforts to Focus State Enforcement Programs on Results. GAO/T-RCED-98-233. June 23, 1998. _____. Environmental Protection: EPA’s and States’ Efforts to “Reinvent” Environmental Regulation. GAO/T-RCED-98-33. Nov. 4, 1997. Page 34 GAO/RCED-99-261 EPA Information Management Bibliography _____. Environmental Protection: Status of EPA’s Initiatives to Create a New Partnership With States. GAO/T-RCED-96-87. Feb. 29, 1996. _____. EPA and the States: Environmental Challenges Require a Better Working Relationship. GAO/RCED-95-64. Apr. 3, 1995. (160461) Page 35 GAO/RCED-99-261 EPA Information Management Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Environmental Information: EPA Is Taking Steps to Improve Information Management, but Challenges Remain
Published by the Government Accountability Office on 1999-09-17.
Below is a raw (and likely hideous) rendition of the original report. (PDF)