oversight

Environmental Information: EPA Is Taking Steps to Improve Information Management, but Challenges Remain

Published by the Government Accountability Office on 1999-09-17.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Requesters




September 1999
                  ENVIRONMENTAL
                  INFORMATION
                  EPA Is Taking Steps to
                  Improve Information
                  Management, but
                  Challenges Remain




GAO/RCED-99-261
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-283313

      September 17, 1999

      The Honorable Christopher S. Bond
      Chairman
      The Honorable Barbara A. Mikulski
      Ranking Minority Member
      Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      United States Senate

      The Honorable James T. Walsh
      Chairman
      The Honorable Alan B. Mollohan
      Ranking Minority Member
      Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      House of Representatives

      The need to manage its programs for environmental results substantially
      increases the Environmental Protection Agency’s (EPA) demand for
      high-quality environmental information. Such information is also needed
      to identify and respond to emerging problems before significant damage is
      done to the environment. Various studies have shown that although much
      scientific and environmental data have already been collected, many gaps
      exist, and the data are often difficult to compile because different
      collection methods have been used. Likewise, much effort is still needed
      to identify, develop, and reach agreement on a comprehensive set of
      environmental measures to link EPA’s activities to changes in human health
      and environmental conditions.

      Recognizing the long-standing and serious shortcomings in the
      environmental information needed to manage for results, the EPA
      Administrator announced plans in October 1998 to create an office with
      central responsibility for information management, policy, and technology.
      The efforts to improve information management that preceded the new
      office, and that are to be absorbed by it, include several agencywide
      initiatives directed at improving the quality of EPA’s data and the agency’s
      ability to share data internally and externally. Reflecting congressional
      interest in EPA’s data management activities, the conference report
      accompanying the VA, HUD, and Independent Agencies fiscal year 1997




      Page 1                             GAO/RCED-99-261 EPA Information Management
                   B-283313




                   appropriations act1 and subsequent discussions with your offices directed
                   us to review (1) recent initiatives designed to help EPA improve the
                   accuracy, completeness, and compatibility of its data; (2) the impact of
                   data gaps and inconsistencies on EPA’s ability to evaluate and report on the
                   results of its programs under the Government Performance and Results
                   Act (the Results Act); and (3) the major management challenges facing
                   EPA’s new central information office.



                   EPA has several data improvement initiatives to obtain the environmental
Results in Brief   information needed to effectively set priorities, assess progress in
                   achieving goals and objectives, and report on accomplishments in a
                   credible way. These initiatives are specifically aimed at identifying critical
                   gaps in EPA’s environmental data, developing data standards to enable
                   separately designed databases to operate compatibly with one another,
                   and identifying and correcting inaccuracies. While these initiatives are
                   steps in the right direction, they are limited in scope and do not provide
                   the overall strategy needed to ensure the completeness, compatibility, and
                   accuracy of EPA’s environmental data. For example, EPA has not yet
                   identified or evaluated options for filling the agency’s data gaps, has not
                   yet developed a plan detailing how it will standardize the data in many of
                   the agency’s key databases, and has not yet identified the specific actions
                   that the agency and its state partners need to take to ensure the accuracy
                   of environmental data.

                   EPA’s ability to evaluate the outcomes of its programs in terms of changes
                   in the environment is limited by gaps and inconsistencies in the quality of
                   its data. Of the 357 measures of performance that EPA has developed for
                   use during fiscal year 2000 to report its accomplishments under the
                   Results Act, the agency reports that only 71 will reflect environmental
                   outcomes; the other measures will reflect program activities, such as the
                   number of actions taken to enforce environmental laws. EPA program
                   managers acknowledge that additional measures of environmental
                   outcomes are needed and that the agency’s forthcoming information plan
                   will encourage such measures in all program offices and establish
                   milestones for creating them. To meet these milestones, EPA’s program
                   offices will have to overcome (1) difficulties in establishing
                   cause-and-effect relationships between program activities and
                   environmental outcomes, (2) a lack of reliable baseline data against which
                   to measure progress and a more generalized lack of reliable data about the



                   1
                    H.R. Rep. No. 104-812, at 70-71 (1996).



                   Page 2                                     GAO/RCED-99-261 EPA Information Management
             B-283313




             environment, and (3) constraints on the resources for gathering and
             analyzing the data.

             Creating a successful central information office from disparate parts of
             EPA will help the agency to address obstacles to obtaining the data it needs
             to manage for results. However, establishing a successful office will
             require appropriate resources and the commitment of senior management.
             One of the office’s most pressing challenges will be to develop a plan that
             identifies clear priorities for the office and the resources it will need to
             successfully lead the agency’s efforts to make significant improvements in
             information management. Other key challenges for the new office, which
             have thwarted EPA’s earlier efforts to improve its information management
             activities, will be (1) obtaining sufficient authority and resources within
             EPA to address the complex information management issues facing the
             agency and (2) working more effectively with the states and regulated
             industries to balance the demand for more data with efforts to reduce the
             reporting burden. We are recommending that EPA develop an action plan
             that details the strategy, milestones, and resources that the new
             information office will require to ensure that EPA’s environmental and
             regulatory data are sufficiently complete, compatible, and accurate to
             meet its needs.


             EPA and the states collect a wealth of environmental data under various
Background   statutory and regulatory requirements, including reports on air emissions
             under the Clean Air Act, wastewater discharges under the Clean Water
             Act, and pollutant levels in drinking water under the Safe Drinking Water
             Act. However, EPA’s existing approach to data management is outmoded in
             many ways. It continues to rely heavily on paper-based reporting, and its
             many separately designed databases are generally not compatible with
             each other. Consequently, EPA has not been able to aggregate much of the
             data from the many different databases to present comprehensive
             information on chemicals, industrial sectors, localities, and environmental
             conditions because basic data elements are not standardized across these
             databases. Moreover, important gaps in the data exist. Data obtained from
             the detailed monitoring of environmental conditions and of human
             exposures to toxic pollutants are limited, and the human health and
             ecological effects of many chemical pollutants are not well understood.
             For example, EPA’s Integrated Risk Information System, which is a
             database of the potential health effects from chronic exposure to various
             substances found in the environment, has toxicological data on only
             one-third of the known hazardous air pollutants.



             Page 3                             GAO/RCED-99-261 EPA Information Management
                           B-283313




                           Since the announcement of its regulatory reinvention program in
                           March 1995 and the issuance of its strategic plan in September 1997, both
                           of which recognized the need for improvement in the management of
                           information, EPA has begun several initiatives to improve how it collects,
                           manages, and disseminates environmental information. In addition, in
                           October 1998, the EPA Administrator announced an organizational change
                           to create a new office responsible for information management,
                           information policy, and technology stewardship. This office would be
                           responsible for developing and implementing goals, standards, and
                           accountability systems to manage and improve the quality of data used
                           both within the agency and by the public. The office would also have the
                           authority to set and oversee agencywide standards and policies for
                           managing information resources. The new office, which is scheduled to
                           become operational in early October 1999, will consolidate all or parts of
                           the existing Office of Information Resources Management, the Center for
                           Environmental Information and Statistics, and other organizational
                           components, such as the Toxic Release Inventory program.


                           EPA’s recent major initiatives to improve data quality address
Recent Initiatives         (1) long-standing problems involving gaps in the agency’s environmental
Highlight Obstacles        data; (2) the need for core, or common, data standards so that data from
Facing EPA as It           various information systems can be pulled together to present
                           comprehensive information on geographical locations, chemicals,
Seeks to Improve           industrial sectors, and environmental conditions; and (3) the need for an
Data Quality               agencywide approach to ensuring the accuracy of EPA’s data—particularly
                           a process for correcting errors in the agency’s databases. While EPA has
                           made progress, each initiative has encountered obstacles that highlight the
                           difficulties facing the agency as it attempts to improve its information
                           management activities.


EPA Faces Extensive Gaps   For more than 25 years, EPA, the states, and others have collected data on
in Information About the   the health and environmental effects of a variety of pollutants,
Environment, Health        environmental conditions, and the compliance of, and enforcement
                           actions taken against, the regulated community. Despite the vast array of
Risks, and the Agency’s    data in EPA’s information systems, we, the states, regulated entities, and
Effectiveness              EPA itself have pointed out that the agency does not have much of the
                           information it needs pertaining to environmental conditions and trends
                           (i.e., environmental indicators), the potential human health risks of
                           various pollutants, and the environmental results of EPA’s activities.




                           Page 4                             GAO/RCED-99-261 EPA Information Management
B-283313




These extensive data gaps are a result both of a lack of fundamental
scientific knowledge and of inadequate data collection, according to EPA
and others. EPA evaluations have recognized that the agency has numerous
and significant gaps in its risk and environmental data. For example, one
EPA review concluded that complete data on health effects exist for only
7 percent of the 3,000 most widely used chemicals. Similarly, EPA found
that it lacks basic toxicity data for more than one-third of the chemicals
produced in large volumes as well as for about two-thirds of the known
hazardous air pollutants. Moreover, the environmental data that EPA does
have are often fragmented because they were collected under various
laws, such as the Clean Air Act or the Safe Drinking Water Act.
Consequently, these data are not easily integrated, if they can be
integrated at all, to provide environmental information about specific
locations or the nation as a whole. One expert outside EPA recently
expressed his concern about environmental data gaps: “We don’t have
enough information to tell us where we are or where the trends are going.
We don’t really know whether air quality, and especially water quality, are
really improving or not under current law. As for solid waste, the situation
is hopeless. We don’t even know where it is, much less whether it’s getting
better or worse.”2

EPA does not yet have a strategy in place for prioritizing its needs for
additional data and filling key data gaps. The agency has, however, made
some initial efforts to assess its data needs and how it may obtain the
needed data. For example, officials from EPA’s Office of Planning, Analysis,
and Accountability, which has responsibility for coordinating the agency’s
compliance under the Results Act, said they are conducting an analysis to
determine whether there are gaps in the data needed to measure the
agency’s progress in meeting its strategic objectives.

Another effort dealing with the agency’s need for additional data has been
led by EPA’s Center for Environmental Information and Statistics, which
will become a part of the new information office. In early 1999, the Center
completed the first phase of a strategy to identify and address EPA’s data
gaps. During this initial screening analysis, staff from the Center and EPA’s
program offices identified 26 key environmental problem areas that the
agency has committed itself to address through its strategic plan, goals,
and objectives. The Center then screened these problem areas for major
gaps in data concerning health and ecological toxicity, ambient
environmental conditions, and sources of pollution or other environmental

2
Terry Davies, Director, Center for Risk Management, Resources for the Future, quoted in Mary H.
Cooper, “The Cleanup’s Next Phase: Setting Environmental Priorities for the 21st Century,” CQ
Outlook (June 5, 1999).



Page 5                                        GAO/RCED-99-261 EPA Information Management
    B-283313




    stressors. The eight environmental problem areas in which EPA has the
    most significant data gaps, according to the Center’s draft screening
    analysis, are

•   aquatic ecosystem health—biological stressors (such as the impact of
    nonnative species),
•   aquatic ecosystem health—physical alterations (such as erosion),
•   indoor air,
•   pesticides—nondietary human health risks (such as exposure of the skin
    to agricultural pesticides),
•   air toxics,
•   pesticides—dietary human health risks,
•   aquatic ecosystem health—toxics, and
•   climate change.

    The second phase of the Center’s effort was to initially focus on two or
    three environmental areas identified in the first phase and develop
    recommendations for addressing the highest-priority data gaps in these
    areas. EPA realized, however, that a strategy for filling data gaps would
    need to be coordinated with other activities within the agency, such as
    efforts to reduce the reporting burden. As a result, the Center has not
    continued its effort to develop this strategy pending its move into the new
    information office. The creation of a strategy for prioritizing and filling key
    data gaps will be a part of the development of EPA’s comprehensive
    information plan, according to officials responsible for the plan.

    Efforts are under way in EPA to develop the data needed to fill at least
    some of these gaps. One such effort is the Environmental Monitoring and
    Assessment Program, which is working with other federal agencies to
    develop information that the public, scientists, and the Congress can use
    to evaluate the overall health of the nation’s ecological resources. Another
    effort, called the High Production Volume Challenge Program, was
    developed by EPA—in partnership with industry and environmental
    groups—to make publicly available a complete set of baseline data on the
    health and environmental effects of each chemical manufactured in, or
    imported into, the United States in amounts of 1 million or more pounds
    per year. Companies participating in this program pledge to evaluate the
    adequacy of existing data for these chemicals and to conduct tests where
    needed to fill data gaps.

    The new information office will be responsible for encouraging EPA’s
    program offices to reach out to other federal agencies, as well as to



    Page 6                               GAO/RCED-99-261 EPA Information Management
                          B-283313




                          universities, research institutes, and other sources of environmental
                          information, for data that EPA does not collect but that may exist
                          elsewhere. To date, EPA’s efforts to share and obtain data from other
                          agencies, other than the state environmental agencies with which it shares
                          responsibility for implementing federal environmental laws, have been
                          limited. Such efforts have been hampered by technological limitations
                          imposed by the myriad of incompatible information systems in use across
                          the government. The new information office will be responsible for
                          promoting EPA’s efforts to exchange data with other federal agencies.

                          EPA also faces concerns about the reporting burden it may place on states
                          and the regulated community as it seeks to fill data gaps. There is concern
                          within the regulated community and among EPA’s state partners about the
                          possibility that EPA will expand its reporting requirements—which it may
                          have to do to obtain the data it needs. The states are calling for any such
                          expansion to be balanced with a reduction in existing reporting
                          requirements. Moreover, much of the data needed, such as environmental
                          monitoring information, will be expensive to obtain. It will thus be
                          important for EPA to prioritize its needs for additional data, to work with
                          the states and industry to reduce the reporting burden, and to encourage
                          efforts to use data that may already have been collected by other federal
                          agencies or other entities.


EPA and the States Have   Because most of the information systems that EPA has developed over the
Taken Initial Steps to    years are not compatible with each other, the agency has not been able to
Increase Data             integrate important data it has collected about the environment and
                          regulated facilities. EPA has been criticized from both outside and within
Compatibility             the agency for having developed and maintained “stovepiped” data
                          systems that cannot share the enormous amounts of data gathered. In
                          recent years, EPA has tried to make its data more compatible, and in 1998,
                          the Administrator committed the agency to begin working toward full data
                          integration across programs and across environmental media.

                          Essential to the effort to integrate data from various EPA databases are
                          common data definitions and formats, known as data standards. EPA has
                          indicated that it needs such standards not only to make its data
                          compatible, but also to facilitate some of its other information initiatives.
                          For example, EPA considers data standards as key to reducing the
                          reporting burden on industry and the states because it would permit
                          integrated reporting of information to EPA.




                          Page 7                               GAO/RCED-99-261 EPA Information Management
B-283313




In recent years, EPA has undertaken several efforts to develop standards
for some of the data items in its information systems, most recently as part
of the 1998 action plan for the Reinventing Environmental Information
(REI) initiative. As part of this initiative, EPA and the states are developing
six data standards to be used in 13 of EPA’s major databases. (See app. II
for a list of these databases.) The standards being developed will apply
common definitions and formats for describing the following items in each
database: (1) date, (2) facility identification, (3) industrial classification,
(4) location (latitude/longitude), (5) identification of chemical names, and
(6) biological taxonomy (i.e., categories for describing plants and animals,
such as class, family, and species). EPA considers these standards as a key
step in moving to full integration of EPA’s data across its major systems.
According to the REI action plan, these six standards will be developed,
approved by EPA in partnership with the states, and in use in the 13
designated databases by the end of fiscal year 2003.

The current effort to develop data standards is attempting to avoid the
limitations of previous efforts, which encountered both technical
difficulties in determining what appropriate standards should be and the
unwillingness of program offices to adopt the standards. To overcome
such problems, the REI effort involves a cooperative interchange among
the EPA officials responsible for developing the standards, the EPA program
offices that will use the data, and the state environmental agencies to
reach consensus. While responsible officials believe that getting “buy-in”
from the key collectors and users of the data is essential for the eventual
successful adoption of the standards, it has contributed to the time
required to launch the standards.

According to EPA, as of August 1999, two of the six standards—the
common definitions and formats for describing the date and the industrial
classification—were ready for implementation pending review by the
states. Definitions and formats have been proposed for the other
standards, which are in varying stages of development and approval by EPA
and review by the states. After agreeing on each standard, the states and
EPA will need to agree on the rules for how each standard will be used.
Only then will each standard be ready to be used for data being entered in
the databases.

To meet the implementation date of fiscal year 2003, EPA has set
milestones that call for the approval of the definitions and formats for all
six standards by the end of 1999. Both EPA officials and representatives




Page 8                               GAO/RCED-99-261 EPA Information Management
B-283313




from the Environmental Council of the States (ECOS),3 which is
spearheading the states’ involvement in this effort, have indicated that the
fiscal year 2003 milestone for implementing the six standards may be met
despite schedule slippages and the complexities of the task. ECOS officials
also believe that as long as the states remain full partners in the
development of the standards, there is a much greater likelihood that the
states as well as EPA will find the standards useful and appropriate and that
the initiative will ultimately be successful.

The current initiative is limited in terms of the number of standards being
developed (six), the number of EPA databases in which the standards will
initially be used (13), and the amount of data in those databases (only the
new data being entered) that will incorporate the standards. EPA
recognizes that its current effort is only a first step toward its goal of full
data integration. ECOS officials believe that EPA’s focusing on the six data
standards and 13 databases is an appropriate way to begin. ECOS officials
also believe, based on the states’ experiences in this area, that the effort
should be limited to new data being entered into the databases. According
to ECOS officials, previous efforts by states to conform existing data in their
systems to new requirements such as data standards have required a
prohibitive expenditure of time, expertise, and other resources.

The focus of the current standards development effort is primarily on the
compatibility of data among EPA’s information systems and those of state
environmental agencies, rather than also emphasizing the compatibility of
EPA’s data with the data of other federal agencies and nongovernmental
sources. EPA’s Science Advisory Board has recommended that EPA do more
to link the agency’s databases with federal or other external databases,
noting that “answering many health-related questions frequently requires
linking environmental data with census, cancer or birth registry data, or
other data systems (such as water distribution maps) to determine
whether there is a relationship between the environmental measures and
health.”4 While EPA officials told us that they recognize the importance of
linking EPA’s databases with those of other federal agencies, resource
constraints and the lack of statutory coordination requirements have
limited their actions in this area to participation in interagency forums and
coalitions. They said that they have no specific plans or target dates to
initiate more formal efforts. For at least one standard, however, the

3
 ECOS is a national nonpartisan, nonprofit association of state and territorial environmental
commissioners.
4
 The EPA Science Advisory Board provides advice to EPA from scientists outside the agency. Science
Advisory Board, Review of the Agency-Wide Quality Management Program, EPA-SAB-EEC-LTR-98-003
(Washington, D.C.: EPA, July 24, 1998).



Page 9                                          GAO/RCED-99-261 EPA Information Management
                        B-283313




                        current initiative is providing EPA with an opportunity to encourage
                        compatibility with outside data sources. In developing a standard for
                        describing the biological categories of plants and animals, EPA has
                        participated in a joint effort by U.S. and Canadian federal agencies, the
                        states, academic institutions, museums, and nongovernmental
                        organizations to develop and maintain standard information—known as
                        the Integrated Taxonomic Information System.


                        Data accuracy, or the extent to which data are free from significant error,
Long-Standing           has long been a serious challenge facing EPA. Various reviews that we, EPA,
Concerns Persist        and others have done have revealed persistent concerns about the
About the Accuracy of   accuracy of data in many of EPA’s information systems. EPA has not
                        conducted an agencywide assessment of the accuracy of its information
EPA’s Data              systems. However, in 1998, agency staff reviewed numerous studies of
                        EPA’s data systems and found that these studies suggested variable error
                        rates, with some systems and types of data more prone to error than
                        others. Data used for some purposes may need to be more accurate than in
                        other instances, which is, in part, why EPA has not set an across-the-board
                        target rate for data accuracy. While EPA acknowledges that data errors
                        exist in many of its systems, the agency believes that, in the aggregate, its
                        data are of sufficient quality to support its programmatic and regulatory
                        decisions.

                        Preventing data errors and correcting errors once they have been
                        identified, essential to data accuracy, have proved to be daunting tasks for
                        EPA. For example, in January 1998, an EPA advisory council on information
                        management issues described the difficulty of correcting errors in EPA’s
                        databases: “Once an error is stored in one or more of the Agency’s
                        systems, making corrections to all those systems is an exercise in
                        frustration and futility. There is no simple way to ensure corrections are
                        made to all possible systems.”5 Moreover, efforts to improve the accuracy
                        of EPA’s information systems will need the cooperation of a complicated
                        network of individuals and entities who provide, collect, manage, or use
                        the data. Among such individuals and entities are EPA employees, staff
                        from state—and, in some instances, local—governments, the regulated
                        community, contractors, and private citizens who use data from these
                        systems.



                        5
                         National Advisory Council for Environmental Policy and Technology, Managing Information as a
                        Strategic Resource: Final Report and Recommendations of the Information Impacts Committee, EPA
                        100-R-98-002 (Washington, D.C.: EPA, Jan. 1998).



                        Page 10                                      GAO/RCED-99-261 EPA Information Management
B-283313




Preventing data errors is an issue of both whether accurate data are being
generated and whether the integrity of the reported data is maintained as
the data flow through the states’ and EPA’s information systems. EPA has an
agencywide quality system that was revised in 1998 to expand and clarify
requirements for how environmental data are collected and managed.6 All
EPA organizational units involved in the use of environmental data are
subject to the quality requirements, which include provisions to define and
meet data quality needs.

EPA’s quality program requires that each entity document its quality system
in a quality management plan. Among other things, the quality
management plans must discuss the criteria for measuring data quality,
describe how the acquired data will be validated and verified, and identify
any constraints on data collection. Currently, about 40 EPA organizational
units are required to develop and implement quality management plans. In
addition, the quality program’s requirements, including the quality
management plan, apply to others outside of EPA, such as states, local and
tribal governments, and contractors who generate and collect
environmental data on behalf of EPA.

Although EPA’s Science Advisory Board recently commended the agency
for its development of a quality system and for the efforts of EPA’s quality
assurance division to champion the need for quality assurance and quality
control, the Board has also found that the system’s implementation has
been uneven within the agency, increasing the likelihood of problems with
data quality and the decisions made based on the data.7 Moreover, the
Board reported that more than 75 percent of the states authorized to
implement EPA’s environmental programs lack approved quality
management plans for all or some of these programs and thus are likely to
be generating data of unknown quality. The Board found the situation
worrisome as it implied that compliance with EPA’s quality system is
unimportant to these states. According to the Board, “Such a state is
exposing itself, the reliability of its decisions, and its credibility, to
criticisms due to its reliance upon data of unknown quality. The same is
true for those agency programs that depend upon those data.” The Board
recommended that EPA place its quality system at a higher level within the


6
 EPA defines environmental data as any measurements or information that describe environmental
processes or conditions or the performance of environmental technology. For EPA, environmental
data include information collected directly from measurements, such as environmental monitoring;
produced from models; and compiled from other sources such as databases or the scientific literature.
7
Science Advisory Board, Review of the Implementation of the Agency-Wide Quality System,
EPA-SAB-EEC-LTR-99-002 (Washington, D.C.: EPA, Feb. 25, 1999).



Page 11                                        GAO/RCED-99-261 EPA Information Management
B-283313




agency structure to bring more attention and priority to quality assurance
and quality control issues.

EPA’s Inspector General has also examined EPA’s quality assurance program
in terms of its ability to ensure data quality. In a 1998 review of the quality
system developed by EPA’s Superfund program, the Inspector General
found that EPA managers had not always fully developed and effectively
implemented their quality assurance programs to the extent that they
could ensure that they obtained data of known and adequate quality.8 The
Inspector General raised concerns that similar weaknesses may exist in
other EPA program offices. As the Science Advisory Board did, the
Inspector General recommended that EPA elevate the responsibility for
quality assurance for the agency as a whole to a level at which its manager
could be an effective and independent advocate for quality assurance. In
addition, the Inspector General recommended that the agency develop a
strategy to institutionalize the quality assurance program, improve
oversight of the program, develop minimum quality assurance
requirements, and report annually on the program’s effectiveness.

In April 1998, EPA’s Deputy Administrator called for a strategic action plan
for implementing an agencywide approach to ensuring data quality. This
plan contained recommendations for preventing and correcting errors. To
encourage error prevention, the plan recommended that EPA develop a
baseline review of each of its major data systems and establish milestones
for improving accuracy over time. It also recommended that EPA develop
data standards that would be used both by the states and EPA, arguing that
the states would be more diligent in maintaining high-quality data for EPA if
they used the same data in the course of running their own programs. To
encourage error correction, the plan’s recommendations called for EPA to
rely on data users to alert the agency to inaccurate data. Specifically, the
plan called for EPA to establish an easy-to-understand guidance system so
that users who noticed errors could report discrepancies to EPA.

Although the data quality strategic plan was submitted to EPA’s Acting
Deputy Administrator in December 1998, the plan has not been adopted,
and thus its recommendations have not been implemented. However, EPA’s
Acting Deputy Administrator has indicated that the plan will be useful to
the new information office. For example, he said that developing an error
correction system would be an “important early effort” of the new
information office and that this system would build on some of the

8
EPA, Superfund: EPA Had Not Effectively Implemented Its Superfund Quality Assurance Program,
Office of Inspector General, E1SKF7-08-0011-8100240 (Washington, D.C.: EPA, Sept. 30, 1998).



Page 12                                     GAO/RCED-99-261 EPA Information Management
                    B-283313




                    thinking that went into the 1998 plan. The system would rely on data
                    providers and others to bring errors to EPA’s attention and then would
                    ensure the prompt correction of the reported errors. The agency has
                    allocated funds to develop an error correction process, and staff have
                    developed a proposal; however, the proposal has not yet been discussed
                    with program and regional offices and has not been approved by
                    management of the new information office.

                    The Acting Deputy Administrator has also indicated that data quality
                    covers a broader array of issues than those addressed in the 1998 data
                    quality strategic plan and that the new information office will be tasked
                    with addressing these issues. As we pointed out in April 1999, EPA does not
                    yet have a common understanding of what data quality means and how the
                    agency and its state partners can most effectively ensure that the data
                    used for decision-making or disseminated to the public are of high quality.9
                    To address this, EPA plans to elevate data quality issues in the new
                    information office and to address both how data quality should be defined
                    and how the quality of the agency’s data can be improved. As part of this
                    effort, the agency plans to create a Quality and Information Council,
                    comprising senior executives from across the agency, that will provide
                    strategic direction and advice to the director of the new information office
                    on data quality, information technology investments, and other issues. The
                    information office will also include a “quality staff” that will focus on the
                    design, policy development, and oversight of the agency’s quality program.


                    Much of the current effort by EPA and its state partners to improve
EPA’s Success in    information management is tied to their initiatives to assess the results of
Developing          their programs. Spurred by the requirements of the Results Act, EPA has
Environmental       begun to set goals and measures of its performance that are intended to
                    help the agency, as well as the Congress and the public, assess the
Measures Will Be    environmental results of the agency’s activities. Under the Results Act, EPA
Dependent on Data   is required to set long-term and annual goals as well as to measure the
                    results of its programs in an annual report to the Congress. The first such
Improvements        report, due by the end of March 2000, will cover the agency’s performance
                    in fiscal year 1999.

                    The states and EPA have also been working together to develop mutually
                    agreeable environmental goals and a set of results-oriented “core
                    performance measures” to use in measuring the effectiveness and success

                    9
                     Environmental Protection: Status of EPA’s Efforts to Create a Central Information Office
                    (GAO/T-RCED-99-147, Apr. 13, 1999).



                    Page 13                                         GAO/RCED-99-261 EPA Information Management
B-283313




of the states’ implementation of national environmental programs. The
core performance measures are a central component of an initiative called
the National Environmental Performance Partnership System (NEPPS).10
The goals and measures are to be used to evaluate state programs with a
focus on outcomes, such as improvements in water quality, rather than
activity measures, such as the number of inspections performed at
manufacturing facilities. To date, EPA and the states have made limited
progress in developing outcome-oriented performance measures under
both the Results Act and NEPPS.

EPA has relatively few environmental outcome measures among the annual
performance measures it developed for fiscal year 2000 reporting under
the Results Act.11 A large majority of the performance measures for fiscal
year 2000 reflect the level of program activities, or outputs, such as the
number of regulations issued or enforcement actions taken. According to
EPA’s categorization, of 357 performance measures associated with these
goals, only 71, or about 20 percent, measure program outcomes. According
to EPA program managers, the agency has long used activity, or output,
measures to manage its programs and will continue to need some such
measures even as it incorporates more results-based goals and measures.
For example, EPA program managers said that some program activity
measures are required by statute and others are necessary for managing
the programs. However, in other cases, EPA officials said that they chose
their annual goals and measures based on what data they had available.




10
  NEPPS was intended, among other things, to give states with strong environmental performance
greater flexibility and autonomy in running environmental programs that EPA has delegated to the
states. We recently reported on the experiences to date in developing these measures and in
implementing the “performance partnership agreements.” Environmental Protection: Collaborative
EPA-State Effort Needed to Improve New Performance Partnership System (GAO/RCED-99-171,
June 21, 1999).
11
 Outcome measures are those expressed in terms of program impacts or human health and
environmental changes, rather than the projects to be completed or the number of activities to be
performed, which are referred to as outputs.



Page 14                                         GAO/RCED-99-261 EPA Information Management
                                        B-283313




Table 1: EPA’s Analysis of the Number
and Type of Annual Performance                                                                             Number of annual
Measures for Its Strategic Goals for                                                                     performance measures
Fiscal Year 2000                        EPA’s strategic goal                                         Output        Outcome                Total
                                        Goal 1: Clean air                                                 16                17               33
                                        Goal 2: Clean and safe water                                      74                11               85
                                        Goal 3: Safe food                                                 16                 1               17
                                        Goal 4: Preventing pollution and reducing
                                        risk in communities, homes, workplaces,
                                        and ecosystems                                                    29                13               42
                                        Goal 5: Better waste management,
                                        restoration of contaminated sites, and
                                        emergency response                                                39                 3               42
                                        Goal 6: Reduction of global and
                                        cross-border environmental risks                                  27                 9               36
                                        Goal 7: Expansion of Americans’ right to
                                        know about their environment                                      25                 6               31
                                        Goal 8: Sound science, improved
                                        understanding of environmental risk and
                                        greater innovation to address environmental
                                        problems                                                          29                 3               32
                                        Goal 9: A credible deterrent to pollution and
                                        greater compliance with the law                                   10                 8               18
                                        Goal 10: Effective management                                     21                 0               21
                                        Total                                                            286                71              357
                                        Source: EPA, Office of Planning, Analysis, and Accountability.



                                        As indicated by the relatively few environmental outcome measures
                                        developed to date, the development of performance measures is proving to
                                        be a difficult task for EPA. The challenges of developing outcome measures
                                        include (1) the difficulty of linking program activities to environmental
                                        outcomes, (2) a lack of baseline data against which to measure progress
                                        and a more generalized lack of reliable data about the environment, and
                                        (3) resource constraints for gathering and analyzing environmental data.12

                                        Linking program activities to environmental outcomes presents a major
                                        technical challenge, as we noted in a 1997 report on the analytical
                                        challenges in measuring performance.13 Changes in environmental

                                        12
                                          Such difficulties are similar to those we identified in a recent report on the efforts of EPA and the
                                        states to develop core performance measures under NEPPS. Environmental Protection: Collaborative
                                        EPA-State Effort Needed to Improve New Performance Partnership System (GAO/RCED-99-171,
                                        June 21, 1999).
                                        13
                                         Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138,
                                        May 30, 1997).



                                        Page 15                                          GAO/RCED-99-261 EPA Information Management
B-283313




conditions come about as a result of a complex web of factors, including
such variables as the weather or economic activity, many of which are out
of the control of EPA and its state partners. Likewise, it may take years for
some environmental outcomes to occur. Officials from EPA’s Office of Air
and Radiation told us, for example, that the outcomes, or environmental
results, of air quality programs may not be seen for many years, making it
particularly difficult to develop measures that assess annual performance
results. Some EPA officials raised concerns about being held accountable
under the Results Act for environmental outcomes that are largely out of
their control and indicated that they would rather be held accountable for
program outputs, over which they have more control.

The lack of appropriate environmental data for developing outcome
measures is tied to resource limitations and reporting burden.
Environmental measures can be costly to develop and use. Moreover,
some state officials mentioned that it is difficult to commit resources to
the development and implementation of new results-oriented performance
measures while still being held responsible for meeting other program
requirements. Also, the states have indicated to EPA that they are not
willing to collect more data for EPA’s needs if the data are not also needed
for managing their own programs and assessing environmental conditions
in their states. The states and EPA recently committed themselves to a joint
effort to look for opportunities to reduce reporting burden. They also
agreed to the principle that the data collected should support their ability
to measure programs’ success in a manner that increasingly is based on
environmental results.

The development of meaningful measures of environmental results is
intrinsically linked to the rest of EPA’s key information management
concerns—both the data quality issues (such as the need to improve data
accuracy, to increase data compatibility, and to reduce data gaps) and the
overarching issues facing the new information office (such as the call for a
reduction in the reporting burden and the need for effective partnerships
with the states). To date, EPA has made only limited progress toward
developing outcome measures. However, agency officials responsible for
designing EPA’s new information office have stated that developing
information on environmental results will be a part of the agency’s major
initiative to overhaul how it collects, manages, and disseminates
information. EPA has indicated that its forthcoming information plan will
articulate the central role that measures of results will play in helping the
agency meet its strategic goals and how efforts to develop such measures
will be fostered and encouraged. The magnitude of the difficulties involved



Page 16                             GAO/RCED-99-261 EPA Information Management
                         B-283313




                         in developing environmental outcome measures makes it likely that EPA
                         will continue to struggle with this issue for a long time and will need to
                         devote a great deal of attention to this effort if it is to make major
                         progress.


                         EPA’s information office will be responsible for improving the quality of
EPA’s New                data used within EPA and provided to the public and for developing and
Information Office       implementing the goals, standards, and accountability systems needed to
Faces Significant        bring about these improvements. To this end, the information office would
                         (1) ensure that the quality of data collected and used by EPA is known and
Management               appropriate for its intended uses, (2) reduce the burden on the states and
Challenges               regulated industries of collecting and reporting data, (3) fill significant
                         data gaps, and (4) provide the public with integrated information and
                         statistics on environmental and public health issues. The office will also
                         have the authority to set and oversee agencywide standards and policies
                         for managing information resources, including those governing the
                         purchase and operation of information technology systems. EPA estimates
                         that the new office will be operational in early October 1999.

                         Although the establishment of this office is an important step in improving
                         how EPA collects, manages, and disseminates information, the office will
                         face many challenges. As we reported in April 1999, developing a plan to
                         show how the agency intends to achieve its vision and goals is a pressing
                         need for the new office.14 Among other challenges facing the office are two
                         that have thwarted previous efforts by EPA to improve its information
                         management activities: (1) obtaining sufficient authority and resources to
                         address the complex information management issues facing the agency
                         and (2) working effectively with the states and regulated industries to
                         balance the demand for more data with the efforts to reduce the reporting
                         burden.


EPA Is Beginning to      EPA has begun work to develop an information strategic plan that it hopes
Develop an Information   will guide the agency’s information technology management and
Strategic Plan           investments on a multiyear basis. The new information office will have
                         leadership responsibility for creating this plan. As we, EPA’s Inspector
                         General, and others have pointed out, EPA has long needed—and long
                         lacked—such a plan for guiding its use of and investments in information
                         technology. Moreover, the Clinger-Cohen Act requires that agencies set

                         14
                          Environmental Protection: Status of EPA’s Efforts to Create a Central Information Office
                         (GAO/T-RCED-99-147, Apr. 13, 1999).



                         Page 17                                        GAO/RCED-99-261 EPA Information Management
                              B-283313




                              goals for improving the effectiveness of their operations through the use of
                              information technology and establish performance indicators to measure
                              how well information technology supports their programs. As we recently
                              reported, despite the agency’s having acknowledged the need to improve
                              data management as a mission-critical problem in its fiscal year 2000
                              performance plan, EPA did not set goals or timeframes for implementing
                              the information technology management requirements of the
                              Clinger-Cohen Act.15

                              The primary purpose of EPA’s information strategic plan will be to ensure
                              that the agency’s information technology will support the agency’s efforts
                              to meet its strategic goals as articulated in the strategic plan prepared
                              under the Results Act. As part of this effort, EPA will conduct various
                              analyses to determine its information needs and corresponding
                              investments in the information technology essential to carrying out its
                              mission and achieving its strategic goals. EPA officials responsible for
                              coordinating the development of the information plan recently set
                              preliminary milestones calling for “startup planning and team
                              organization” to begin in August 1999, with the bulk of fiscal year 2000
                              (beginning in October 1999) being used for assessing the current state of
                              information in the agency and deciding on information priorities for future
                              action.

                              In addition to or as part of this long-range plan to guide EPA’s information
                              priorities and investments, the information office will need a strategy and
                              an action plan that articulates the office’s priorities and the resources
                              needed to accomplish them. In particular, the office will need to articulate
                              its strategy to address the data quality problems—accuracy concerns, lack
                              of data compatibility, and data gaps—discussed in this report. Although
                              such a strategy or action plan is not yet in place, the Administrator has
                              identified 13 projects for the office that will receive early attention. Among
                              these projects, some of which are new while others are ongoing initiatives,
                              are the development of the data standards and the development of the
                              strategic information plan.


EPA’s New Information         The EPA Administrator and the senior-level officials charged with creating
Office Will Need Sufficient   the information office acknowledge that the reorganization will raise a
Authority and Resources       variety of complex information policy and technology issues. To date, the
                              focus has largely been on determining which organizational components


                              15
                               Observations on the Environmental Protection Agency’s Fiscal Year 2000 Performance Plan
                              (GAO/RCED-99-237R, July 31, 1999).



                              Page 18                                      GAO/RCED-99-261 EPA Information Management
B-283313




and staff members should be transferred into the office. While such
decisions are clearly important, EPA will also need to ensure that the office
has sufficient authority and resources to overcome organizational
obstacles that hindered previous attempts to adopt agencywide
information policies and a strategy for information resources
management. As EPA’s Chief Information Officer, the head of the
information office will be expected to provide accountability at a senior
management level for information technology issues agencywide and to
ensure greater accountability for delivering effective information
technology systems and services.

As we reported in September 1998, EPA has not developed agencywide
policies and procedures to govern key aspects of its projects to
disseminate information, nor has it developed standards to assess the
information’s accuracy and mechanisms to identify and correct errors.16
EPA recognizes the need for such agencywide policies governing
information collection, management, and dissemination, and the new
office will be responsible for developing them. However, EPA has a nearly
30-year history of operating in a decentralized fashion, with strong
program offices that are responsible for implementing different statutes,
such as the Clean Air Act and the Clean Water Act, with differing sets of
reporting requirements. Moreover, EPA’s program offices have historically
developed and managed their own information systems and made their
own decisions about disseminating information. For example, program
offices have been making their own, sometimes conflicting, decisions
about the types of information to be released and the extent of the
explanations needed about how information should be interpreted. The
new office will need to have the clear authority to develop such
procedures and policies and ensure that they are adhered to by the
program offices.

The office will also need significant resources and expertise to address the
information management challenges facing EPA. While the new
organizational structure will offer EPA an opportunity to better coordinate
and prioritize its information initiatives, the agency will also need to
determine whether its current information management resources,
including staff expertise, are sufficient for the office to achieve its goals.




16
 Environmental Information: Agencywide Policies and Procedures Are Needed for EPA’s Information
Dissemination (GAO/RCED-98-245, Sept. 24, 1998).



Page 19                                      GAO/RCED-99-261 EPA Information Management
                            B-283313




EPA Has Begun Working       In implementing environmental programs, EPA and the states have
With the States and         collected a wealth of environmental data under various statutory and
Regulated Industries to     regulatory authorities. However, EPA needs additional information on
                            environmental conditions and changes over time if it is to identify problem
Balance the Need for More   areas that are emerging or that need additional regulatory action or other
Data With the Efforts to    attention. In contrast to the need for more and better data is a call from
Reduce the Burden of Data   states and regulated industries to reduce the paperwork burden associated
Management and              with managing and reporting data.
Reporting
                            Overall reductions in the reporting burden have typically proven difficult
                            for EPA to achieve. For example, in March 1996, we reported that while EPA
                            was pursuing a paperwork reduction of 20 million hours, its overall
                            paperwork burden was actually increasing because of changes in
                            programs and other factors.17 EPA has continued to undertake a number of
                            activities to reduce the reporting and record keeping burden on regulated
                            entities. However, the Office of Management and Budget has reported that
                            these reductions have not yet been able to offset the growth in the
                            reporting burden resulting from new collections, which are frequently
                            associated with new rules needed to meet EPA’s statutory requirements.18
                            For fiscal year 1999, EPA has reported that the information collection
                            burden it imposed on the public would be about 120 million hours, a slight
                            increase over fiscal year 1998. The states and regulated industries have
                            indicated that they will look to EPA’s new information office to reduce the
                            burden of reporting requirements.

                            EPA has recently initiated some efforts to reduce the reporting burden. For
                            example, an EPA-state information management work group looking into
                            this issue has proposed an approach to assess environmental data and
                            reporting requirements based on the data’s value compared with the cost
                            of collecting, managing, and reporting it. EPA has announced that in the
                            coming months, its regional offices and the states will be exploring
                            possibilities for reducing paperwork requirements, testing initiatives in
                            consultation with EPA’s program offices, and establishing a clearinghouse
                            of successful initiatives and pilot projects.

                            Given that EPA depends on state regulatory agencies to collect much of the
                            data it needs and to help ensure the quality of that data, EPA acknowledges


                            17
                             Environmental Protection: Assessing EPA’s Progress in Paperwork Reduction (GAO/T-RCED-96-107,
                            Mar. 21, 1996).
                            18
                             Office of Information and Regulatory Affairs, Information Collection Budget of the United States
                            Government, Fiscal Year 1999, Office of Management and Budget (Washington, D.C.: OMB, Apr. 16,
                            1999), pp. 4, 14, and 147.



                            Page 20                                        GAO/RCED-99-261 EPA Information Management
                 B-283313




                 the need to work in a close partnership with the states on a wide variety of
                 information management activities, including the creation of its
                 information office. Some partnerships have already been created, such as
                 the effort discussed previously on reducing the reporting burden.
                 Similarly, EPA and its state partners in the NEPPS program are determining
                 what the appropriate data would be for developing environmental goals
                 and measures. Representatives of state environmental agencies and ECOS
                 have expressed their ideas and concerns about the role of the new
                 information office and have frequently reminded EPA that they expect to
                 share the responsibility for setting that office’s goals, priorities, and
                 strategies. According to an ECOS official, the states have had more input in
                 the development of the new EPA office than they typically have had in
                 other major policy issues, and the states view this change as an
                 improvement in their relationship with EPA.


                 The EPA Administrator’s decision to implement fundamental improvements
Conclusions      in the agency’s environmental information is an important step in the right
                 direction. However, additional actions are needed to resolve the complex
                 information management problems that have beset EPA since its inception.
                 To finish the job, EPA will need to follow up and expand its data
                 improvement initiatives to fill what the agency considers to be the key
                 gaps in its data, take advantage of opportunities to develop and implement
                 data standards to achieve compatibility among environmental databases,
                 and ensure the accuracy of its data. Such actions, which will require
                 several years to complete, are complex and will involve considerable
                 difficulties in obtaining the scientific and environmental data that are
                 needed, reaching agreements with key internal and external stakeholders
                 and EPA’s state regulatory partners on the appropriate environmental
                 measures of EPA’s programs, and ensuring that key databases are accurate
                 and compatible. Given these difficulties and the multiyear commitment
                 that will be needed to overcome them, an action plan detailing the
                 strategies, resources, benchmarks, and milestones for completing specific
                 actions would be useful to ensure that EPA’s information improvement
                 actions continue to receive appropriate attention within the agency and
                 that these actions can be monitored by the Congress and others.


                 To help EPA obtain the data it needs to effectively set priorities, assess
Recommendation   progress in achieving goals and objectives, and report on
                 accomplishments in a credible way, we recommend that the EPA
                 Administrator direct the program manager of the new information office to



                 Page 21                            GAO/RCED-99-261 EPA Information Management
                  B-283313




                  develop an action plan that details the key steps that the agency needs to
                  take to ensure that EPA’s environmental and regulatory data are sufficiently
                  complete, compatible, and accurate to meet its needs. This action plan
                  should include the office’s strategy, milestones, and resource needs to
                  (1) fill key gaps that have been identified in the agency’s information on
                  environmental conditions; (2) identify and develop all needed data
                  standards and implement them in all major databases; (3) coordinate EPA’s
                  data standardization efforts with the states, federal agencies, and other
                  organizations that maintain major environmental databases; (4) improve
                  the collection of accurate data by implementing its quality assurance
                  program throughout the agency as well as in the states; and (5) identify
                  procedures needed so that data errors detected in one EPA information
                  system can be corrected agencywide.


                  We provided copies of a draft of this report to EPA for review and
Agency Comments   comment. We also met with representatives of the EPA offices responsible
                  for the activities discussed in this report and with the agency’s Associate
                  Deputy Administrator. EPA said that the report accurately describes both
                  the agency’s problems with and efforts to improve information
                  management. EPA concurred with our recommendation and said that it is
                  consistent with the agency’s intentions concerning information
                  management. EPA also pointed out that its forthcoming information
                  strategic plan should provide the overall strategy needed to ensure the
                  completeness, compatibility, and accuracy of EPA’s environmental data.
                  The agency also offered several technical comments and clarifications,
                  which we incorporated as appropriate.


                  The scope and methodology for our work are discussed in appendix I. We
                  performed our work from December 1998 through August 1999 in
                  accordance with generally accepted government auditing standards.




                  Page 22                            GAO/RCED-99-261 EPA Information Management
B-283313




We will send copies of this report to the Honorable Carol Browner, EPA
Administrator, and to other interested parties. We will also make copies
available to others on request. Please call me at (202) 512-6111 if you or
your staff have any questions. Key contributors to this report are listed in
appendix III.




Peter F. Guerrero
Director, Environmental
  Protection Issues




Page 23                             GAO/RCED-99-261 EPA Information Management
Contents



Letter                                                                                           1


Appendix I                                                                                      26

Scope and
Methodology
Appendix II                                                                                     28

Major Databases
Covered by EPA’s REI
Action Plan
Appendix III                                                                                    30

GAO Contact and Staff
Acknowledgments
Bibliography                                                                                    31


Table                   Table 1: EPA’s Analysis of the Number and Type of Annual                15
                         Performance Measures for Its Strategic Goals for Fiscal Year
                         2000




                        Abbreviations

                        ECOS      Environmental Council of the States
                        EPA       Environmental Protection Agency
                        HUD       Department of Housing and Urban Development
                        NEPPS     National Environmental Performance Partnership System
                        REI       Reinventing Environmental Information
                        VA        Department of Veterans Affairs


                        Page 24                          GAO/RCED-99-261 EPA Information Management
Page 25   GAO/RCED-99-261 EPA Information Management
Appendix I

Scope and Methodology


             To identify the Environmental Protection Agency’s (EPA) initiatives to
             improve information management within the agency and between EPA and
             its various stakeholders, we held discussions with officials of the following
             EPA offices: the Office of Information Transition and Organizational
             Planning, which has responsibility for establishing a central information
             office to improve EPA’s data management and information sharing
             activities; the Office of Information Resources Management; the Center for
             Environmental Information and Statistics; the Reinvention Office; the
             Office of Planning, Analysis, and Accountability; and the Office of
             Research and Development. To determine the status of these initiatives,
             we conducted interviews and reviewed documents obtained from officials
             responsible for carrying out the projects. We also reviewed reports and
             written comments on the projects and needed improvements by EPA’s
             Science Advisory Board, National Advisory Council on Environmental
             Policy and Technology, and Office of Inspector General; the National
             Academy of Public Administration; the Center for Strategic and
             International Studies; Resources for the Future; and by other public
             interest and environmental organizations.

             We also discussed the projects and EPA’s overall efforts to improve
             environmental information management with officials of the
             Environmental Council of the States, a national nonpartisan, nonprofit
             association of state and territorial environmental commissioners that
             monitors and provides input on EPA’s improvement efforts and is working
             in partnership with EPA to improve their data sharing. In addition, we
             attended EPA-sponsored workshops and stakeholder meetings used to
             share information and solicit input on the agency’s new information office
             and on various efforts to improve the accuracy, compatibility, and
             completeness of EPA’s environmental data. We also attended a May 1999
             conference of environmental data users and held discussions with agency
             representatives involved in managing EPA’s data quality reforms and in
             making environmental information available to the public.

             To determine the status of EPA’s new central information office, including
             its organization, resources, duties, and responsibilities, we interviewed the
             director of the task force for the office’s information transition,
             operations, and planning, and task force members. We also interviewed
             the new office’s deputy national program manager designee; the director
             designees for its office of information technology services and office of
             planning, resources, and outreach; and the director and deputy director
             designees for its office of information collection. The new office’s national
             program manager had not been designated at the time of our review.



             Page 26                             GAO/RCED-99-261 EPA Information Management
Appendix I
Scope and Methodology




We conducted our work from December 1998 through August 1999 in
accordance with generally accepted government auditing standards.




Page 27                          GAO/RCED-99-261 EPA Information Management
Appendix II

Major Databases Covered by EPA’s REI
Action Plan


              Database                                      Function
              Air programs
              Aerometric Information Retrieval System/Air   Contains monitoring data on ambient air
              Quality Subsystem (AIRS/AQS)                  quality from about 10,000 stations around
                                                            the country
              AIRS Facility Subsystem (AFS)                 Contains data on emissions, compliance,
                                                            and permits for nearly 150,000 stationary
                                                            air pollution sources
              Risk Management Plan Information System       Scheduled to become operational in 1999;
              (RMP*Info)                                    will contain all Risk Management Program
                                                            data (excluding off-site consequence
                                                            analysis) for facilities
              Water programs
              Permit Compliance System (PCS)                Contains National Pollutant Discharge
                                                            Elimination System data; tracks permits
                                                            and monitoring data for more than 64,000
                                                            facilities
              Safe Drinking Water Information System        Contains information on the nation’s
              (SDWIS)                                       drinking water, including sampling data
                                                            and information on noncompliance with
                                                            regulatory standards
              Water Quality Information System              Contains monitoring data on ambient water
              (STORET X)                                    quality and biological samples from over
                                                            850,000 stations across the nation
              Land disposal programs
              RCRA Information System (RCRIS)               Contains identification and location data
                                                            for all hazardous waste handlers and
                                                            tracks permits, site closure status,
                                                            compliance, and cleanup activities
              Biennial Reporting System (BRS)               Contains data on the generation of
                                                            hazardous waste from large-quantity
                                                            generators and data on waste
                                                            management practices from treatment,
                                                            storage, and disposal facilities
              CERCLIS Information System (CERCLIS 3)        Contains information on hazardous waste
                                                            sites, including data on site inspections,
                                                            preliminary assessments, and remediation
              Public awareness programs
              Toxic Release Inventory System (TRIS)         Contains data from industry on the release
                                                            of over 300 toxic chemicals into the air,
                                                            water, and land
              Envirofacts Data Warehouse (EF)               Provides a single point of access to data
                                                            from seven major EPA databases
                                                                                            (continued)




              Page 28                                  GAO/RCED-99-261 EPA Information Management
Appendix II
Major Databases Covered by EPA’s REI
Action Plan




Database                                        Function
Enforcement programs
OECA Docket (Docket)                            Tracks and reports information on civil
                                                judicial and administrative enforcement
                                                cases brought under the authority of
                                                environmental statutes
National Compliance Database (NCDB)             Tracks compliance monitoring and
                                                enforcement activities for the Pesticides
                                                and Toxic Substances Compliance and
                                                Enforcement Program

Source: Based on “Burden Reduction and State Environmental Agencies,” Environmental
Council of the States, 1999.




Page 29                                     GAO/RCED-99-261 EPA Information Management
Appendix III

GAO Contact and Staff Acknowledgments


                  Edward A. Kratzer, (202) 512-6553
Contact
                  In addition to the individual named above, Susan E. Swearingen, J.
Acknowledgments   Kenneth McDowell, Donald E. Pless, and John A. Crossen made key
                  contributions to this report.




                  Page 30                             GAO/RCED-99-261 EPA Information Management
Bibliography


                National Advisory Council for Environmental Policy and Technology.
Environmental   Managing Information as a Strategic Resource: Final Report and
Information     Recommendations of the Information Impacts Committee. EPA
                100-R-98-002. Washington, D.C.: Jan. 1998.

                _____. Using Information Strategically to Protect Human Health and the
                Environment: Recommendations for Comprehensive Information
                Resources Management. IRM Strategic Planning Task Force.
                Environmental Information and Assessment Committee. EPA 270-K-94-002.
                Washington, D.C.: Aug. 1994.

                Office of Information and Regulatory Affairs. Information Collection
                Budget of the United States Government, Fiscal Year 1999. Office of
                Management and Budget. Washington, D.C.: Apr. 16, 1999.

                Science Advisory Board. Review of the Center for Environmental
                Information and Statistics’ (CEIS) Draft Data Suitability Assessment of
                Major EPA Databases. EPA-SAB-EC-99-010. Washington, D.C.: Feb. 19, 1999.

                Cooper, Mary H. “The Cleanup’s Next Phase: Setting Environmental
                Priorities for the 21st Century.” CQ Outlook, June 5, 1999.

                U.S. Environmental Protection Agency. Information Resources
                Management: Office of Water Data Integration Efforts. Office of Inspector
                General. E1NWG6-15-0001-8100177. Washington, D.C.: June 22, 1998.

                _____. Special Review of EPA’s Information Systems Program. Vol. 1. Office
                of Inspector General. E1SKG3-15-0098-4400038. Washington, D.C.: Mar. 24,
                1994.

                U.S. General Accounting Office. Environmental Enforcement: EPA Needs a
                Better Strategy to Manage Its Cross-Media Information. GAO/IMTEC-92-14.
                Apr. 2, 1992.

                _____. Environmental Information: Agencywide Policies and Procedures
                Are Needed for EPA’s Information Dissemination. GAO/RCED-98-245. Sept. 24,
                1998.

                _____. Environmental Information: EPA Could Better Address Concerns
                About Disseminating Sensitive Business Information. GAO/RCED-99-156.
                June 25, 1999.




                Page 31                            GAO/RCED-99-261 EPA Information Management
Bibliography




_____. Environmental Protection: EPA Faces Formidable Challenges
Managing Water Quality Data. GAO/T-AIMD-93-2. Aug. 5, 1993.

_____. Environmental Protection: EPA’s Actions to Improve Long-Standing
Information Management Weaknesses. GAO/T-IMTEC-93-4. Mar. 29, 1993.

_____. Environmental Protection: EPA’s Plans to Improve Long-Standing
Information Resources Management Problems. GAO/AIMD-93-8. Sept. 16,
1993.

_____. Environmental Protection: EPA’s Problems With Collection and
Management of Scientific Data and Its Efforts to Address Them.
GAO/T-RCED-95-174. May 12, 1995.


_____. Environmental Protection: Status of EPA’s Efforts to Create a
Central Information Office. GAO/T-RCED-99-147. Apr. 13, 1999.

_____. EPA Toxic Substances Program: Long-Standing Information
Planning Problems Must Be Addressed. GAO/AIMD-94-25. Nov. 17, 1993.

_____. Hazardous Waste: Benefits of EPA’s Information System Are
Limited. GAO/AIMD-95-167. Aug. 22, 1995.

_____. Pesticides: EPA’s Efforts to Collect and Take Action on Exposure
Incident Data. GAO/RCED-95-163. July 12, 1995.

_____. Pesticides: Information Systems Improvements Essential for EPA’s
Reregistration Efforts. GAO/IMTEC-93-5. Nov. 23, 1992.

_____. Toxic Substances Control Act: Legislative Changes Could Make the
Act More Effective. GAO/RCED-94-103. Sept. 26, 1994.

_____. Toxic Substances: EPA Needs More Reliable Source Reduction Data
and Progress Measures. GAO/RCED-94-93. Sept. 23, 1994.

_____. Toxic Substances: EPA Should Focus Its Chemical Use Inventory on
Suspected Harmful Substances. GAO/RCED-95-165. July 7, 1995.

_____. Waste Minimization: Major Problems of Data Reliability and Validity
Identified. GAO/PEMD-92-16. Mar. 23, 1992.




Page 32                            GAO/RCED-99-261 EPA Information Management
                 Bibliography




                 Center for Strategic and International Studies. The Environmental
EPA Management   Protection System in Transition: Toward a More Desirable Future. Final
                 report of Enterprise for the Environment. Washington, D.C.: Jan. 1998.

                 National Academy of Public Administration. Resolving the Paradox of
                 Environmental Protection: An Agenda for Congress, EPA, and the States.
                 Washington, D.C.: Sept. 1997.

                 _____. Setting Priorities, Getting Results: A New Direction for EPA.
                 Washington, D.C.: Apr. 1995.

                 National Performance Review. Reinventing Environmental Regulation.
                 Washington, D.C.: Mar. 16, 1995.

                 Resources for the Future. Regulating Pollution: Does the U.S. System
                 Work? Washington, D.C.: 1997.

                 Science Advisory Board. Review of the Agency-Wide Quality Management
                 Program. EPA-SAB-EEC-LTR-98-003. Washington, D.C.: July 24, 1998.

                 _____. Review of the Implementation of the Agency-Wide Quality System.
                 EPA-SAB-EEC-LTR-99-002. Washington, D.C.: Feb. 25, 1999.


                 U.S. Environmental Protection Agency. Superfund: EPA Had Not
                 Effectively Implemented Its Superfund Quality Assurance Program. Office
                 of Inspector General. E1SKF7-08-0011-8100240. Washington, D.C.: Sept. 30,
                 1998.

                 U.S. General Accounting Office. Environmental Protection Agency:
                 Protecting Human Health and the Environment Through Improved
                 Management. GAO/RCED-88-101. Aug. 16, 1988.

                 _____. Environmental Protection: Assessing EPA’s Progress in Paperwork
                 Reduction. GAO/T-RCED-96-107. Mar. 21, 1996.

                 _____. Environmental Protection: Key Management Issues Facing EPA.
                 GAO/RCED-98-153R. Apr. 23, 1998.


                 _____. Major Management Challenges and Program Risks: Environmental
                 Protection Agency. GAO/OCG-99-17. Jan. 1999.




                 Page 33                             GAO/RCED-99-261 EPA Information Management
                    Bibliography




                    _____. Managing for Results: Analytic Challenges in Measuring
                    Performance. GAO/HEHS/GGD-97-138. May 30, 1997.

                    _____. Managing for Results: EPA’s Efforts to Implement Needed
                    Management Systems and Processes. GAO/T-RCED-97-116. Apr. 8, 1997.

                    _____. Managing for Results: EPA’s Efforts to Implement Needed
                    Management Systems and Processes. GAO/RCED-97-156. June 18, 1997.

                    _____. Observations on the Environmental Protection Agency’s Fiscal Year
                    2000 Performance Plan. GAO/RCED-99-237R. July 31, 1999.

                    _____. Paperwork Reduction: Burden Reduction Goal Unlikely to Be Met.
                    GAO/T-GGD/RCED-96-186. June 5, 1996.


                    _____. Paperwork Reduction: Government Goals Unlikely to Be Met.
                    GAO/T-GGD-97-114. June 4, 1997.


                    _____. Performance Plans: Selected Approaches for Verification and
                    Validation of Agency Performance Information. GAO/GGD-99-139. July 30,
                    1999.

                    _____. Results Act: Observations on EPA’s Draft Strategic Plan.
                    GAO/RCED-97-209R. July 30, 1997.



                    Environmental Council of the States. Burden Reduction and State
EPA-State Working   Environmental Agencies. Washington, D.C.: 1999.
Relationship
                    U.S. General Accounting Office. Environmental Protection: Collaborative
                    EPA-StateEffort Needed to Improve New Performance Partnership System.
                    GAO/RCED-99-171. June 21, 1999.


                    _____. Environmental Protection: EPA’s and States’ Efforts to Focus State
                    Enforcement Program on Results. GAO/RCED-98-113. May 27, 1998.

                    _____. Environmental Protection: EPA’s and States’ Efforts to Focus State
                    Enforcement Programs on Results. GAO/T-RCED-98-233. June 23, 1998.

                    _____. Environmental Protection: EPA’s and States’ Efforts to “Reinvent”
                    Environmental Regulation. GAO/T-RCED-98-33. Nov. 4, 1997.




                    Page 34                            GAO/RCED-99-261 EPA Information Management
           Bibliography




           _____. Environmental Protection: Status of EPA’s Initiatives to Create a
           New Partnership With States. GAO/T-RCED-96-87. Feb. 29, 1996.

           _____. EPA and the States: Environmental Challenges Require a Better
           Working Relationship. GAO/RCED-95-64. Apr. 3, 1995.




(160461)   Page 35                             GAO/RCED-99-261 EPA Information Management
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested