oversight

Managing for Results: EPA's Efforts to Implement Needed Management Systems and Processes

Published by the Government Accountability Office on 1997-06-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Requesters




June 1997
                  MANAGING FOR
                  RESULTS
                  EPA’s Efforts to
                  Implement Needed
                  Management Systems
                  and Processes




GAO/RCED-97-156
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-276608

      June 18, 1997

      The Honorable Christopher S. Bond
      Chairman, Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      United States Senate

      The Honorable Jerry Lewis
      Chairman, Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      House of Representatives

      For over a decade, internal and external studies have called for the
      Environmental Protection Agency (EPA) to “manage for environmental
      results” as a way to both improve and better account for its performance.
      To implement this type of management, EPA must establish goals and
      objectives identifying the improvements in environmental conditions that
      it is seeking, develop strategies and allocate resources to achieve these
      goals and objectives, and assess and report the results achieved. A 1995
      study by the National Academy of Public Administration (NAPA)
      recommended a series of steps for EPA to take to manage for results,
      including actions to improve and integrate its processes for planning,
      budgeting, and ensuring accountability.1

      In March 1996, EPA responded to NAPA’s recommendations by announcing
      its plans to create a new office that would develop and implement an
      integrated planning, budgeting, and accountability system for the agency.
      Because of your continued interest in how EPA sets its priorities and plans
      its environmental activities, you asked us to review the status of EPA’s
      efforts to (1) establish the new office and develop and implement an
      integrated system and (2) ensure that the agency has comprehensive
      scientific and environmental data and appropriate environmental
      measures of progress for use in strategic planning, budgeting, and
      accountability.2 In addition, you asked us to discuss EPA’s previous efforts
      to implement this type of system and identify elements of an effective


      1
       Setting Priorities, Getting Results: A New Direction for EPA, NAPA (Apr. 1995).
      2
       The preliminary results of our review of these issues were reported to the Subcommittees on VA,
      HUD, and Independent Agencies, Senate and House Committees on Appropriations in Managing for
      Results: EPA’s Efforts to Implement Needed Management Systems and Processes
      (GAO/T-RCED-97-116, Apr. 8, 1997, and GAO/T-RCED-97-133, Apr. 15, 1997).



      Page 1               GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                   B-276608




                   system that could be used as benchmarks to gauge progress in this current
                   effort.


                   In January 1997, the EPA Administrator approved the organizational
Results in Brief   structure and staffing for a new office—called the Office of Planning,
                   Analysis, and Accountability—which is responsible for developing and
                   implementing an integrated planning, budgeting, and accountability
                   system. Earlier, in March 1996, EPA had established a work group,
                   composed of employees on temporary assignment, to begin developing the
                   new system. Although the work group was never fully staffed and made
                   limited progress in developing the system, it developed a conceptual
                   framework that the new office is beginning to implement. Members of the
                   work group were temporarily assigned to the new office, which will not be
                   fully staffed before July 1997.

                   EPA faces long-term challenges to obtain the scientific and environmental
                   information needed to fully support its new system. Although much
                   scientific and environmental information has already been collected, many
                   gaps exist, and the data are often difficult to compile because different
                   data collection methods have been used. Likewise, much effort is still
                   needed to identify, develop, and reach agreement on a comprehensive set
                   of environmental measures to link the agency’s activities to changes in
                   human health and environmental conditions. Currently, EPA has to rely
                   mainly on administrative measures, such as the number of permits issued
                   or inspections made, to measure its performance or success.

                   EPA has been trying since the early 1970s to revise its management systems
                   to better manage for results. In the early 1990s, for example, EPA
                   announced a strategic planning, budgeting, and accountability effort that
                   was very similar to the system being proposed today. However, such
                   efforts proved to be difficult and fell short of EPA’s expectations. EPA’s
                   current plans are no less complex and challenging. Given this complexity,
                   we identified various elements of an effective planning, budgeting, and
                   accountability system that EPA can use as benchmarks to monitor its
                   progress as it works to implement its plans for an integrated system. These
                   elements describe organizational, informational, and procedural
                   capabilities needed to improve and integrate EPA’s management processes.
                   For example, EPA needs comprehensive data on health and environmental
                   risks and a process to compare these risks across environmental problems
                   and programs. EPA also needs performance partnership agreements with
                   the states to set out expectations for their contributions to meeting



                   Page 2         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
             B-276608




             national environmental and agencywide goals and to specify measures for
             assessing their performance.


             NAPA’s April 1995 report recommended, among other things, that EPA
Background   establish specific environmental goals and strategies to attain them, use
             comparative risk analyses to inform the selection of priorities and the
             development of specific program strategies, consolidate the planning and
             budgeting functions, use the budget process to allocate resources to the
             agency’s priorities, and establish accountability by setting and tracking
             benchmarks and evaluating performance.

             NAPA’s recommendations on setting goals and objectives and ensuring
             accountability for meeting them are similar to the requirements for federal
             agencies established by the Government Performance and Results Act of
             1993 (GPRA). Under GPRA, agencies are to develop strategic plans by
             September 30, 1997. In addition, agencies are to submit to the Office of
             Management and Budget annual performance plans, including annual
             performance goals and performance measures. The first annual
             performance plans are to be submitted in the fall of 1997 for fiscal year
             1999. Agencies are also required to submit annual performance reports to
             the President and the Congress on the extent to which their programs are
             meeting annual performance goals. The act requires the first performance
             reports, which are to cover fiscal year 1999, by March 31, 2000. In
             establishing these planning and performance reporting requirements, GPRA
             did not specify the management processes that the individual agencies are
             to use to implement them.

             In response to NAPA’s recommendations, EPA created a task force in July
             1995 to identify ways to improve and better integrate its processes for
             planning, budgeting, and ensuring accountability. The task force’s
             February 1996 report recommended a major restructuring of these
             processes into a “strong, centrally managed” system. The recommended
             system was envisioned to provide for (1) the more consistent and
             comprehensive use of science and analysis in setting the agency’s
             priorities and (2) greater management and fiscal accountability for
             implementing programs and activities according to budgets and operating
             plans.

             In March 1996, the EPA Administrator and Deputy Administrator directed
             the agency to implement the task force’s recommendations. They
             announced plans to create a new office by January 1997 to develop and



             Page 3         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                          B-276608




                          implement the integrated planning, budgeting, and accountability system
                          proposed by the task force. In the interim, EPA established the Planning,
                          Budgeting, Accountability, and Analysis Work Group composed of
                          employees on temporary assignment, to begin to develop the new system.


                          Although approved by the Administrator in January 1997, EPA’s new Office
EPA’s New Office Is       of Planning, Analysis, and Accountability will not be fully staffed until July
Being Formed to           1997. The Planning, Budgeting, Accountability, and Analysis Work Group
Develop an Integrated     began to develop the new system in March 1996, and its staff has been
                          detailed to the new office until permanent employees are selected.
System                    However, the work group was not fully staffed, and much of its time was
                          devoted to preparing the strategic plan due under GPRA by September 30,
                          1997. Given this level of staffing and focus on meeting the requirements of
                          GPRA, EPA has made progress in designing an integrated system but has not
                          yet implemented all the needed processes and procedures.


Staffing the New Office   The Office of Planning, Analysis, and Accountability, which is located in
                          the Office of the Chief Financial Officer, is authorized 47 employees.3 (An
                          organizational chart for the Office of the Chief Financial Officer is shown
                          in fig. 1.)




                          3
                           As part of the reorganization creating the Office of Planning, Analysis, and Accountability, EPA’s
                          budget and financial divisions in the Office of Administration and Resource Management were moved
                          to the Office of the Chief Financial Officer. The Budget Division was made responsible for both the
                          annual planning and budgeting processes because of their close link under EPA’s new system. The
                          Chief Financial Officer reports to the Administrator.



                          Page 4               GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                                            B-276608




Figure 1: Organizational Chart for EPA’s Office of the Chief Financial Officer



                                            Administrator
                                        Deputy Administrator




                                       Chief Financial Officer
                                           Deputy CFO




          Office of Planning,                                             Office of the
            Analysis, and                                                 Comptroller
            Accountability



                                                                          Financial                   Financial Services
                                       Annual Planning and
                                                                         Management                        Division
                                         Budget Division
                                                                           Division



            Analysis staff




            Planning staff




          Accountability staff




                                            Page 5             GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                           B-276608




                           As of May 1997, EPA had filled 17 of the new positions—16 from within EPA
                           and 1 from outside the agency. The remaining positions are anticipated to
                           be filled by July 1997. The Acting Chief Financial Officer told us that
                           staffing could not begin until the Administrator had approved the plans for
                           the office. In addition, time is required to select staff using a competitive
                           selection process.

                           After its formation in March 1996, the Planning, Budgeting, Accountability,
                           and Analysis Work Group began to develop the new planning, budgeting,
                           and accountability system. The work group’s staff, who were detailed to
                           the Office of Planning, Analysis, and Accountability until the permanent
                           staff could be selected, have devoted much of their effort to preparing the
                           strategic plan that GPRA requires by September 30, 1997. Although the work
                           group was to have 40 to 50 temporary employees assigned from other
                           parts of EPA, it has had no more than 27 employees assigned at any one
                           time. Currently, 10 of the work group’s members remain on detail to the
                           Office of Planning, Analysis, and Accountability. According to the Acting
                           Chief Financial Officer, the work group had to compete with other
                           high-priority activities for staff.


Developing an Integrated   Because of the work group’s staffing level and focus on meeting GPRA’s
System                     deadlines, EPA is in the early stages of developing an integrated system. EPA
                           has a conceptual framework for the new system and has begun to develop
                           the individual components. The agency is reviewing its former
                           accountability process to find out what did and did not work well,
                           contacting other federal agencies to determine how they account for
                           progress in meeting their goals, and examining reporting systems in the
                           agency’s program offices to identify their potential use in the new system.
                           EPA hopes to have the accountability component in place by
                           September 1999.

                           EPA is also preparing the strategic plan that is to be completed in
                           September 1997. On April 25, 1997, the agency provided the Congress with
                           a detailed outline of the plan, including a draft of its mission statement and
                           goals. It provided its draft objectives on May 21, 1997. The Congress is
                           scheduled to receive a complete draft of the plan for review and
                           consultation in July 1997. Once the strategic plan is completed, the
                           development of the new budgeting component will begin. According to EPA
                           officials, the agency plans to seek approval from the Office of Management
                           and Budget and the Congress to restructure its fiscal year 1999 budget
                           along the lines of the goals in the strategic plan. (A more detailed



                           Page 6          GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                         B-276608




                         discussion of the framework and the status of EPA’s implementation of the
                         system is contained in app. I.)


                         EPA’s new planning, budgeting, and accountability system depends on the
EPA Faces                availability of credible information to identify environmental problems,
Information              establish priorities among competing needs, develop effective strategies to
Challenges to            achieve goals and objectives, and assess progress or performance in
                         meeting the goals and objectives. Although EPA continues to make
Implement Its New        improvements, the environmental data that it compiles have substantial
System                   shortcomings, including gaps and inconsistencies in quality. In addition,
                         the agency’s information systems are not integrated.

                         To evaluate the impact of its programs on the environment and determine
                         whether they are achieving the desired results, EPA will need a
                         comprehensive set of environmental measures. Although the agency’s
                         efforts to develop such measures have been valuable, focusing these and
                         future efforts could enable EPA to more effectively address scientific and
                         technical obstacles and reach agreement with the states on the measures
                         that will be used to assess their performance in carrying out
                         environmental programs at the local level.


Shortcomings in EPA’s    The need to assess EPA’s performance in terms of changes in
Environmental Data and   environmental conditions substantially increases the demand for
Systems                  high-quality environmental data. Such data are also needed to identify and
                         respond to emerging problems before significant damage is done to the
                         environment. Although EPA has tried to improve the quality of its data,
                         these data are often unreliable, and the agency’s many disparate
                         information systems are not integrated. These shortcomings have been
                         raised in various external and internal reports on EPA, including the Vice
                         President’s report on reinventing government.4

                         In its April 1995 report, NAPA identified the lack of high-quality data on
                         environmental conditions as a particularly important problem for EPA. NAPA
                         specifically noted the limited amount of information based on the
                         real-time monitoring of environmental conditions. NAPA also concluded
                         that much remains to be done to improve the agency’s overall
                         management of environmental information. It noted that EPA had
                         numerous information systems and that the agency’s program offices,
                         which are responsible for their own data, use different methods and

                         4
                          Reinventing Environmental Regulation, National Performance Review (Mar. 16, 1995).



                         Page 7              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                         B-276608




                         definitions to gather data. In addition, EPA relies on data compiled by other
                         federal agencies and the states. According to NAPA, these agencies and the
                         states also use divergent methods of collecting data.

                         More recently, a 1996 EPA report concluded that the agency needs to
                         redesign its many disparate fiscal and environmental data systems so that
                         it and others can measure its success in meeting environmental goals and
                         can determine the costs of doing so.5 The difficulty EPA has had in
                         demonstrating its performance or the impact of its actions is illustrated by
                         the findings of a team of agency personnel, which was formed in 1995 to
                         evaluate the agency’s needs for environmental information. The team
                         identified various problems with the information needed to report on
                         environmental goals, such as gaps in the data and inconsistencies in the
                         methods of collecting and/or reporting data across states or federal
                         agencies. Specific examples include the lack of (1) national reporting on
                         risk reduction at waste sites, (2) reliable data on the nature and cause of
                         pesticide poisonings, (3) effective reporting on progress in improving the
                         nation’s water quality, and (4) complete data on air pollutants.6


Need to Focus EPA’s      Although EPA and state officials recognize that measures of activities are
Efforts to Develop       still needed, they believe that environmental measures are more useful for
Environmental Measures   assessing programs’ performance and for informing the public about
                         environmental conditions. However, they understand that scientific and
                         technical issues have to be addressed before EPA can widely use indicators
                         of environmental conditions and trends. Greater use of these measures
                         also depends on agreement with the states—EPA’s partners in
                         implementing environmental programs—specifying the measures that will
                         be used to assess their performance.

                         EPA and the states have various efforts under way to develop and use
                         environmental measures. According to EPA officials, these efforts have
                         been valuable but disparate. Furthermore, at a conference convened by
                         EPA in September 1996 to better coordinate these efforts, as well as in
                         interviews conducted by EPA staff to prepare for the conference, regional
                         and state representatives cited several concerns. They said, for example,
                         that (1) clarification is needed on EPA’s and the states’ direction in


                         5
                          Managing for Results, EPA’s Planning, Budgeting, and Accountability Task Force (Feb. 23, 1996).
                         6
                          The Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 establish an investment
                         framework for having agencies better plan and manage their technology efforts and link these efforts
                         directly to the achievement of their programs’ goals and missions. If successfully implemented, these
                         acts hold a key to ensuring that agencies put systems in place to collect needed performance data.



                         Page 8               GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
B-276608




developing indicators; (2) some qualities of a good indicator are not well
understood; and (3) in some cases, determining whether the best
indicators have been chosen will take many years. The representatives
also believed that the data and resources needed to develop and use
environmental indicators are inadequate. Focusing these disparate efforts
could enable EPA to better address (1) scientific and technical issues
associated with developing measures and (2) the states’ concerns about
the appropriate measures to use.

A major technical challenge in developing and using environmental
measures is to be able to scientifically establish a direct cause-and-effect
relationship between a program’s activities and changes in environmental
conditions. This relationship is especially difficult to establish because
factors beyond a program’s control, such as changes in weather patterns
and economic conditions, can substantially affect environmental
conditions. Furthermore, the data needed to understand these
relationships can be extensive. Another challenge is having sufficient
monitoring data available to document trends or changes in the
environment. In many cases, data or indicators are not available for a
specific aspect of the environment because of high costs or technical
difficulties. Thus, it will be some time before EPA is able to develop and use
a set of environmental indicators that accurately and comprehensively
reflect the impact of its programs or their results.7

Reaching agreement within EPA and among the states and the agency’s
other stakeholders on the specific environmental indicators to use will
also be a challenge. A consensus may be difficult to achieve because of the
potential for disagreement about what should be measured and whether a
relatively small number of measures can adequately reflect the effects of
an agency’s or a program’s activities. For example, EPA will need a set of
measures common to all the states to report to the Congress and the
public on the agency’s performance and the condition of the nation’s
environment. At the same time, the development of national measures, to
the extent that such measures drive the implementation of environmental
programs, will reduce the states’ flexibility to tailor the programs to meet
local needs and conditions, a major concern of the states. Reporting on
new measures will also increase the states’ costs unless other reporting
requirements are eliminated or reduced.


7
 Our report Managing for Results: Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997) discusses in more detail the analytic and technical challenges
that EPA and other federal agencies are experiencing as they try to measure their programs’
performance under GPRA.



Page 9              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
B-276608




EPA is working with the states to develop a set of core performance
measures, including some environmental indicators, that the agency’s
regional offices are to use in negotiating annual work plans and
performance partnership agreements with the states.8 The core measures
are to be focused and limited in number, representing measurable
priorities for each of EPA’s national program managers. They are to serve
as the minimum measures in performance agreements with the states,
which may develop additional measures to represent their own
environmental or programmatic issues. In addition, a particular core
measure may not be required if a state can demonstrate that the measure
does not apply or cannot be addressed.

According to EPA, its national program managers finalized their core
measures in April 1997, and its regional staff began negotiations with the
states to incorporate these measures into the performance partnership
agreements for fiscal year 1998. However, because of various state
concerns, the core measures were issued as interim measures and
stamped as “draft” to permit work to continue on the measures while
negotiations on the partnership agreements take place. For example, the
environmental commissioners of the New England states have expressed
to EPA their opposition to making any national indicator or core
performance measure mandatory. It is too soon to know how extensively
EPA’s regional offices will be negotiating measures that reflect the direct
effects of programs on human health and the environment.

According to Office of Planning, Analysis, and Accountability officials,
additional efforts to develop environmental measures are needed, but the
key is to find the right balance of environmental and activity measures.
The officials said that environmental measures can be costly to develop
and use because they require so much environmental information. In their
view, activity measures may be the only available or appropriate measures
for some programs or activities.




8
 EPA enters into performance partnership agreements with states that are participating in the National
Environmental Performance Partnership System. This new system of federal oversight for state
environmental programs, which is being established under a 1995 agreement with state leaders, places
greater emphasis on the use of environmental goals and performance indicators and provides
opportunities for reducing EPA’s oversight of state programs that exhibit high performance in certain
areas. About half the states have signed performance partnership agreements to participate in the
system.



Page 10              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                       B-276608




                       For EPA, revising its management processes to implement an integrated
Previous EPA Efforts   planning, budgeting, and accountability system that will enable it to
Suggest That           manage for environmental results is not easy. According to EPA officials,
Expectations or        implementing the system as currently planned is a major challenge that
                       could take several years to complete. This is also not the first time that the
Benchmarks Are         agency has tried to develop and implement such a system. EPA’s attempts
Needed to Guide        to develop and use environmental measures go back to 1974, and when we
                       performed our general management review of the agency in 1988, we
System’s               reported that one of its major management goals emphasized managing for
Implementation         measurable environmental results.9 In a March 1990 response to our
                       report, EPA said that the new Administrator had made managing for
                       environmental results a major policy and operational focus for the agency
                       and was initiating a strategic planning, budgeting, and accountability effort
                       to improve the agency’s ability to protect human health and the
                       environment. The goals of that effort were very similar to those of the
                       system being proposed today.

                       Given the complexity of the task and time likely needed to fully implement
                       the proposed new system, it is important for EPA to identify the processes,
                       organization, and information that the system will need to work effectively
                       when implemented. These elements can be used as benchmarks to help
                       guide the development of the system and to monitor the progress of its
                       implementation. Over time, support for fully implementing the system may
                       be eroded by personnel changes in top management or other key positions
                       or by constraints on resources.

                       The major elements of an effective planning, budgeting, and accountability
                       system for EPA that we identified are shown in appendix II. We identified
                       these key elements through our prior reviews of EPA’s management and
                       programs, NAPA’s report on EPA, and various other studies, including the
                       EPA task force’s study recommending the conceptual framework that the
                       agency is currently implementing. We also considered the requirements of
                       GPRA in developing the system’s elements. While GPRA requires federal
                       agencies to set goals, measure performance, and report on results, it does
                       not discuss the characteristics of the management processes needed to
                       meet these requirements. Thus, the specific elements that we developed
                       supplement GPRA’s general management framework.

                       We also obtained and incorporated comments on our descriptions of the
                       key elements from four former EPA officials who were responsible for

                       9
                       Environmental Protection: Protecting Human Health and the Environment Through Improved
                       Management (GAO/RCED-88-101, Aug. 16, 1988).



                       Page 11            GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
B-276608




planning, budgeting, and/or ensuring accountability while at the agency.10
These former officials generally agreed with the key elements but
suggested various additions and clarifications. We also obtained and
incorporated comments from representatives of NAPA and EPA’s Science
Advisory Board. The Board, which was established in the Office of the
Administrator to provide advice on scientific matters, has evaluated
various aspects of EPA’s priority-setting and other management systems
that make use of scientific and technical information and analyses.

As shown in appendix II, we identified key elements for each of the
planning, budgeting, and accountability components of an effective
system. These elements describe the organizational, informational, and
procedural capabilities that EPA needs to improve and integrate its
management processes. For example, EPA needs to be able to establish
priorities on the basis of human health and environmental risks. Key
elements of the planning component would include (1) comprehensive
data on risks and (2) a process to compare these risks across
environmental problems and the agency’s programs. Similarly, the agency
needs to work with the states as partners in carrying out environmental
programs. Hence, performance partnership agreements with the states are
part of the agency’s accountability component. These agreements would
set out expectations for the states’ contributions to meeting national
environmental and agencywide goals and would specify measures for
assessing the states’ performance.

In addition to the planning, budgeting, and accountability elements, we
included the elements of an effective analysis component to reflect the
importance of information and analytical capability to the system’s
effectiveness. For example, one of the elements of an effective central
planning, budgeting, and accountability office is an analysis function to
independently compile and assess relevant information on health and
environmental risks and on the performance of programs across the
agency. Our list of elements for the analysis component describes the
environmental, risk, and performance information needed to make
decisions about planning, budgeting, and accountability.

In total, the key elements that we identified would provide EPA with a
structured approach for establishing direction, setting priorities, and
measuring performance. They would aim to ensure that the agency (1) has
adequate environmental, risk, and performance data to support


10
  These former EPA officials include a deputy administrator and three assistant administrators—two
for policy, planning, and evaluation and one for administration and resource management.



Page 12              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                  B-276608




                  decisionmakers and to inform stakeholders and (2) can link its mission
                  and priorities to programs’ actions by setting and reporting on its progress
                  toward accomplishing measurable goals and objectives. Thus, the
                  elements would be consistent with the plans for EPA’s new system and the
                  requirements of GPRA.


                  EPA is likely to need several years to develop and fully implement an
Conclusions       integrated planning, budgeting, and accountability system. Even given this
                  much time, the agency will have difficulty obtaining the scientific and
                  environmental data and developing and reaching agreement on the
                  appropriate environmental measures of its programs’ and its own
                  performance called for by the new system. Furthermore, the efforts EPA
                  has made over the years to improve and integrate its planning, budgeting,
                  and accountability processes illustrate both the importance and the
                  difficulty of these efforts. Given the complexity of the efforts and the time
                  required to complete them, clear expectations or benchmarks for fully
                  implementing an integrated system would better enable the agency’s
                  management, the Congress, and other stakeholders to monitor progress
                  toward implementing such a system.


                  To help ensure the full implementation of the agency’s plans for an
Recommendation    integrated planning, budgeting, and accountability system, we recommend
                  that the EPA Administrator, in consultation with key stakeholders, establish
                  expectations or benchmarks for how the new system is to operate when
                  fully implemented and use them to monitor the agency’s progress in
                  implementing the system.


                  We provided a draft of this report to EPA for review and comment. EPA
Agency Comments   officials, including the Acting Deputy Director of the Office of Planning,
                  Analysis, and Accountability, said that the report was well done and
                  accurately described the status of the agency’s efforts to implement the
                  new planning, budgeting, and accountability system. In addition to
                  suggesting clarifications, which we have incorporated where appropriate,
                  the officials noted that the ultimate success of EPA’s new system will
                  largely depend on others, especially the states, which primarily implement
                  the agency’s programs. According to the officials, in working with the
                  states to implement the system, the agency will face the inherent conflict
                  between the states’ desire for flexibility and EPA’s need to hold them
                  accountable for their performance.



                  Page 13         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
B-276608




The EPA officials agreed with our recommendation and the need to identify
some measures of success for implementing the new planning, budgeting,
and accountability system—that is, some ways for them to know whether
they are progressing toward the type of system intended. They stated that
the key elements listed in appendix II, including the heavy involvement of
stakeholders, are consistent with their plans for implementing GPRA.


We conducted our audit work from June 1996 through May 1997 in
accordance with generally accepted government auditing standards. A
detailed discussion of our objectives, scope, and methodology appears in
appendix III.

As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 15 days from the
date of this letter. At that time, we will send copies to the EPA
Administrator and other interested parties. We will also make copies
available upon request.

Please call me at (202) 512-6111 if you or your staff have any questions.
Major contributors to this report are listed in appendix IV.




Peter F. Guerrero
Director, Environmental
  Protection Issues




Page 14         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Page 15   GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Contents



Letter                                                                                                   1


Appendix I                                                                                              18
                        EPA’s Framework                                                                 18
EPA’s Conceptual        Implementation Activities                                                       20
Framework for and
Activities to
Implement a New
Planning, Budgeting,
and Accountability
System
Appendix II                                                                                             25
                        Planning Component                                                              25
Key Elements of an      Budgeting Component                                                             27
Effective Planning,     Accountability Component                                                        27
                        Analysis Component                                                              28
Budgeting, and
Accountability System
for the Environmental
Protection Agency
Appendix III                                                                                            31

Objectives, Scope,
and Methodology
Appendix IV                                                                                             33

Major Contributors to
This Report
Figures                 Figure 1: Organizational Chart for EPA’s Office of the Chief                     5
                          Financial Officer
                        Figure I.1: General Framework for EPA’s New Planning,                           19
                          Budgeting, and Accountability System




                        Page 16        GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Contents




Abbreviations

EPA        Environmental Protection Agency
GAO        General Accounting Office
GPRA       Government Performance and Results Act of 1993
NAPA       National Academy of Public Administration
OPAA       Office of Planning, Analysis, and Accountability


Page 17         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix I

EPA’s Conceptual Framework for and
Activities to Implement a New Planning,
Budgeting, and Accountability System
                  The Environmental Protection Agency (EPA) has developed an outline, or
                  conceptual framework, for its new planning, budgeting, and accountability
                  system and has begun to develop the individual components.11 This
                  appendix describes the framework and the agency’s implementation
                  activities.


                  Under its new system, EPA is to (1) establish strategic or long-term goals
EPA’s Framework   and objectives; (2) develop strategies to achieve the goals and objectives;
                  (3) identify the desired outcomes from these strategies; (4) translate these
                  goals, objectives, strategies, and desired outcomes into annual goals,
                  planned activities, and performance measures; and (5) use the
                  performance measures to assess the agency’s and individual programs’
                  performance in achieving the desired outcomes. This information on
                  performance is to be considered whenever the agency’s goals, objectives,
                  and strategies are reexamined.

                  As figure I.1 shows, EPA’s framework calls for the agency to first prepare a
                  strategic plan that defines its mission and sets out its goals and objectives
                  for the next 5 years or so. This plan is also to describe the principles and
                  general strategies that are to be employed to achieve the goals and
                  objectives. In turn, a multiyear action plan is to be prepared for each
                  objective in the strategic plan, setting out, for a specific period, outcomes
                  or results to be achieved, ways to measure progress towards these
                  outcomes, alternative strategies to achieve the outcomes, the costs to EPA
                  for each of the strategies, and measures of costs and benefits or
                  cost-effectiveness for each of the alternatives.




                  11
                   EPA refers to its new system as the planning, budgeting, accountability, and analysis system to
                  highlight the greater role planned for the use of scientific information and analysis in these
                  management processes.



                  Page 18              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                                         Appendix I
                                         EPA’s Conceptual Framework for and
                                         Activities to Implement a New Planning,
                                         Budgeting, and Accountability System




Figure I.1: General Framework for EPA’s New Planning, Budgeting, and Accountability System


                              Strategic plan
                                Goals
                                Objectives
                                Principles
                                General strategies




                                Multiyear action
                                     plans
                                                                        Resource "envelope" projections


         Performance
          feedback

                             Annual performance
                                    plans
                                                                          Annual budget                       Annual
                                                                             request                        appropriation




                                                                         Annual operating
                                                                              plan
                                                                                                     Actual resources
                             Actual performance
                              Achievement of
                               performance measures
           Accountability     Resource use
             process
                             Performance evaluation


                                                                       Performance
                                                                         reports




                                         Source: EPA’s Office of Planning, Analysis, and Accountability.



                                         Page 19              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                 Appendix I
                 EPA’s Conceptual Framework for and
                 Activities to Implement a New Planning,
                 Budgeting, and Accountability System




                 Annual performance plans would then link strategic planning to annual
                 planning and budgeting by setting out annual performance goals and
                 annual measures of performance tied to strategic goals and objectives.
                 These plans, which are to be developed for each objective in the strategic
                 plan, also describe specific activities to be carried out during the year
                 (outputs), specific results to be achieved through these activities
                 (outcomes), and the effects of reductions or increases in available
                 resources on outputs and outcomes. The annual plans, especially the
                 sections on needed resources, are to be the basis for preparing annual
                 budget requests.

                 The system’s accountability component provides for examining actual
                 performance and costs against goals and objectives and cost projections.
                 In addition to evaluating achievement against the performance measures
                 set out in the annual performance plans, EPA is to conduct broader
                 performance evaluations. The results of both types of evaluations are to be
                 fed back into the planning and budgeting processes and used to produce
                 the annual performance reports required by the Government Performance
                 and Results Act of 1993 (GPRA) and other internal management reports on
                 performance.12


                 As discussed below, EPA has activities under way to develop the planning,
Implementation   budgeting, and accountability components of its new system. Most of
Activities       these efforts thus far have been directed to preparing the strategic plan
                 that GPRA requires by September 30, 1997.


Planning         Although EPA has experienced some delays, it plans to complete its
                 strategic plan by September 30, 1997, as required. On April 25, 1997, EPA
                 provided the Congress with an outline of its strategic plan, including a
                 draft of its mission statement, goals, and strategic principles.13 This outline
                 is to serve as the basis for discussion with the Congress during formal


                 12
                   Over the longer term, improved financial reporting and auditing, as required by the Chief Financial
                 Officers Act of 1990, is to strengthen the cost basis and reliability of performance information.
                 Reporting concepts and standards developed by the Federal Accounting Standards Advisory Board are
                 intended to provide congressional and other decisionmakers for the first time with annual “report
                 cards” on the costs, management, and effectiveness of federal agencies in order to help strengthen
                 federal decision-making and agencies’ accountability.
                 13
                   The term “strategic principles” refers to statements describing the critical points to be considered
                 when making decisions about EPA’s priorities and activities. These principles—for example, an equal
                 focus on human health and ecological health and the use of risk reduction as the first criterion for
                 establishing ecological and human health priorities—are to provide guidance to decisionmakers and to
                 serve as a common basis for making decisions across the agency’s programs.



                 Page 20              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix I
EPA’s Conceptual Framework for and
Activities to Implement a New Planning,
Budgeting, and Accountability System




consultation on the plan. A list of the agency’s draft objectives, which are
to be measurable statements of what EPA plans to accomplish under each
goal, was provided to the Congress on May 21, 1997. EPA plans to give the
Congress sections of the plan as they become available rather than waiting
until they all are completed. According to the agency’s projections, a
complete draft of the plan will be available by early July 1997.

The Office of Planning, Analysis, and Accountability (OPAA) has compiled a
list of EPA’s strategic objectives from lists of objectives prepared by the
agency’s individual offices, using the agency’s draft goals. In developing
their objectives, the program offices were directed to obtain input from
states, Indian tribes, associations, and other principal stakeholders.
According to the Acting Deputy Director of OPAA, the office will ask these
stakeholders for their input on the mission statement, goals, and
objectives while the agency is consulting with the Congress on the
strategic plan.

Two studies that could affect the selection of EPA’s goals and objectives
are not scheduled to be completed in time for this initial strategic plan.
One study, EPA’s National Environmental Goals Project, is being performed
to establish a set of long-range national environmental goals with realistic
and measurable milestones for the year 2005. In January 1997, EPA made a
draft of the national goals available for “full government review,” including
reviews by the legislative branch, other federal agencies, and the states.
Project officials anticipate that the goals will be available for public
comment in late 1997 and will be issued in 1998. The other study, the
Integrated Risk Project, is being done by EPA’s Science Advisory Board,
which advises the Administrator on scientific matters. A draft report on
the results of this study, which is being done to rank the relative risks of
environmental problems and to develop methodologies that EPA can use to
rank risks in the future, is to be provided to other scientists and experts
for peer review in August 1997. According to EPA officials, strategic
planning is an iterative process and the plan will be updated, as
appropriate, to reflect the final results of these and other studies or
factors. The officials said that a draft of the national goals was considered
in developing the agency’s draft goals, and program offices were
instructed to consider risk and the potential for risk reduction in
developing their objectives. According to EPA officials, the agency is
beginning to develop the data that it will need to compare risks across
programs.




Page 21           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
            Appendix I
            EPA’s Conceptual Framework for and
            Activities to Implement a New Planning,
            Budgeting, and Accountability System




            The framework for EPA’s new system calls for the agency’s program offices
            to prepare multiyear action plans after the strategic plan is completed and
            before annual performance plans are begun. These multiyear plans are to
            bridge the strategic plan and the annual performance plans by describing
            for each objective in the strategic plan the results or outcomes that are to
            be achieved, the methods of measuring progress toward achieving these
            outcomes, and the costs of achieving the outcomes under alternative
            strategies. This information on the outcomes, strategies, and costs
            associated with achieving goals and objectives is then to be used for
            annual planning and budgeting. However, EPA officials believe that they
            need to begin preparing the annual performance plans as soon as the
            strategic goals and objectives are selected if the annual plans are to be
            completed by September 30, 1997, as required by the Office of
            Management and Budget. According to agency officials, the preparation of
            multiyear action plans, which are not required by GPRA, will be delayed
            until the fiscal year 2000 planning cycle. The officials said that some items
            that would have been discussed in the action plans may be incorporated
            into the strategic plan that is currently being prepared.

            Once the strategic goals and objectives are approved by EPA management,
            the program offices are to prepare annual performance plans for fiscal
            year 1999. These plans are to be prepared for each of the objectives in the
            strategic plan. OPAA is currently developing guidance for the program
            offices on how these plans are to be prepared.


Budgeting   According to EPA officials, the agency plans to organize its budget, starting
            with the one for fiscal year 1999, on the basis of its strategic goals and
            objectives. Under GPRA, this change has to be negotiated with the Office of
            Management and Budget and the agency’s appropriations subcommittees.
            EPA officials told us that they are currently deciding how best to
            restructure the budget and have discussed their views with Office of
            Management and Budget officials. Agency officials said that more than one
            budgeting cycle may be needed to determine the best structure for the
            budget.

            GPRA  requires federal agencies to use the program activities appearing in
            the President’s Budget as the basis for performance planning and
            measurement. This requirement aims at ensuring a simple, straightforward
            link among plans, budgets, and performance information and the related
            congressional oversight and resource allocation processes. However,
            program activity structures and their suitability for planning and



            Page 22           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                 Appendix I
                 EPA’s Conceptual Framework for and
                 Activities to Implement a New Planning,
                 Budgeting, and Accountability System




                 measurement purposes under GPRA vary from agency to agency. Under
                 GPRA, agencies can consolidate, aggregate, or disaggregate program
                 activity structures for performance planning purposes, where needed.
                 (Using this approach requires subsequent crosswalks between
                 performance plans and the program activities appearing in the budget
                 request.) Agencies may also attempt to renegotiate program activities with
                 their appropriations subcommittees and the Office of Management and
                 Budget.14

                 According to EPA officials, the ongoing strategic and annual performance
                 planning will influence budget decisions for fiscal year 1999. The Office of
                 Management and Budget will review EPA’s and other agencies’ fiscal year
                 1999 annual performance plans when it considers their budget requests for
                 that year.


Accountability   EPA’s two major activities to develop this component are the
                 implementation of accountability pilot projects and the development of an
                 agencywide reporting system. EPA is implementing 46 accountability pilot
                 projects across the agency’s programs, offices, and regions. According to
                 agency officials, these projects are being conducted to provide
                 organizational units with experience in using performance measures and
                 to obtain input into the development of an agencywide accountability
                 system. Under the pilot projects, the responsible units are to prepare
                 performance plans, identify performance measures, and use the measures
                 to assess performance in achieving the goals set out in the plans. The final
                 results of the pilot projects are due to OPAA in September 1997. In addition,
                 during fiscal years 1994 through 1996, EPA, as well as other agencies,
                 participated with the Office of Management and Budget in carrying out
                 projects to pilot-test GPRA’s performance planning and reporting
                 requirements. EPA’s six pilot projects were in the following programs:
                 (1) Acid Rain, (2) Leaking Underground Storage Tanks, (3) Public Water
                 System Supervision, (4) Environmental Technology Initiative,
                 (5) Chesapeake Bay, and (6) Superfund.




                 14
                   In a March 1997 report, Performance Budgeting: Past Initiatives Offer Insights for GPRA
                 Implementation (GAO/AIMD-97-46, Mar. 27, 1997), we noted a potential tension between legislative
                 staff and agency officials on using program activities as a basis for agencies’ performance planning and
                 measurement. Agency officials generally saw a need to be flexible in using program activities for
                 planning and considered changes to program activity structures likely and desirable to better align
                 programs with GPRA’s goals and objectives. Legislative staff generally viewed these structures as
                 fundamental to congressional oversight of agencies’ activities. The staff were generally comfortable
                 with the existing structures and suggested that changes might frustrate congressional oversight.



                 Page 23              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix I
EPA’s Conceptual Framework for and
Activities to Implement a New Planning,
Budgeting, and Accountability System




EPA plans to have a formal reporting system as part of the accountability
process. The target date for having the system in place is September 1999,
when the agency will need to begin preparing the annual performance
reports required under GPRA. The act requires agencies to submit these
reports covering fiscal year 1999 by March 31, 2000. According to the
Acting Chief Financial Officer, designing the accountability system will
take some time because the system needs to be linked to the financial
management and personnel rewards systems. Decisions on certain issues,
such as the extent to which the system will be centralized or
decentralized, have not yet been made. OPAA is reviewing previously used
agencywide reporting systems to find out what did and did not work well,
contacting other federal agencies to determine how they account for
progress in meeting their goals, and examining reporting systems in the
agency’s program offices to identify their potential use in the new system.




Page 24           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix II

Key Elements of an Effective Planning,
Budgeting, and Accountability System for
the Environmental Protection Agency
                     GAO has identified the following elements of a planning, budgeting, and
                     accountability system for the Environmental Protection Agency (EPA) as
                     important to its efforts to improve its processes for establishing its
                     direction, setting priorities, allocating resources, and assessing results.
                     These elements are based on prior GAO work; the findings and
                     recommendations of various external and internal reports on EPA,
                     including those of the National Academy of Public Administration; the
                     requirements of the Government Performance and Results Act of 1993;
                     and input from several former high-level officials responsible for planning,
                     budgeting, and/or accountability while at the agency and others
                     knowledgeable about EPA’s management. The elements are grouped by
                     major management process or system component, that is, planning,
                     budgeting, and accountability. They also are shown for an analysis
                     component. The individual analysis elements support and could have been
                     included under one or more of the other components. They have been
                     segregated and are shown together to emphasize and highlight their
                     importance to the effectiveness of the system.


                     OBJECTIVE: To establish direction and priorities for EPA by clarifying its
Planning Component   mission; setting measurable, outcome-oriented goals and objectives; and
                     determining how these goals and objectives will be met and how
                     performance/results will be measured.

                     Central office of planning, budgeting, accountability, and analysis
                     to oversee, manage, and facilitate an integrated, agencywide planning,
                     budgeting, and accountability structure that focuses on the
                     (1) identification of, (2) appropriate allocation of resources for, and
                     (3) measurement of results in achieving, priorities related to the agency’s
                     mission.

                     Strategic plan to define the agency’s mission and clearly establish its
                     priorities in the longer term, that is, over the next 5 to 6 years. The
                     priorities are to be expressed in measurable, outcome-related goals and
                     objectives and linked to the agency’s mission, national environmental
                     goals, comparative or relative rankings of health and environmental risks,
                     and potential for reducing risk. The adequacy and suitability of the goals
                     and objectives are to be periodically reviewed. At a minimum, this review
                     could take place during updates of the action plans described below,
                     which would occur about half way through the period covered by the
                     strategic plan and again during the preparation of a strategic plan for the
                     next 5 to 6 years.



                     Page 25         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix II
Key Elements of an Effective Planning,
Budgeting, and Accountability System for
the Environmental Protection Agency




The plan is to describe how the agency intends to achieve its goals and
objectives, including how external factors and resources may affect what
can be achieved. The plan is also to discuss how nonregulatory,
incentive-based, and traditional regulatory approaches will be considered
and what changes in the agency’s staffing or mix of skills will be required.
Furthermore, the plan is to have a “futures” component that looks 20 to 30
years ahead to identify emerging environmental problems and new
abatement strategies or techniques that may warrant research or other
actions during the period covered by the plan. Because abatement and
control actions can move pollutants from one environmental medium to
another, the goals, objectives, and strategies set out in the plan are to
apply to multiple media, rather than to a single medium—such as air,
water, or land—wherever possible and appropriate. Stakeholders,
particularly the Congress, the states, Indian tribes, and appropriate public
interest groups, are to participate in the strategic plan’s development.

Multiyear action plan for each objective in the strategic plan to detail
decisions for the shorter term—over the next 2 to 3 years—on what the
agency intends to achieve, how it will be achieved, and how progress will
be measured. The plans are to also (1) establish milestones for completing
the specified actions or activities, (2) determine the resources needed to
achieve each milestone, and (3) define the roles and responsibilities of EPA
and its partners, such as the states, for achieving the milestones.15

Annual performance plan for each objective in the strategic plan to
describe the specific activities that are to be carried out and the specific
results that are to be achieved in a particular fiscal year in support of the
strategies and milestones in the action plan.16 The annual plan is to
establish specific performance goals for these activities, summarize the
necessary resources to conduct the activities, and set out the performance
indicators to be used to measure performance. The plan links day-to-day
program activities and outcomes to longer-term agency goals through the
annual performance goals. It supports the budgeting process by providing
information on the outputs and outcomes that could not be achieved if



15
 These functions of the multiyear action plans could also be carried out as part of the strategic
planning process and incorporated into the strategic plan. In this case, separate documents would not
be prepared.
16
 GPRA requires the use of program activities appearing in the President’s Budget as the basis for
performance planning. GPRA allows agencies to consolidate, aggregate, or disaggregate program
activities in conducting performance planning. Agencies may also attempt to renegotiate the program
activity structures of their budgets with their appropriations subcommittees and the Office of
Management and Budget.



Page 26              GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                      Appendix II
                      Key Elements of an Effective Planning,
                      Budgeting, and Accountability System for
                      the Environmental Protection Agency




                      resources were reduced and on the additional outputs and outcomes that
                      could be achieved with additional resources.


                      OBJECTIVE: To allocate the agency’s resources to the highest priorities so
Budgeting Component   as to achieve the greatest reduction in risks to human health and the
                      environment.

                      Annual budget document to finalize resource allocation decisions made
                      during the internal planning process. The document is essentially the
                      resources section of the annual performance plan prepared to present and
                      defend the agency’s allocation decisions to the Office of Management and
                      Budget and the Congress.

                      Base program analysis to evaluate the contribution of individual
                      programs to meeting the agency’s goals and identify areas of lower priority
                      for potential disinvestment and shifting of resources to new initiatives or
                      higher priorities in other programs. These analyses are to be performed by
                      individual program managers under the guidance of the central planning,
                      budgeting, accountability, and analysis office for review by the agency’s
                      management.

                      Goals-based program element budget structure to organize the
                      presentation and execution of budget decisions around the agency’s goals,
                      performance plans, and measures of success.


                      OBJECTIVE: To provide EPA managers, the Congress, and the public with
Accountability        the information on programs’ finances and results needed to judge the
Component             agency’s performance in making efficient and effective use of resources to
                      carry out its mission.

                      Formal agencywide accountability system to compile—through
                      internal reporting requirements, program evaluation results, and other
                      analyses—relevant and essential information on the accomplishments and
                      resources expended relative to the commitments made in annual
                      performance plans, focusing on key measures of success for each goal.

                      Performance measures to indicate the degree to which strategic goals
                      are being met and to gauge the impact of programs by linking their
                      activities or actions to environmental results. Include measures of the
                      programs’ activity levels as necessary but focus on indicators of ambient



                      Page 27           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
                     Appendix II
                     Key Elements of an Effective Planning,
                     Budgeting, and Accountability System for
                     the Environmental Protection Agency




                     (environmental) conditions. The key performance measures, especially the
                     technical measures, are to be peer-reviewed by independent sources, such
                     as the Science Advisory Board.

                     Program evaluations to provide information on the agency’s
                     implementation of individual programs, including both what the agency is
                     achieving given its mandate and how the mandate is contributing to the
                     achievement of national goals. These evaluations are to be performed
                     periodically, preferably near the end of the period covered by the current
                     multiyear action plans and before the new plans are prepared. At a
                     minimum, the evaluations are to be completed for major programs in time
                     to be used in preparing new strategic plans.

                     National environmental performance partnership agreements with
                     the states to set out expectations for their contribution to achieving
                     national environmental and agencywide goals and measures for assessing
                     their performance in achieving these goals. Expectations are to be in the
                     form of negotiated, goal-oriented commitments for EPA-funded and
                     delegated program responsibilities and linked to individual national and
                     agency goals. The states are to have input into the agency’s processes for
                     setting these goals, selecting the strategies to achieve them, and allocating
                     the resources to implement the strategies.

                     Annual performance reports to evaluate and compare a program’s
                     achievements to the goals set out in the annual performance plans. If a
                     goal is not met, the report is to explain why and identify the actions
                     needed to achieve the goal. These reports are to be provided to the Office
                     of Management and Budget and the Congress. They are also to be made
                     available to the public.


                     OBJECTIVE: To provide the data and analyses decisionmakers need to
Analysis Component   make informed judgments about the environmental problems or concerns
                     that the agency should address, its strategies for dealing with them, and
                     the resources to be allocated for these purposes.

                     ORGANIZATIONAL/PROCESS NEEDS:

                     Bureau or center of environmental information and statistics that is
                     independent of program/project offices and the central planning,
                     budgeting, and accountability office. This entity is responsible to the
                     Administrator for defining data quality and collection methods; identifying



                     Page 28           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix II
Key Elements of an Effective Planning,
Budgeting, and Accountability System for
the Environmental Protection Agency




data gaps; and independently assembling, analyzing, and reporting
comprehensive, objective, and reliable statistics that reflect environmental
quality and trends.

Independent analytical capability to compile and assess relevant
information on health and environmental risks, risk reduction potential,
and performance across the agency’s programs. This capability is to be
located within the central planning, budgeting, and accountability office.
The information is to be provided by the program offices; other units
within the agency, such as the research and development office; and
outside sources.

“Lookout” panel of scientists and other experts to identify future or
emerging environmental problems for consideration by the agency’s
management during the planning and budgeting process. Conflict of
interest rules are to apply to panel members. Participants in the agency’s
grants and contracts are generally not to participate on the panel.

INFORMATION NEEDS:

Monitoring data to comprehensively document environmental
conditions and trends on a regional and national basis. In describing how
performance is to be measured, multiyear action plans, annual
performance plans, and national environmental performance partnership
agreements are also to describe the extent to which ambient monitoring
data are available for use in assessing programs’ results and determining
how critical gaps in the data are to be filled.

National environmental goals to express intended, long-range results of
national (federal, state, local, industry, and private) efforts to protect the
environment, reflecting stakeholders’ judgment as to the measurable
progress the nation can and should achieve.

Comparative or relative risk information to permit the assessment or
weighing of health and environmental risks and the potential for reducing
risks across environmental problems and programs.

Options information to help identify cost-effective measures or actions
to address environmental problems. This information is to include data on
the benefits and costs of alternative ways of achieving similar results and
the incremental social costs of given actions compared with their
incremental benefits.



Page 29           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix II
Key Elements of an Effective Planning,
Budgeting, and Accountability System for
the Environmental Protection Agency




Performance information, including risk reductions achieved; program
evaluation results; and reporting on environmental indicators/outcome
measures, program activity levels, and financial data, to identify how well
and at what cost programs and projects are meeting their objectives.




Page 30           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix III

Objectives, Scope, and Methodology


               The Chairmen of the Subcommittees on VA, HUD, and Independent
               Agencies, Senate and House Committees on Appropriations, requested
               that we review EPA’s plans to implement a new planning, budgeting, and
               accountability system. Specifically, the Chairmen asked us to review the
               status of EPA’s efforts to (1) establish a new planning, budgeting, and
               accountability office and develop and implement an integrated system and
               (2) ensure that the system uses comprehensive scientific and
               environmental data and appropriate environmental measures. They also
               asked us to discuss EPA’s previous efforts to implement this type of system
               and to identify elements of an effective system that could be used as
               benchmarks to gauge progress in this current effort.

               To determine the status of EPA’s efforts to establish the new office and to
               develop and implement an integrated system, we reviewed relevant
               documents on EPA’s plans, target dates for these efforts, and actions taken;
               the staffing provided for the Planning, Budgeting, Accountability, and
               Analysis Work Group that was set up until the new planning, budgeting,
               and accountability organization could be established; and the status of the
               new office’s approval and staffing. We also discussed the status of EPA’s
               plans with the Acting Chief Financial Officer; the Comptroller; and other
               personnel of the Planning, Budgeting, Accountability, and Analysis Work
               Group and the new Office of Planning, Analysis, and Accountability.

               In addition, we compared activities for implementing the system with the
               recommendations of EPA’s Planning, Budgeting, and Accountability Task
               Force, which were accepted by the EPA Administrator. We also reviewed
               relevant documents and held discussions with staff of the Office of Policy,
               Planning, and Evaluation to determine the status of the National
               Environmental Goals Project. In addition, we discussed the purpose and
               status of the Integrated Risk Project with staff of the Science Advisory
               Board, reviewed pertinent documents on the project, and attended various
               sessions of the Board’s committees and subcommittees established to
               carry out the project.

               To determine EPA’s efforts to ensure that its new system uses
               comprehensive scientific and environmental data and appropriate
               environmental measures, we reviewed pertinent studies and other
               documents to identify improvements needed in the data and measures. We
               compared the needed improvements to EPA’s plans for the use of data and
               measures in the new system. In addition, we discussed the status of EPA’s
               efforts to identify and develop environmental measures with various EPA
               regional and state officials, including representatives of the Environmental



               Page 31         GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix III
Objectives, Scope, and Methodology




Council of the States. Furthermore, we attended two EPA workshops on
environmental data and measures. These were “Managing for
Environmental Results: Using Goals and Indicators,” held in September
1996, and “Environmental Management in the Nineties: Using the Right
Tools,” held in January 1997.

To identify elements that could be used as benchmarks for monitoring
EPA’s progress in implementing an integrated planning, budgeting, and
accountability system, we reviewed the findings and recommendations of
the NAPA report on EPA, the requirements of the Government Performance
and Results Act, our prior reviews of EPA, and other external and internal
EPA studies and evaluations to identify key elements of an effective
planning, budgeting, and accountability system for EPA. We then obtained
and incorporated comments on these key elements from four former top
EPA officials who had responsibilities for the planning, budgeting, and/or
accountability functions when at the agency—a deputy administrator and
three assistant administrators (two for policy, planning, and evaluation
and one for administration and resource management). We also obtained
comments from representatives of NAPA and EPA’s Science Advisory Board.
We conducted our review from June 1996 through May 1997 in accordance
with generally accepted government auditing standards.




Page 32           GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Appendix IV

Major Contributors to This Report


                        Stanley J. Czerwinski, Associate Director
Resources,              Edward A. Kratzer, Assistant Director
Community and           Raymond H. Smith, Jr., Evaluator-in-Charge
Economic                Frank J. Gross, Senior Evaluator
                        Jennifer W. Clayborne, Staff Evaluator
Development
Division, Washington,
D.C.
                        Bruce Skud, Senior Evaluator
Boston Field Office




(160359)                Page 33        GAO/RCED-97-156 EPA’s Planning, Budgeting, and Accountability System
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested