oversight

Regulatory Reinvention: EPA's Common Sense Initiative Needs an Improved Operating Framework and Progress Measures

Published by the Government Accountability Office on 1997-07-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Requesters




July 1997
                  REGULATORY
                  REINVENTION
                  EPA’s Common Sense
                  Initiative Needs an
                  Improved Operating
                  Framework and
                  Progress Measures




GAO/RCED-97-164
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-277140

      July 18, 1997

      Congressional Requesters

      In response to your request, this report examines the progress of the Environmental Protection
      Agency’s (EPA) Common Sense Initiative toward its goal of finding “cleaner, cheaper, smarter”
      ways of reducing or preventing pollution and the methods EPA uses to measure progress.

      We are sending copies of this report to other appropriate congressional committees; the
      Administrator, EPA; and the Director, Office of Management and Budget. We will also make
      copies available to others upon request.

      Please call me at (202) 512-4907 if you or your staff have any questions. Major contributors to
      this report are listed in appendix III.




      Peter F. Guerrero
      Director, Environmental Protection
        Issues
B-277140

List of Requesters

The Honorable John Chafee
Chairman, Committee on Environment
  and Public Works
United States Senate

The Honorable Christopher Bond
Chairman, Subcommittee on VA, HUD,
  and Independent Agencies
Committtee on Appropriations
United States Senate

The Honorable Tom Bliley
Chairman, Committee on Commerce
House of Representatives

The Honorable Michael G. Oxley
Chairman, Subcommittee on Finance
  and Hazardous Materials
Committee on Commerce
House of Representatives

The Honorable Bud Shuster
Chairman, Committee on Transportation
  and Infrastructure
House of Representatives

The Honorable Sherwood Boehlert
Chairman, Subcommittee on Water
  Resources and Environment
Committee on Transportation
  and Infrastructure
House of Representatives

The Honorable Jerry Lewis
Chairman, Subcommittee on VA, HUD,
  and Independent Agencies
Committee on Appropriations
House of Representatives




                     Page 2             GAO/RCED-97-164 EPA’s Common Sense Initiatives
B-277140

The Honorable David M. McIntosh
Chairman, Subcommittee on National
  Economic Growth, Natural Resources,
  and Regulatory Affairs
Committee on Government Reform
  and Oversight
House of Representatives




                   Page 3               GAO/RCED-97-164 EPA’s Common Sense Initiatives
Executive Summary


             Although the United States has made significant progress in environmental
Purpose      protection, the Environmental Protection Agency (EPA) believes that
             resolving future environmental challenges will require a fundamentally
             different approach, which the agency calls “regulatory reinvention.” In
             July 1994, EPA began its Common Sense Initiative—termed the
             “centerpiece” of its regulatory reinvention efforts—with a goal of finding
             “cleaner, cheaper, smarter” ways of reducing or preventing pollution and
             recommending changes in the existing approach to environmental
             management, including the existing statutes and regulations. To
             accomplish this goal, EPA established an advisory forum with industry and
             other affected stakeholders to achieve both higher environmental
             protection standards than are currently required and more cost-effective
             results for industry and the public.

             Congressional committees and others have questioned the progress of
             EPA’s reinvention efforts and of the Common Sense Initiative in particular.
             To better understand the Initiative and the progress it has made, Senate
             and House Committee and Subcommittee Chairmen asked GAO to assess
             (1) EPA’s progress in achieving the goal the agency set for the Initiative and
             (2) the methods EPA uses to measure the progress of the Initiative toward
             its goal.


             EPA’s Common Sense Initiative was designed to achieve greater
Background   environmental protection at less cost by creating strategies for controlling
             and preventing pollution for individual industries rather than for individual
             pollutants, as is currently done. The Initiative is organized under the
             Federal Advisory Committee Act (FACA) and consists of a Council and
             specialized subcommittees that focus on selected industrial sectors and
             carry out studies and projects to develop recommendations, based on
             consensus among stakeholders, for administrative, regulatory, and
             statutory changes.

             EPA administers the Initiative, determines its multistakeholder
             membership, and publicly reports on its FACA-based operations, activities,
             and costs, as well as on the results of Council and subcommittee meetings.
             EPA selected six industrial sectors—automobile manufacturing, computers
             and electronics, iron and steel, metal finishing, petroleum refining, and
             printing—to participate in the Initiative. As of April 1997, the Council’s
             industrial sector subcommittees were carrying out about 40 projects in
             areas such as pollution prevention, streamlined permitting and reporting,
             enhanced public participation in environmental management, the use of



             Page 4                            GAO/RCED-97-164 EPA’s Common Sense Initiatives
                   Executive Summary




                   innovative technologies, and the setting of environmental goals and
                   indicators.

                   In reviewing EPA’s progress in achieving the Initiative’s goal, GAO applied
                   concepts underlying the Government Performance and Results Act of
                   1993. The act—known as GPRA or the Results Act—focuses on program
                   outcomes, comparing the results of a program activity to its intended
                   purpose rather than focusing on activity levels and tasks completed. GAO
                   applied the concepts to 15 of the Initiative’s 36 projects that were ongoing
                   in September 1996.


                   In the almost 3 years the Initiative has been under way, it has produced
Results in Brief   three formal recommendations to EPA, none of which has suggested the
                   types of changes in the existing approach to environmental management
                   that EPA expected. Although stakeholders have begun to work
                   collaboratively on environmental solutions, progress toward the
                   Initiative’s goal has been limited by several factors, such as the length of
                   time needed to collect and analyze data; the difficulties stakeholders have
                   had in reaching consensus on the approaches needed to address large,
                   complex issues or policies; and variations in stakeholders’ commitments
                   of time and understanding of the technical aspects of environmental
                   issues. In addition, the Council and its subcommittees and workgroups
                   have spent considerable time discussing how they would carry out their
                   work and developing their own operating standards. An improved
                   operating framework that better defined the Initiative’s goal and expected
                   results and included specific guidance on how the Initiative would
                   accomplish its work would enable the Council and its industrial sector
                   subcommittees and workgroups to concentrate more of their effort on
                   substantive issues.

                   EPA gauges the progress of the Initiative primarily on the basis of
                   accomplishments associated with its various processes or activities, such
                   as stakeholder meetings, and not on the basis of its results. Although such
                   process-oriented information is important, it does not measure the
                   agency’s progress in meeting the Initiative’s goal, consistent with GPRA’s
                   intent. As a result, EPA cannot determine the extent to which the Initiative
                   may cost-effectively reduce or prevent pollution or ascertain whether such
                   improvements are due to changes in the agency’s approach to
                   environmental management. In addition, the Initiative’s projects typically
                   do not establish or provide for performance measures to gauge the extent
                   to which they are decreasing pollution and/or reducing costs. GAO found



                   Page 5                            GAO/RCED-97-164 EPA’s Common Sense Initiatives
                              Executive Summary




                              that 11 of the 15 ongoing projects it reviewed did not provide for
                              measuring results. Without such measures, it is difficult to assess progress
                              or demonstrate whether a project’s expected outcome has occurred.



Principal Findings

Several Factors Have          EPA expected that, within the first year, the Common Sense Initiative
Limited the Progress of the   Council would make recommendations for changes that would provide the
Initiative Toward Its Goal    basis for a new approach to environmental management. Such
                              recommendations have not been made, in part because the subcommittee
                              workgroups that conduct the studies and projects to develop
                              recommendations need considerable time to gather data and to design,
                              test, and analyze reinvention alternatives. Most of the projects that were
                              ongoing as of April 1997 were initiated during 1995, the Council’s first full
                              year of operation. Furthermore, primarily because their participants hold
                              diverse viewpoints, many of the workgroups have not been able to reach
                              agreement on the objectives for their projects or on the approaches
                              needed to address big issues or policies and have therefore tended to
                              undertake more narrowly focused projects. At this point, it is uncertain
                              how EPA and other stakeholders will address broad, complex issues and
                              whether the stakeholders will be willing and able to provide the time and
                              resources necessary to carry out the program.

                              Various subcommittees and workgroups have also spent considerable time
                              on process-related issues, such as how consensus is defined, when and at
                              what level it is necessary, and whether all stakeholder groups need to be
                              represented in projects and workgroups. Such issues raise questions about
                              the adequacy of the guidance and of the operating framework that EPA
                              provided for carrying out the Council’s and subcommittees’ activities.
                              Because progress has been slow, some stakeholders have expressed
                              concerns about the time and resources required and the lack of tangible
                              results, and some industry representatives have questioned their
                              continued participation in the program. Another factor that could affect
                              the Initiative’s future progress is uncertainty about the degree of flexibility
                              in existing federal, state, or local laws and regulations and about whether
                              legislative or regulatory changes will be required before some pilot
                              projects can be carried out. For example, statutory and regulatory
                              permitting requirements that differ for the various environmental laws




                              Page 6                             GAO/RCED-97-164 EPA’s Common Sense Initiatives
                            Executive Summary




                            could make it more difficult to develop and implement a multimedia
                            permit program.


Results-Oriented Measures   The agency has not yet established results- or outcome-based measures for
Are Needed to Assess        assessing the extent to which the Initiative has reduced or prevented
Progress                    pollution at less cost to industry and the taxpayer through regulatory
                            reinvention. A 1997 EPA contractor study1 of the Initiative identified the
                            program’s complexity and relatively early stage as key reasons why EPA
                            has not yet established such measures. EPA is considering the study’s
                            findings and conclusions; however, at the time of GAO’s review, it was not
                            clear what changes would be forthcoming.

                            In the absence of results-oriented performance measures for assessing
                            progress, EPA has focused, for the most part, on processes and
                            activities—such as meetings, workshops, and conferences—and not on the
                            results and outcomes that are expected under the Initiative. In addition,
                            the industrial sectors’ projects, which are the program’s principal means
                            for testing regulatory reinvention alternatives, typically do not contain, or
                            provide for in their design, performance measures to gauge their progress
                            toward achieving their individual objectives. The 1997 EPA contractor
                            study stressed the importance of developing appropriate results-oriented
                            performance measures for assessing the extent to which the Initiative’s
                            goal and individual projects’ objectives are achieved. EPA needs such
                            measures to accurately track the Initiative’s progress.


                            To permit the Initiative’s subcommittees and workgroups to devote more
Recommendations             of their attention to substantive issues, GAO recommends that the
                            Administrator, EPA, provide an improved operating framework that
                            (1) more clearly defines the Initiative’s “cleaner, cheaper, smarter”
                            environmental protection goal—including its expected results—and
                            (2) specifies how the Council and its subcommittees and workgroups will
                            accomplish their work, clarifying issues such as how and when consensus
                            will be achieved, how the Initiative’s goal should be interpreted and
                            applied to individual projects, and to what extent representatives of all
                            stakeholder groups should be included in activities at each level of the
                            Initiative, including its projects and workgroups.

                            To provide a basis for evaluating the progress of the Initiative in
                            cost-effectively reducing or preventing pollution, GAO recommends that the

                            1
                             Review of the Common Sense Initiative, The Scientific Consulting Group, Inc. (Feb. 19, 1997).



                            Page 7                                       GAO/RCED-97-164 EPA’s Common Sense Initiatives
                  Executive Summary




                  Administrator, EPA, require the development of results-oriented
                  performance measures for assessing the extent to which the Council’s and
                  subcommittees’ actions have produced real, measurable environmental
                  improvements at less cost to industry and the public.


                  EPA provided written comments on a draft of this report (see app. II). EPA
Agency Comments   agreed with the report’s recommendations and stated that the agency has
                  begun to address the recommendations in the changes being implemented
                  and considered for the Initiative. However, EPA expressed concern that the
                  focus of GAO’s review on the Initiative’s progress in accomplishing
                  regulatory, statutory, or administrative changes (1) does not adequately
                  recognize the breadth of the Initiative’s accomplishments and (2) is an
                  unfair measure of the Initiative’s progress at this point in its development.

                  GAO  recognizes that the Initiative is complex and that its success is
                  dependent on accomplishing numerous tasks and activities. GAO believes,
                  however, that emphasizing the statutory, regulatory, and administrative
                  changes expected to result from the Initiative is appropriate and that this
                  report adequately recognizes the breadth of the accomplishments that EPA
                  is reporting for the Initiative. GAO believes its focus is appropriate, given
                  that EPA (1) considers the Initiative “the centerpiece” of its regulatory
                  reinvention efforts and (2) established a goal for the Initiative of finding
                  “cleaner, cheaper, smarter” ways of reducing or preventing pollution and
                  recommending changes in the existing approach to environmental
                  management, including existing statutes and regulations. Also, as the
                  report points out, GAO used an approach that applies the concepts
                  underlying GPRA, which focuses on program outcomes, comparing the
                  results of a program activity to its intended purpose rather than focusing
                  on activity levels and tasks completed. As the report states, the
                  accomplishments reported by EPA include various processes and activities
                  that the Initiative has undertaken to achieve its goal. Although GAO did not
                  discuss all of these accomplishments, the report summarizes them and
                  recognizes their importance and utility as a means for achieving the
                  expected statutory, regulatory, and administrative changes.

                  GAO  agrees that it takes time to identify, test, and evaluate innovative
                  approaches to achieving environmental improvements. However, GAO also
                  believes that the report fairly assesses the progress of the Initiative toward
                  its stated goal at the time of GAO’s review and discusses the factors that
                  prevented the Initiative from achieving the Administrator’s
                  expectation—namely, that it would bring about a fundamental change in



                  Page 8                            GAO/RCED-97-164 EPA’s Common Sense Initiatives
Executive Summary




the approach to environmental protection and achieve tangible results in
the first year. The report points out that the progress of the Initiative to
date has been slow in view of the high expectations EPA set for it and
discusses a number of factors that contributed to this slow progress. In its
comments on GAO’s draft report, EPA agreed that it underestimated the time
required to do the things needed to allow the development of
recommendations for regulatory, statutory, or administrative changes. It
said, however, that barriers had to be reduced before environmental
results could be accomplished.




Page 9                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
Contents



Executive Summary                                                                                      4


Chapter 1                                                                                             12
                           Status of the Common Sense Initiative                                      12
Introduction               Objectives, Scope, and Methodology                                         15
                           Agency Comments and Our Evaluation                                         16

Chapter 2                                                                                             18
                           Recommendations for Change Were Expected in the Initiative’s               18
Several Factors Have         First Year
Limited the Initiative’s   Several Factors Have Contributed to the Initiative’s Limited               19
                             Progress
Progress in Bringing       EPA and the Council Are Considering Options to Address Issues              26
About Changes in             Identified in a Contractor’s Review
Environmental              Need for Legislative or Regulatory Changes Is Unclear and Could            26
                             Delay Some Projects’ Implementation
Management                 Conclusions                                                                27
                           Recommendation                                                             28

Chapter 3                                                                                             29
                           Initiative Promises Significant Regulatory Reinvention                     29
Results-Oriented           Accomplishments Focus Principally on Processes and Not on                  30
Measures of Progress          Regulatory Changes
                           EPA Lacks Results-Oriented Measures for Assessing Progress                 35
Are Needed                    Toward Achieving the Initiative’s Goal
                           Conclusions                                                                41
                           Recommendation                                                             41

Appendixes                 Appendix I: GAO’s Performance Measurement Model and the                    42
                             Projects Reviewed
                           Appendix II: Comments From the Environmental Protection                    46
                             Agency
                           Appendix III: Major Contributors to This Report                            49

Figure                     Figure I.1: GAO’s Performance Measurement Model                            43




                           Page 10                         GAO/RCED-97-164 EPA’s Common Sense Initiatives
Contents




Abbreviations

DFO        designated federal officer
EPA        Environmental Protection Agency
FACA       Federal Advisory Committee Act
FTE        full-time equivalent
GPRA       Government Performance and Results Act
NPR        National Performance Review
R&D        research and development
VOC        volatile organic compound


Page 11                       GAO/RCED-97-164 EPA’s Common Sense Initiatives
Chapter 1

Introduction


                       While the current system of environmental regulation in the United States
                       is the most advanced in the world, it is complex and prescriptive and often
                       results in conflict and gridlock. This regulatory structure has impaired
                       EPA’s ability to experiment with innovative and more cost-effective ways of
                       reducing pollution (such as preventing pollution by eliminating or
                       minimizing it at its source, instead of containing it at the end of the pipe)
                       or using market-based incentives (such as taxing pollution or trading
                       emission “rights”). In addition, the structure has restricted EPA’s ability to
                       exercise flexibility with regulated entities, states, environmental groups,
                       and other “stakeholders” in the regulatory process. EPA has reported that it
                       is subject to over 600 lawsuits at any given time and that the current
                       process often diverts valuable resources from the real work of protecting
                       the environment and public health.

                       In response to various legislative mandates, EPA has organized its activities
                       around environmental media (such as air, water, or land) and the
                       substances it regulates (such as hazardous waste, pesticides, and toxic
                       substances)—resulting in a regulatory structure that closely parallels the
                       statutes authorizing its activities. Each of EPA’s program offices focuses
                       primarily on implementing the medium-specific or substance-specific
                       responsibilities detailed in individual statutes, rather than addressing the
                       full range of pollution sources in a cross-cutting manner. The agency’s
                       medium-specific focus can result in both the intended and unintended
                       transfer of pollution from one medium to another. For example, removing
                       contaminants from public sewage systems or industrial smokestacks can
                       create sludge and waste that themselves can be toxic and lead to further
                       air, water, or land pollution. As a result, EPA and others have
                       acknowledged a need for increased attention to such intermedia transfers
                       to ensure that the agency’s pollution reduction strategies have the best
                       overall impact on the environment.

                       To fundamentally reexamine and reshape its efforts to protect the
                       environment, EPA is attempting to apply common sense, flexibility, and
                       creativity through a number of initiatives designed to move beyond the
                       “one-size-fits-all system of the past” and better protect public health and
                       the environment at a reduced cost. In 1994, as part of the agency’s new
                       approach to “reinvent” environmental protection, EPA began its Common
                       Sense Initiative (Initiative).


                       The goal of EPA’s Common Sense Initiative—termed the “centerpiece” of
Status of the Common   the agency’s regulatory reinvention efforts—is to find “cleaner, cheaper,
Sense Initiative

                       Page 12                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
                           Chapter 1
                           Introduction




                           smarter” ways of reducing or preventing pollution and to recommend
                           changes in the existing approach to environmental management, including
                           the existing statutes and regulations. According to EPA, “cleaner” means
                           that participating industries would achieve real, measurable improvements
                           in environmental protection; “cheaper” means that tailoring environmental
                           protection requirements to individual industries would save billions of
                           dollars; and “smarter” means that giving industry more flexibility in
                           meeting strong environmental goals would promote creativity and
                           encourage the development of innovative technology.

                           The Initiative was designed to achieve greater environmental protection at
                           less cost by creating strategies for controlling and preventing pollution for
                           individual industries rather than for individual pollutants, as is currently
                           done. EPA selected six industrial sectors—automobile manufacturing,
                           computers and electronics, iron and steel, metal finishing, petroleum
                           refining, and printing—to participate in the Initiative.

                           The Initiative is organized under the Federal Advisory Committee Act
                           (FACA) and comprises a Council and six specialized subcommittees, each
                           of which focuses on a particular industrial sector and carries out studies
                           and projects to develop recommendations to the EPA Administrator for
                           administrative, regulatory, and statutory changes. EPA administers the
                           Initiative; determines its multistakeholder membership; and publicly
                           reports on its FACA-based operations, activities, and costs, as well as on the
                           results of Council and subcommittee meetings.


Initiative’s Charter and   In accordance with the requirements of FACA, a charter establishing the
Organization               Council was filed with the Congress in October 1994. According to the
                           charter, the Initiative is designed to bring affected stakeholders together to
                           find “cleaner, cheaper, smarter” environmental management solutions. The
                           objectives and scope of the Council’s activities include holding meetings,
                           analyzing issues, conducting reviews, performing studies and site-specific
                           projects, producing reports, and making consensus recommendations on
                           issues related to the following program elements: regulation; pollution
                           prevention; recordkeeping and reporting; compliance and enforcement;
                           permitting; and environmental technology. The charter also provides for
                           expanding activities to include additional elements.2 While the Initiative’s
                           charter was originally established for 2 years, it was renewed for 2 more
                           years in October 1996.

                           2
                            Current projects are also addressing issues such as community technical assistance/community
                           involvement, brownfields, publicly owned treatment works, access to capital, and industry strategic
                           planning.



                           Page 13                                      GAO/RCED-97-164 EPA’s Common Sense Initiatives
                          Chapter 1
                          Introduction




                          Under the charter, the Council consists of a group of independent experts
                          appointed by the EPA Administrator. These experts are selected from
                          industry; state and local regulatory agencies; national and local
                          environmental groups; and other stakeholder categories such as labor,
                          tribal, environmental justice, and community organizations. Members are
                          sought in a variety of ways, including consultation with affected
                          stakeholders, industries and industry associations, senior EPA managers,
                          agency staff involved in supporting the Council, Council members, and the
                          general public.

                          The charter also provides for the Administrator to designate members to
                          serve as the Chair and Vice-Chair of the Council and its subcommittees.
                          Currently, the Council is chaired by the EPA Administrator and cochaired
                          by the Deputy Administrator, while each subcommittee is cochaired by an
                          EPA assistant administrator and regional, or deputy regional, administrator.
                          The Council and its subcommittees are supported by a team of
                          headquarters and regional EPA staff.

                          Among other policies, FACA requires open meetings of the Council and its
                          subcommittees and provides for interested persons to attend Council
                          meetings and appear before or file statements with the Council. EPA
                          employees act as the designated federal officers (DFO) for the Council and
                          its subcommittees, and the DFO assigned to each committee is present at
                          all federal advisory committee meetings.


Initiative’s Activities   As of April 1997, the Council’s subcommittees were testing new concepts
                          in about 40 projects addressing such areas as pollution prevention,
                          streamlined permitting and reporting, enhanced public participation in
                          environmental management, brownfields3 redevelopment, the use of
                          innovative technologies, and the setting of environmental goals and
                          indicators. The Council and its subcommittees anticipate that these
                          projects will lead to significant recommendations for changes in
                          regulatory structure and approach, which will result in “cleaner, cheaper,
                          smarter” environmental management.


Initiative’s Costs        Each year, EPA is required to report the costs associated with its federal
                          advisory councils to the General Services Administration, which in turn
                          prepares a report on all federal advisory committees for the Office of

                          3
                           Brownfields are abandoned or underused facilities, usually in industrial or commercial areas, where
                          redevelopment is hampered by real or perceived environmental contamination.



                          Page 14                                      GAO/RCED-97-164 EPA’s Common Sense Initiatives
                     Chapter 1
                     Introduction




                     Management and Budget. For fiscal year 1995, EPA reported that the
                     Council incurred costs of $2.8 million and employed 19.6 full-time
                     equivalent (FTE) staff. For fiscal year 1996, reported costs and staffing
                     were $3.6 million and 27.3 FTEs. The costs reported by EPA, however,
                     include only the costs directly associated with the scope and duties of the
                     Council and its subcommittees—such as portions of the salaries of the
                     designated federal officers and other administrative costs for scheduling
                     meetings, funding invitational travel, and complying with other reporting
                     requirements under the act—and do not include such items as grants or
                     the costs of contracts for the individual subcommittee projects, which are
                     funded by EPA’s program offices. For fiscal year 1997, EPA estimated costs
                     of $4.3 million and staffing of 30.4 FTEs for the Council.


                     To address questions about EPA’s Common Sense Initiative and the
Objectives, Scope,   progress it has made, the Chairman, Subcommittee on VA, HUD, and
and Methodology      Independent Agencies, Senate Committee on Appropriations; Chairman,
                     Senate Committee on Environment and Public Works; Chairmen, House
                     Committee on Commerce and its Subcommittee on Finance and
                     Hazardous Materials; Chairmen, House Committee on Transportation and
                     Infrastructure and its Subcommittee on Water Resources and
                     Environment; Chairman, Subcommittee on VA, HUD, and Independent
                     Agencies, House Committee on Appropriations; and the Chairman,
                     Subcommittee on National Economic Growth, Natural Resources, and
                     Regulatory Affairs, House Committee on Government Reform and
                     Oversight, asked GAO to assess (1) EPA’s progress in achieving the goal the
                     agency set for the Initiative, and (2) the methods EPA uses to measure the
                     progress of the Initiative toward its goal.

                     To address these objectives, we reviewed the Initiative’s program and
                     project files and interviewed program officials and staff, as well as
                     subcommittee sector officials and project team leaders. We also attended
                     meetings of the Council, its subcommittees, and project workgroups. In
                     addition, we obtained program information from EPA’s Common Sense
                     Initiative World Wide Web Internet site.4

                     We also reviewed reports on the Initiative, including a February 1997
                     EPA-funded study, a March 1996 EPA Inspector General’s report discussing
                     increases in FACA-related costs, and two reports critical of the



                     4
                      EPA’s Common Sense Initiative Internet address is: http://www.epa.gov/commonsense



                     Page 15                                    GAO/RCED-97-164 EPA’s Common Sense Initiatives
                     Chapter 1
                     Introduction




                     Initiative—one by the House Transportation and Infrastructure Committee
                     on EPA’s reinvention efforts and the other by Resources for the Future.5

                     In addition, we identified and reviewed the goals and objectives for 36 of
                     the Initiative’s projects that were ongoing as of September 1996. We
                     further reviewed 15 of the 36 projects using a results-oriented performance
                     measurement model to aid in determining whether the projects were
                     designed to (1) achieve the Initiative’s cleaner, cheaper environmental
                     management objectives and (2) measure progress toward achieving those
                     objectives. Our model applies the concepts underlying the Government
                     Performance and Results Act (GPRA) of 1993, which focuses on program
                     outcomes, comparing the results of a program activity to its intended
                     purpose rather than focusing on activity levels and tasks completed. The
                     act—known as GPRA or the Results Act—also incorporates performance
                     measurement as one of its most important features. Appendix I describes
                     our performance measurement model and lists the projects reviewed using
                     this model. Our detailed review of 15 projects represents a cross section of
                     projects from each of the six industrial sectors and 10 of the program
                     elements (e.g., permitting, innovative technology, compliance, and
                     pollution prevention) addressed by the projects.

                     We conducted our work from July 1996 through June 1997 in accordance
                     with generally accepted government auditing standards.


                     We provided a draft of this report to EPA for review and comment. EPA
Agency Comments      provided written comments, which are reproduced in appendix II. EPA
and Our Evaluation   agreed with the report’s recommendations and stated that the agency has
                     begun to address the recommendations in the changes being implemented
                     and considered for the Common Sense Initiative. However, EPA expressed
                     concern that our review’s focus on the Initiative’s progress in
                     accomplishing regulatory, statutory, or administrative changes (1) does
                     not adequately recognize the breadth of the Initiative’s accomplishments
                     and (2) is an unfair measure of the Initiative’s progress at this point in its
                     development.




                     5
                      Review of the Common Sense Initiative, The Scientific Consulting Group, Inc. (Feb. 19, 1997); EPA
                     FACA Committees’ Costs Increase, EPA, Office of Inspector General (Audit Report No.
                     E1XMG5-13-0071-6100147, Mar. 29, 1996); An Assessment of EPA’s Reinvention, House Committee on
                     Transportation and Infrastructure (Sept. 17, 1996); and Industry Incentives for Environmental
                     Improvement: Evaluation of U.S. Federal Initiatives, Resources for the Future (Sept. 1996). Resources
                     for the Future is an independent nonprofit organization that conducts research and public education
                     on natural resources and environmental issues.



                     Page 16                                      GAO/RCED-97-164 EPA’s Common Sense Initiatives
Chapter 1
Introduction




We recognize that the Initiative is complex and that its success is
dependent on accomplishing numerous tasks and activities. We believe,
however, that our emphasis on the statutory, regulatory, and
administrative changes expected to result from the Initiative is appropriate
and that we have adequately recognized the breadth of accomplishments
that EPA is reporting for the Initiative. We believe our focus is appropriate,
given that EPA (1) considers the Initiative “the centerpiece” of its
regulatory reinvention efforts and (2) established a goal for the Initiative
of finding “cleaner, cheaper, smarter” ways of reducing or preventing
pollution and recommending changes in the existing approach to
environmental management, including the existing statutes and
regulations. Also, as our report points out, we used an approach that
applies the concepts underlying GPRA, which focuses on program
outcomes—the results of a program activity compared to its intended
purpose—rather than activity levels and tasks completed. As our report
states, the accomplishments reported by EPA include various processes
and activities that the Initiative has undertaken to achieve its goal.
Although we did not discuss all of these accomplishments, the report
summarizes them and recognizes their importance and utility as a means
for achieving the expected statutory, regulatory, and administrative
changes.

We agree that time is required to identify, test, and evaluate innovative
approaches to achieving environmental improvements. However, we also
believe that our report fairly assesses the progress of the Initiative toward
its stated goal at the time of our review and discusses the factors that
prevented it from achieving the Administrator’s expectation—namely, that
it would bring about a fundamental change in the approach to
environmental protection and achieve tangible results in the first year. Our
report points out that the progress of the Initiative to date has been slow in
view of the high expectations EPA set for it and discusses a number of
factors that contributed to this slow progress. In its comments on our draft
report, EPA agreed that it underestimated the time required to do the things
needed to allow the development of recommendations for regulatory,
statutory, or administrative changes. It said, however, that barriers had to
be reduced before environmental results could be accomplished.




Page 17                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
Chapter 2

Several Factors Have Limited the Initiative’s
Progress in Bringing About Changes in
Environmental Management
                          EPA expected the Common Sense Initiative to result in significant
                          improvements to current regulations, as well as proposals for the
                          Congress to consider when legislative reform might be required. The
                          Administrator, recognizing that some changes would take longer,
                          expressed her belief that the Council could recommend tangible changes
                          in the first year. Although the Initiative’s six subcommittees moved quickly
                          to identify issues and develop projects, few recommendations have been
                          developed. To date, the Council has made three formal recommendations
                          to EPA. None of these recommendations suggested the types of changes in
                          existing regulations or legislation that EPA had expected. (The
                          recommendations are discussed further in ch. 3.)

                          Several factors have contributed to the Initiative’s slow pace in developing
                          recommendations to EPA. Some of these factors include (1) the time
                          needed to collect and analyze data; (2) stakeholders’ difficulty in obtaining
                          consensus on projects addressing broad, complex issues; (3) stakeholders’
                          time commitments and differences in stakeholders’ understanding of the
                          technical aspects of environmental issues; and (4) the time spent resolving
                          questions about work processes.

                          The lack of tangible results has caused some industry representatives to
                          question their continued participation in the Initiative. To determine the
                          effectiveness of the Initiative’s process, EPA contracted for an independent
                          review. The contractor reported that, overall, the Initiative has value, but
                          identified several areas where improvements in the process could be
                          made. EPA and the Council are now considering various options for
                          correcting problems in the process identified by the contractor.


                          In July 1994, when the EPA Administrator announced the establishment of
Recommendations for       the Common Sense Initiative, she said that consensus proposals generated
Change Were               by the Initiative would be designed to better protect the environment,
Expected in the           reduce U.S. pollution overall, and reduce the costs that industries face by
                          billions of dollars. Consistent with the Administrator’s belief that the
Initiative’s First Year   Council could make recommendations for change within the first year,
                          soon after the Council and its six subcommittees were established and
                          began meeting in late 1994 and early 1995, the senior EPA officials who
                          chair each of the subcommittees challenged their groups to quickly define
                          issues and develop projects to carry out their mission. In opening remarks
                          to the May 1995 meeting of the computers and electronics subcommittee,
                          the EPA assistant administrator cochairing the meeting said that the
                          success of the Initiative depends, in part, on the development and



                          Page 18                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
                       Chapter 2
                       Several Factors Have Limited the Initiative’s
                       Progress in Bringing About Changes in
                       Environmental Management




                       implementation of concrete activities that represent a new way to do
                       business. In this meeting, as well as in another subcommittee meeting, the
                       senior EPA officials indicated that the subcommittee members should try to
                       have proposals or recommendations to the full Council within 6 to 12
                       months.


                       The subcommittees have had difficulty completing projects that provide a
Several Factors Have   basis for making recommendations to the Council. Many of the projects
Contributed to the     initially undertaken by the subcommittees’ workgroups are still in
Initiative’s Limited   progress. Twenty-nine of the 38 projects we identified as ongoing as of
                       April 1997 were begun in 1995—the first full year of the Council’s
Progress               operation. Several factors, as discussed below, have contributed to the
                       length of time it is taking to complete projects that provide a basis for the
                       subcommittees to make recommendations to the Council.


Data Collection and    As some workgroups began projects, they found that the information they
Analysis Add Time to   needed to carry out the projects was not readily available. In some cases,
Process                the information could be obtained by doing a literature search. For
                       example, within the automobile manufacturing sector, the team working
                       on the alternative sector regulatory system/community technical
                       assistance project found that it needed to gather information on
                       community and plant interactions into a database to inform the project’s
                       process. Through on-line research, the team developed a bibliography that
                       met its needs.

                       In other cases, however, workgroups needed considerable time to develop
                       information through surveys of stakeholders or contracted studies.
                       Obtaining the information has, in some cases, taken months. For example,
                       in 1995, the reporting and information access workgroup in the computers
                       and electronics sector undertook a project to develop a combined uniform
                       report for the environment. They then decided to develop this report
                       through a pilot project in one state. First, they had a contractor identify all
                       of the reporting requirements that a computer and electronics firm in the
                       state would have to meet. Next, they began to identify the environmental
                       information needs of the various stakeholder groups. A contractor is
                       developing this information through the use of focus groups. Gathering
                       information for the needs assessment alone is expected to take about 6
                       months. Several more steps, which will take more than a year, are planned
                       before the project can be completed.




                       Page 19                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
                          Chapter 2
                          Several Factors Have Limited the Initiative’s
                          Progress in Bringing About Changes in
                          Environmental Management




                          Other information requested by workgroups and subcommittees—only
                          some of which is readily available, particularly by industry—includes
                          information on wastes, hazardous waste sites, accidents and spills, human
                          health and safety, environmental effects, chemical use and releases, water
                          and energy use, compliance histories, descriptions of processes, profiles of
                          stakeholders, demographic information, cost accounting information
                          needed to determine environmental costs, reviews of environmental
                          programs, and existing laws and regulations.

                          EPA is not always able to provide the information requested by the
                          workgroups. In some instances, the information is difficult for the agency
                          to obtain. For example, if EPA requests data through a survey of 10 or more
                          entities, it must have the survey approved by the Office of Management
                          and Budget and must estimate the paperwork burden associated with the
                          request. In these instances, EPA has suggested that stakeholder groups
                          obtain the information by polling their members. In other instances, the
                          information may be confidential and EPA cannot release it. For example, a
                          subcommittee that wanted to assist EPA in developing effluent guidelines
                          for the iron and steel industry asked whether the results of a survey to be
                          conducted could be made available to the subcommittee’s members.
                          Because the survey’s results would contain confidential business
                          information, EPA could not release the results without separating the data
                          from specific facilities. According to an EPA official, this effort might take a
                          year. Subsequently, the workgroup decided not to pursue this project.


Stakeholders Have Had     Although some projects have been designed to address complex
Difficulty Obtaining      multimedia issues, some workgroups have not been able to reach
Consensus on Approaches   agreement on specific objectives for their projects or on the approach to
                          be used, particularly when complex issues are involved. In some cases, to
to Projects               reach agreement, the project’s scope or the product has been modified or
                          narrowed. For example, in March 1995, the iron and steel subcommittee
                          was considering a proposal by one of its workgroups to develop and
                          pilot-test a multimedia permit for a plant. The subcommittee sent the
                          proposal back to the workgroup for reassessment, expressing concern
                          about the feasibility of the project, in part because (1) EPA was currently
                          developing new air-permitting requirements that alone would be difficult
                          to implement and (2) EPA had tried and failed to develop a less extensive
                          consolidated permit approach in the 1980s. The subcommittee suggested
                          that the workgroup talk to individuals who had been involved in the earlier
                          process or focus on a smaller facility, or a limited process within a facility.




                          Page 20                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
                        Chapter 2
                        Several Factors Have Limited the Initiative’s
                        Progress in Bringing About Changes in
                        Environmental Management




                        The proposed project was subsequently modified to, among other things,
                        be tested in a smaller facility, and it is now in progress.

                        When the automobile manufacturing subcommittee met in February 1996,
                        the spokesperson for one project team, commenting on two interim
                        reports from the team, said that the team was looking for potential
                        recommendations, but he questioned the ability of the team to reach
                        consensus on any recommendations. The subcommittee plans to submit
                        these two reports without recommendations to the Council at its July 1997
                        meeting.

                        At an August 1996 meeting of the automobile manufacturing
                        subcommittee, some members expressed concern about the complexity of
                        the issues being dealt with in project teams, the lack of common
                        understanding among the team members, and the slow pace of dealing
                        with complex issues. To address these concerns, the EPA cochair of this
                        subcommittee noted that a variety of technical assistance and support
                        continued to be available for those who needed it. At that same meeting,
                        another participant suggested that the Initiative’s whole process was too
                        abstract and that the goal of designing a new system was beyond the
                        subcommittee’s abilities; therefore the group should focus on developing
                        the new system one project at a time. Still another participant noted that
                        the subcommittee is attempting to address the big picture and suggested
                        that the group might be more comfortable working at a more pragmatic
                        level. As discussed in the next section, stakeholders’ difficulty in achieving
                        consensus on projects that involve broad, complex issues has resulted, in
                        part, in narrowing the scope of some projects.


Narrow-Scope Projects   Generally, subcommittees have had more success in designing and
Have Achieved Some      implementing narrow-scoped projects. For example, one such project
Success                 within the computers and electronics sector appears to be making
                        progress and will likely result in a recommendation to the Council. This
                        project, undertaken by the computers and electronics subcommittee’s
                        reporting and public access workgroup in 1996, was designed to
                        streamline the emergency response planning process. It will do this by
                        consolidating various requirements for an emergency response plan so
                        that only one document will meet the needs of all stakeholders (including
                        emergency responders, the community, facility workers, and regulatory
                        agencies). This project, which could be completed within the year, has an
                        objective similar to that of the project to develop a combined uniform
                        report for the environment (discussed earlier in this chapter). However,



                        Page 21                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
                        Chapter 2
                        Several Factors Have Limited the Initiative’s
                        Progress in Bringing About Changes in
                        Environmental Management




                        because this project addresses only one reporting requirement for
                        emergency response plans, it will be completed much sooner than the
                        combined uniform report for the environment, which addresses all
                        environmental reporting requirements for firms in the computers and
                        electronics industry. One workgroup member compared the two projects,
                        saying that the emergency response project was “just the toe of the
                        elephant,” whereas the combined uniform report for the environment
                        project was “the entire elephant.”

                        Whether narrowly scoped projects, such as those designed to provide
                        information alone, will produce results consistent with the Initiative’s goal
                        of developing recommendations for change is unclear. For example, a
                        workgroup in the iron and steel subcommittee sponsored a spent pickle
                        liquor6 workshop for the subcommittee members, and a workgroup in the
                        computers and electronics subcommittee cosponsored, with the National
                        Safety Council, an electronic product recovery and recycling conference.
                        The primary purpose of both of these projects was to provide a forum for
                        exchanging and disseminating information. Although the information
                        obtained in these forums may assist the workgroups in making
                        recommendations to the Council and ultimately to EPA, neither of these
                        projects was designed to result in recommendations to the Council.


Stakeholders’ Time      Limits on stakeholders’ time commitments and understanding of the
Commitments and         technical aspects of various environmental issues have also slowed the
Understanding of        pace of projects. EPA has recognized that participating in the Council
                        requires a large commitment of time and effort from its members, but once
Technical Issues Vary   they were selected, they were expected to participate. However, sporadic
                        attendance and attrition have been problems for the Council and some of
                        its committees and workgroups. In the computers and electronics
                        subcommittee, for example, the cochair noted in August 1995 that some
                        individuals did not attend enough meetings to completely understand what
                        was going on and were therefore slowing down the decision-making
                        process.

                        In addition, stakeholders’ understanding of the technical aspects of
                        environmental issues varies, and insufficient understanding on the part of
                        some has slowed some projects. After hearing from some workgroup
                        members that they did not fully understand the issues being discussed, the
                        cochair of the automobile manufacturing subcommittee promised to slow

                        6
                         Pickle liquor is the acidic chemical solution used to remove surface scale and other impurities from
                        steel. It becomes spent after its effectiveness has been exhausted.



                        Page 22                                       GAO/RCED-97-164 EPA’s Common Sense Initiatives
                           Chapter 2
                           Several Factors Have Limited the Initiative’s
                           Progress in Bringing About Changes in
                           Environmental Management




                           the current pace of the subcommittee and its workgroups to accommodate
                           individuals who needed time to become familiar with the technical aspects
                           of the issues.

                           The Council recognized that people have different amounts of time to
                           invest in projects. To correct the problem of poor or sporadic attendance,
                           an EPA official proposed undertaking fewer projects in order to make sure
                           that they had adequate resources. Reducing the number of projects could,
                           however, limit the range of issues that the Initiative is able to address.


Work Process Issues Have   In addition to discussing substantive work on projects, subcommittee and
Consumed Considerable      workgroup members have spent much of their meeting time discussing
Time                       processes. In particular, they have discussed (1) the definition of
                           consensus and how it would be applied within the subcommittees and
                           workgroups; (2) the scope of the standard individual projects or groups of
                           projects had to meet—whether “cleaner, cheaper, and smarter” criteria or
                           a “cleaner, cheaper, or smarter” criterion—and, (3) the representation of
                           stakeholders on subcommittees, workgroups, and projects. Because EPA’s
                           guidance for the Initiative did not adequately clarify these issues, the
                           subcommittees and workgroups took time to develop their own answers.

                           Although the Council’s operating principles, published in June 1996,
                           contain an operating definition of consensus, stakeholders are not certain
                           whether the same definition applies to workgroups. As late as
                           August 1996, the automobile manufacturing subcommittee was discussing
                           whether workgroups needed to reach consensus. The workgroups were
                           encouraged to bring issues on which they could not agree to the full
                           subcommittee for discussion. One subcommittee member pointed out,
                           however, that consensus is difficult to obtain unless projects contain
                           something for everyone. In order to facilitate reaching agreement on
                           projects, the sector decided that members of workgroups who were not
                           also members of the subcommittee could sit at the subcommittee table to
                           present their views.

                           In some instances, reaching consensus on which projects should be
                           undertaken was difficult or impossible because workgroup and
                           subcommittee members held differing views on whether and how the
                           Initiative’s “cleaner, cheaper, smarter” environmental management goal
                           should be incorporated into projects and the overall work of the
                           subcommittees and workgroups. In carrying out the Initiative, “cleaner” is
                           seen as the principal interest of the environmental representatives and



                           Page 23                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
Chapter 2
Several Factors Have Limited the Initiative’s
Progress in Bringing About Changes in
Environmental Management




“cheaper” is seen as the principal interest of the industry representatives.
In several groups, a project’s approval or progress was hindered as
members discussed the need for balance between these elements. For
example, in an iron and steel subcommittee’s workgroup considering
multimedia permitting, the workgroup was able to reach consensus on
solutions for 12 of the 50 permit issues it had identified. According to a
workgroup member, the solutions developed through the process included
cleaner, cheaper, and smarter elements. The need to consider all three of
these elements was also a factor discussed during the process for
approving a project within another of the iron and steel subcommittee’s
workgroups, which was considering alternative compliance strategies.
Subcommittee members expressed concern that under the project’s plan,
the project would provide regulatory relief (cheaper) but would not
provide for increased environmental protection (cleaner). In this case, the
problem was resolved by eliminating any reference to potential solutions
until an initial data-gathering and analysis phase could be completed.

For one of the automobile manufacturing subcommittee’s workgroups, a
team considering regulatory initiatives tentatively agreed on a proposal
that a regulatory determination requiring the use of the best available
control technology be based on the best technology available at the time
the application for a permit is submitted. However, because some
members of the team believed the proposal addressed the “cheaper” and
possibly the “smarter” but not the “cleaner” element, they would not allow
the proposal to be brought to the subcommittee until a package of
proposals addressing all three elements could be developed. The activities
of that workgroup were later suspended, and that proposal was not
forwarded to the subcommittee.

The representation of stakeholders within workgroups has also been the
subject of much discussion. In designing a project to identify regulatory,
statutory, and administrative barriers that hinder the development and
implementation of technology, a workgroup within the iron and steel
subcommittee proposed to address the definition of solid waste as one of
those barriers. At meetings where the proposal was discussed, members of
both the subcommittee and the workgroup were concerned because no
representative of an environmental group was included in the workgroup.
Initially, efforts to recruit an environmental stakeholder for this
workgroup failed. According to one subcommittee member, the difficulty
in getting a representative from an environmental group to participate in
this workgroup was directly related to the issue of defining solid waste.
This member said that the issue is highly contentious and could not



Page 24                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
                           Chapter 2
                           Several Factors Have Limited the Initiative’s
                           Progress in Bringing About Changes in
                           Environmental Management




                           previously be resolved through consensus; therefore, environmental
                           groups do not want to be drawn into a debate on it again. Without a
                           representative from an environmental group, the workgroup proposed that
                           it would limit its work to discussing the issue and would not develop
                           recommendations. Members of the subcommittee suggested, however,
                           that the workgroup stop working on the issue until an environmental
                           stakeholder could be recruited. After about 5 months, an environmental
                           representative was found. Two months later, the workgroup reported that
                           the solid waste issue was being dropped because consensus could not be
                           reached.

                           In another industrial sector, in July 1996, after discussing the need for
                           including representatives of all stakeholder groups in a proposed pilot
                           project, a workgroup decided to seek the subcommittee’s guidance on
                           which stakeholder groups needed to be represented in a pilot project to
                           ensure a credible process and a high-quality outcome. The group also
                           asked about providing financial support to ensure participation and about
                           EPA’s determination that it was not legal to provide grant funds to support
                           a local nongovernmental organization’s participation in an Initiative
                           workgroup.

                           Subcommittees and workgroups also discussed other work process issues
                           such as (1) how funding levels for projects were determined; (2) what
                           types of projects were appropriate for workgroups to undertake, what
                           types of products were appropriate to forward to the Council, and how the
                           success of the products and of the subcommittees would be determined;
                           and (3) how the Initiative’s projects would be coordinated with the work
                           of other EPA programs addressing the same issues.


Slow Progress Has Led      The difficulty in making progress within certain subcommittees has led the
Some Industry              representatives of some industries to question the value of their continued
Representatives to         participation in the Initiative. In early 1997, groups representing two
                           industrial sectors—automobile manufacturing and petroleum
Question Their Continued   refining—indicated that their members would not participate in any new
Participation              sector activities. Some representatives of these two industrial sectors
                           stated that their companies could not justify their continued participation
                           without tangible results. Since the Initiative began, the two sectors have
                           forwarded only one recommendation to the Council.

                           Although some individual members chose not to continue their
                           participation in the Initiative, most of them did agree to continue for



                           Page 25                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
                         Chapter 2
                         Several Factors Have Limited the Initiative’s
                         Progress in Bringing About Changes in
                         Environmental Management




                         another year so that they could judge whether changes in the Initiative,
                         expected in response to our review and the contractor’s (described
                         below), would allow the Council to function more effectively.


                         Recognizing that some of these factors had slowed the Council’s progress,
EPA and the Council      EPA, in November 1996, retained a contractor to perform an independent
Are Considering          review of the Initiative. The purpose of the study was to review the
Options to Address       Initiative after 2 years of operation, determine its level of success, and
                         identify any changes needed to maximize its effectiveness. The
Issues Identified in a   contractor’s report, issued on February 19, 1997, concluded that the
Contractor’s Review      Initiative has significant value, particularly as a tool to improve
                         environmental policy and management. Its value for developing more
                         effective environmental protection approaches, according to the report, is
                         that it includes all relevant stakeholders in a nonadversarial,
                         consensus-based forum to address environmental issues by industrial
                         sectors in a comprehensive, multimedia fashion.

                         According to the contractor’s report, the Initiative could not realistically
                         be expected to establish itself; form working relationships among
                         disparate stakeholders; and accomplish changes in regulations, reductions
                         in reporting burdens, or other time-intensive changes in the relatively
                         short term of its existence. The report identified four major unresolved
                         issues—consistent with those we identified during our review—including
                         (1) the Initiative’s lack of specific objectives, (2) the lack of a clearly
                         defined role for the Council, (3) the perceived need for unanimity to obtain
                         consensus, and (4) the slow pace of the process. However, the report did
                         not make any recommendations to resolve these issues. EPA and the
                         Council are now considering a number of options for addressing the issues
                         identified in the contractor’s report.


                         Nine of the Initiative’s 38 projects ongoing as of April 1997 involve
Need for Legislative     developing pilot projects to test multimedia permitting, reporting, or
or Regulatory            flexible regulation. Stakeholders and others involved in these types of
Changes Is Unclear       projects have raised questions about whether EPA has the legislative or
                         regulatory authority to carry out such pilot projects. In one pilot
and Could Delay          multimedia permitting project, for example, a February 1997 contractor’s
Some Projects’           report7 to EPA said that the state where the pilot project is located was
                         considering legislation that would provide for enforceable agreements
Implementation           with facilities that would modify or waive existing statutory or regulatory

                         7
                          Multi-Media Permit Process Report, Eastern Research Group, Inc. (Feb. 1997).



                         Page 26                                     GAO/RCED-97-164 EPA’s Common Sense Initiatives
              Chapter 2
              Several Factors Have Limited the Initiative’s
              Progress in Bringing About Changes in
              Environmental Management




              requirements if the proposed pollution prevention, reduction, or control
              strategies resulted in equivalent or greater overall benefits to human
              health and the environment. The contractor’s report noted that the
              proposed state legislation could eliminate any potential problems for a
              multimedia permit with state statutory or regulatory requirements;
              however, it would not eliminate any potential conflicts with federal
              statutory or regulatory requirements. For example, federal statutes and
              regulations governing the various environmental media may impose
              different monitoring, reporting, and other requirements that may make it
              more difficult for a multimedia permit program to be developed and
              implemented.

              Generally, EPA has maintained that it has the authority to carry out these
              types of pilot projects under the current statutory framework. However,
              whether regulatory or legislative changes will be needed to carry out
              specific Initiative projects has not been fully evaluated. Uncertainty about
              the degree of flexibility in existing federal, state, or local regulations could
              delay some projects, particularly those involving multimedia permitting,
              reporting, or flexible regulatory approaches. As we noted in an earlier
              report, this issue affects other EPA reinvention programs as well as the
              Initiative.8 According to the Deputy Administrator of EPA, the agency will
              reexamine this issue when it receives the recommendations of a key
              advisory group (the Enterprise for the Environment) later this year.


              Although some projects are under way to test approaches for reinventing
Conclusions   EPA’s regulatory approach to environmental protection, the progress of the
              Initiative has been slow in view of the high expectations EPA set for it. In
              setting these expectations, EPA underestimated the time required to
              (1) gather and analyze the information needed as a basis for developing
              recommendations and (2) establish the relationships among the various
              stakeholder groups needed for them to reach agreement on complex
              issues. Although most stakeholders have agreed to continue their
              participation in the short term, now that the process is taking longer than
              expected, it is unclear whether stakeholders, as a group or individually,
              will be willing and able to invest the resources required by a longer-term
              process. We believe that the process could be expedited if EPA would
              provide an improved operating framework better defining the Initiative’s
              goal and expected results and including specific guidance on how the
              Initiative will accomplish its work.

              8
               Environmental Protection: Challenges Facing EPA’s Efforts to Reinvent Environmental Regulation
              (GAO/RCED-97-155, July 2, 1997).



              Page 27                                    GAO/RCED-97-164 EPA’s Common Sense Initiatives
                 Chapter 2
                 Several Factors Have Limited the Initiative’s
                 Progress in Bringing About Changes in
                 Environmental Management




                 To permit the Initiative’s subcommittees and workgroups to devote more
Recommendation   of their attention to substantive issues, GAO recommends that the
                 Administrator, EPA, provide an improved operating framework that
                 (1) more clearly defines the Initiative’s “cleaner, cheaper, smarter”
                 environmental protection goal—including its expected results—and
                 (2) specifies how the Council and its subcommittees and workgroups will
                 accomplish their work, clarifying issues such as how and when consensus
                 will be achieved, how the Initiative’s goal should be interpreted and
                 applied to individual projects, and to what extent representatives of all
                 stakeholder groups should be included in activities at each level of the
                 Initiative, including projects and workgroups.




                 Page 28                                  GAO/RCED-97-164 EPA’s Common Sense Initiatives
Chapter 3

Results-Oriented Measures of Progress Are
Needed

                         The Environmental Protection Agency considers the Initiative to be the
                         most significant cross-program, multimedia initiative ever undertaken. The
                         agency believes that its Council and subcommittees will make significant
                         consensus recommendations to the EPA Administrator for changes in
                         regulatory structure and approach. Moreover, according to the agency, the
                         Initiative has already produced significant accomplishments on several
                         fronts. The accomplishments, however, primarily reflect the completion of
                         steps in the Initiative’s process—such as stakeholder meetings, pilot
                         projects, and various other subcommittee work products or activities.
                         Although such accomplishments are important, they do not reflect the
                         agency’s progress in meeting the Initiative’s goal, consistent with GPRA’s
                         intent to have federal agencies focus on program results and outcomes.
                         EPA has not yet established results- or outcome-based measures for
                         assessing the extent to which the Initiative has reduced or prevented
                         pollution at less cost to industry and the taxpayer through regulatory
                         reinvention. The 1997 EPA contractor study of the Initiative noted that the
                         program’s complexity and relatively early stage were key reasons why EPA
                         did not have such measures.

                         Without performance-based measures for assessing the progress of the
                         Initiative in achieving its expected outcomes, EPA cannot determine to
                         what extent the Council’s or subcommittees’ activities have reduced or
                         prevented pollution at less cost to industry and the public. Particularly for
                         the subcommittee projects used to test reinvention alternatives, such
                         measures are needed to demonstrate that an expected outcome has been
                         achieved and that a subcommittee’s resulting recommendation for
                         administrative, regulatory, or statutory change is warranted.


                         In launching the Initiative in July 1994, the EPA Administrator established
Initiative Promises      high expectations for its success, describing it as unprecedented in scope
Significant Regulatory   and almost immediate in impact. According to the Administrator, the
Reinvention              Initiative is the “centerpiece” of the agency’s regulatory reinvention effort
                         and a new paradigm for environmental management and regulatory
                         reform. The Administrator presented the Initiative as a multifaceted
                         alternative to the status quo—capable of producing significant
                         improvements in environmental protection rather than incremental
                         successes, substituting recommendations based on consensus for
                         command and control regulation, and achieving tangible results within the
                         first year.




                         Page 29                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
                             Chapter 3
                             Results-Oriented Measures of Progress Are
                             Needed




                             EPA also said that all aspects of environmental policy—from reporting
                             requirements to significant statutory reform—would be on the table and
                             that consensus solutions within EPA’s control would be implemented
                             immediately. EPA further stated that if agreement emerged for legislative
                             change, then the EPA Administrator would use that support in working
                             with the Congress to change the law.

                             Clearly, EPA’s expectation for the Common Sense Initiative was that it
                             would bring about fundamental change in the approach to environmental
                             protection and achieve tangible results in the first year. As discussed
                             below, however, the accomplishments reported by EPA appear to
                             emphasize steps in the Initiative’s process rather than the achievement of
                             the fundamentally “cleaner, cheaper, smarter” environmental solutions
                             that EPA expected.


                             In the almost 3 years since the Common Sense Initiative’s start, EPA has
Accomplishments              reported significant results on many levels, including six
Focus Principally on         recommendations made by subcommittees to the Council. Three of these
Processes and Not on         recommendations were formally approved by the Council and forwarded
                             to EPA for implementation. For the most part, however, the
Regulatory Changes           accomplishments that EPA reported are steps in the process that EPA
                             designed to produce recommendations for the Council’s and, ultimately,
                             the EPA Administrator’s, consideration. Generally, they are not the
                             significant “cleaner, cheaper, smarter” outcomes or results that EPA said it
                             expects from the process.


Subcommittees’               According to EPA, at the time of our review, four of the six subcommittees
Recommendations to the       had formally presented recommendations to the Council for approval. In
Council                      order of presentation to the Council, they were as follows:

                         •   In October 1995, the metal finishing subcommittee recommended that EPA
                             remove iron and aluminum from the list of pretreatment standards for its
                             then-proposed metals products and machinery phase I effluent limitations
                             guidelines.
                         •   In October 1995, the computers and electronics subcommittee
                             recommended that EPA establish a process to ensure that EPA’s regulatory
                             interpretations and determinations—intended to affect the environmental
                             management practices of the regulated community—be compiled, made
                             easily accessible, and publicized to interested stakeholders.




                             Page 30                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
    Chapter 3
    Results-Oriented Measures of Progress Are
    Needed




•   In October 1995, the auto manufacturing subcommittee recommended that
    EPA expeditiously implement the streamlining of the 1990 Clean Air Act’s
    title V permit process as outlined in a July 1995 EPA white paper.9
•   In March 1996, the iron and steel subcommittee recommended that the
    Council transmit to EPA 10 guiding principles that the subcommittee
    proposed for EPA to apply, as appropriate, in establishing principles for use
    in the development of iron and steel brownfields sites.
•   In June 1996, the computers and electronics subcommittee recommended
    that the Council (1) endorse the vision, goals, and objectives contained in
    its proposed outline for a facility-based alternative system of
    environmental protection for the computers and electronics industry and
    (2) pass its proposal forward to EPA to be used as a framework for the
    many reinvention efforts going on at EPA.
•   In February 1997, the metal finishing subcommittee requested that the
    Council support its national metal finishing environmental research and
    development (R&D) plan, which recommended that (1) EPA and other
    federal agencies use the plan to coordinate and support research and
    development directed toward the needs of the metal finishing industry and
    (2) EPA use the plan as a possible approach for other industrial sectors’
    research and development plans.

    Of the three recommendations presented in October 1995, all but the metal
    finishing subcommittee’s recommendation were approved by the Council
    and, according to EPA, were being implemented by the agency at the time
    of our review. EPA reported that the Council also approved the metal
    finishing subcommittee’s recommendation in concept but returned it to
    the subcommittee for further work needed to resolve a technical issue that
    prevented consensus from being reached at the subcommittee level.
    According to EPA officials, however, the agency’s Office of Water was
    independently implementing the proposed recommendation as a result of
    the Council’s discussion of its merits.

    The Council also returned both of the 1996 recommendations to their
    respective subcommittees for further work. According to Council
    documents,

•   the iron and steel subcommittee’s guiding principles for the development
    of brownfields sites were returned with improvements suggested during
    the Council’s discussion and the principles’ formal transmission to EPA was



    9
     White Paper for Streamlined Development of Part 70 Permit Applications, EPA, Office of Air Quality
    Planning and Standards (July 10, 1995).



    Page 31                                      GAO/RCED-97-164 EPA’s Common Sense Initiatives
    Chapter 3
    Results-Oriented Measures of Progress Are
    Needed




    postponed so that the subcommittee could respond to the Council’s
    suggestions and test the guidelines in planned pilot projects and
•   the computers and electronics subcommittee’s proposed alternative
    system of environmental protection was returned for pilot testing at the
    subcommittee level.

    In reviewing the guiding principles for the development of iron and steel
    brownfields sites, Council members expressed concern about their
    language and policy implications and decided to send the guidelines back
    to the subcommittee describing the Council’s concerns and stressing the
    need for testing. According to the Council’s chair (the EPA Administrator),
    the Council acted appropriately, given the diverse responses to the
    subcommittee’s proposal, and EPA had already benefited from the
    discussion even without formally sending the recommendation forward.

    Responding to the computers and electronics subcommittee’s
    recommendation, the Council indicated its support for the subcommittee’s
    vision for an alternative system of environmental protection but wanted
    further development of alternative regulatory strategies through pilot
    projects addressing issues raised during the Council’s discussions. The
    Council urged other sectors to use the subcommittee’s vision document,
    as appropriate and consistent with the Council’s comments, in their
    discussion of alternative strategies. Of specific concern to the Council was
    the need to establish criteria for determining which facilities would qualify
    for an alternative system.

    According to EPA officials, the Council granted the metal finishing
    subcommittee’s February 1997 request for support of its national metal
    finishing environmental R&D plan. Instead of discussing the
    subcommittee’s recommendation, the Council provided its members with
    a copy of the plan and a “review and mail-back” form on which they were
    asked to indicate whether they did or did not support the plan and
    whether they had any comments. According to EPA officials, all of the
    responses supported the plan and the subcommittee was notified of the
    Council’s approval, which will be officially recorded at the Council’s next
    meeting, scheduled for July 1997. According to the subcommittee, with the
    Council’s support, the plan will be sent to senior management and
    research and development leaders in federal agencies and private-sector
    organizations and meetings will be arranged to discuss the plan’s
    implementation.




    Page 32                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
                                 Chapter 3
                                 Results-Oriented Measures of Progress Are
                                 Needed




                                 We observed during our review that, for the most part, the six
                                 subcommittee recommendations forwarded to the Council for approval
                                 generally did not request changes in existing regulation or a basic change
                                 in the way EPA carries out its programs. In fact, the iron and steel
                                 subcommittee specifically requested in its recommendation on the iron
                                 and steel industry’s brownfields sites that its proposed guiding principles
                                 not be formalized by EPA as a policy directive or regulation. Instead, the
                                 subcommittee believed that the principles should be regarded as a group
                                 of goals to be reached when considering what a brownfields policy should
                                 accomplish. We also observed during our review that none of the three
                                 recommendations approved by the Council and forwarded to EPA
                                 suggested the types of changes in the existing approach to environmental
                                 management that EPA expected.

                                 Although the recommendations approved thus far have not suggested
                                 changes in existing statutes or regulations, EPA believes that some of the
                                 Initiative’s current projects will result in such recommendations. In
                                 May 1997, EPA identified two ongoing projects that the agency believes will
                                 lead to recommendations for regulatory or statutory change in 1997 and
                                 nine projects that could lead to such recommendations between now and
                                 1999.


Initiative’s Processes and       In addition to counting formal recommendations, EPA also measures the
Activities                       Initiative’s progress by the various processes and activities that the
                                 Initiative has undertaken to achieve its stated purpose. According to EPA,
                                 these include

                             •   the Initiative’s ability to draw diverse stakeholders—some of them
                                 traditional adversaries—to the table with a commitment to work together
                                 to build trust in areas where, historically, there has been little gain or
                                 interaction;
                             •   more than 300 meetings held by the Council and its subcommittees and
                                 their workgroups during fiscal years 1995 and 1996 to first “sort out the
                                 process” and then to achieve common ground and develop new ways of
                                 achieving stronger, yet more flexible and cost-effective ways to protect
                                 public health and the environment;
                             •   almost 40 sectorwide projects that are under way and have the potential
                                 for significant change in environmental policy and management at all
                                 levels;
                             •   products and activities that are a part of the sectorwide projects, such as
                                 the pilot projects being used to test alternatives for achieving regulatory



                                 Page 33                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
    Chapter 3
    Results-Oriented Measures of Progress Are
    Needed




    flexibility and streamlined permitting or reporting processes or the
    development of tools for sharing information and providing guidance;
•   various other actions taken by subcommittees that did not require support
    from the Council or changes by EPA and are being implemented directly by
    the subcommittees, such as collaborative information exchange activities
    that involve trade associations, community organizations, government
    agencies, and others and include the establishment of advisory groups and
    cosponsored conferences; and
•   “catalytic benefits” that occurred, EPA said, only because stakeholders
    came together as part of the Initiative and include improved working
    relationships between domestic and international manufacturers on
    environmental matters and new customer-supplier relationships involving
    the recycling and recovery of certain materials.

    Such process-oriented activities make up the majority of the
    accomplishments that EPA reported for the Initiative at the time of our
    review. However, such measures of progress are not focused on outcomes
    and do not clearly indicate what specific contributions the activities have
    made or are expected to make in bringing about the ambitious results that
    EPA has established as expectations for the Initiative. For example, one of
    the claimed accomplishments is a subcommittee’s production of a draft
    guidance document. This document is intended to assist firms in meeting
    their environmental requirements and to encourage their adoption of
    alternatives for preventing pollution. Eventually, when the document has
    been finalized and its guidance has been implemented, it may produce
    outcomes consistent with the Initiative’s goals, including improved
    compliance and reduced pollution. At this stage, however, the document’s
    production is simply a step in the process of implementing the Initiative
    and counting its production as an accomplishment does not measure the
    Initiative’s progress in terms of environmental results.

    The formal recommendations acted on by the Council and the
    process-oriented accomplishments together appear to provide the
    principal measures of success that EPA is using to assess the Initiative’s
    progress. For example, they are the measures of progress that EPA has
    used in official status reports prepared to recharter the Council and to
    annually report its progress under FACA’s requirements. Also, as discussed
    below, EPA officials told us they had not yet established results-oriented
    performance measures for assessing the extent to which the Initiative or
    many of its subcommittees’ projects have produced the expected “cleaner,
    cheaper, smarter” environmental protection solutions.




    Page 34                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
                       Chapter 3
                       Results-Oriented Measures of Progress Are
                       Needed




                       Although the Congress, the administration, and EPA have stressed the
EPA Lacks              importance of establishing results-oriented performance measures for
Results-Oriented       evaluating programs’ success, EPA has not yet established performance
Measures for           measures for assessing the Initiative’s results. According to the 1997 EPA
                       contractor study of the Initiative, the program’s complexity and relatively
Assessing Progress     early stage were key reasons why EPA did not have such measures. Our
Toward Achieving the   review found that the subcommittees’ projects, like the Initiative as a
                       whole, generally do not establish or provide for performance measures to
Initiative’s Goal      gauge their progress toward finding cleaner, cheaper approaches to
                       environmental management.

                       In an era of tight budgeting, the federal government is emphasizing
                       results-oriented performance and requiring agencies to demonstrate the
                       outcomes their programs are accomplishing with the funds appropriated
                       to them. In 1993, before EPA established the Initiative, the Congress
                       enacted GPRA and the administration put in place the National Performance
                       Review (NPR), both of which called for establishing performance measures
                       to assess programs’ results. Specifically, GPRA requires agencies to clearly
                       define their missions, set goals, link activities and resources to goals,
                       measure their performance, and report on their accomplishments in a
                       manner that focuses on programs’ expected outcomes and measurable
                       results. In addition, a number of NPR recommendations were intended to
                       achieve more results-oriented management. These included a
                       recommendation that EPA establish measurable environmental goals and
                       develop performance measures for selected goals and strategies consistent
                       with GPRA. Collectively, these actions seek to focus federal management
                       and oversight on the outcomes or results of federal programs.

                       In November 1993, EPA established the National Environmental Goals
                       Project, which was designed, in part, to address GPRA’s and NPR’s
                       objectives. In December 1996, EPA issued a draft report10 on the project’s
                       results for government agencies to review. It stated the importance of
                       having clear, measurable environmental goals for establishing tangible
                       results that national environmental programs, such as the Initiative, should
                       aim to deliver, and for assessing real environmental progress. The draft
                       report specifically stated that such goals provide an environmental results
                       context for the Initiative and establish a basis for measuring its
                       environmental progress and effectiveness.

                       According to EPA officials and the agency’s February 1997 contractor study
                       of the Initiative, EPA has not established results-oriented performance

                       10
                         Environmental Goals for America With Milestones for 2005, EPA draft proposal (Dec. 20, 1996).



                       Page 35                                      GAO/RCED-97-164 EPA’s Common Sense Initiatives
    Chapter 3
    Results-Oriented Measures of Progress Are
    Needed




    measures for the Initiative, in part because of the Initiative’s complexity
    and the need for a range of measures to gauge performance. EPA officials
    said that they found it difficult to measure successes under the Initiative
    and have “agonized over how to evaluate” its projects. According to the
    February 1997 study, several factors made it difficult to develop and apply
    valid and appropriate measures of the Initiative’s success. The factors
    include the following:

•   The Initiative is at a relatively early stage. Most activities are still in
    process and their effects have not yet been felt.
•   The Initiative lacks measurable objectives, and participants bring their
    own agendas and objectives to the table.
•   It is difficult to demonstrate that the Initiative has caused particular
    environmental effects. Many factors affect environmental performance,
    and distinguishing the Initiative’s effects from other influences will always
    be difficult.

    The study also noted that the Initiative is a complex program that cannot
    measure success by a few quantitative indicators. The study concluded
    that

•   a wide variety of measures are needed to capture the Initiative’s
    effectiveness;
•   the Initiative should be evaluated at all levels, from individual projects to
    overall activities, and specific measures should be established for each of
    these levels;
•   evaluation and self-assessment should be built into the Initiative’s
    components and used to indicate success as well as to improve processes
    and products;
•   only a few aspects of the Initiative—its individual projects, for
    example—can be measured quantitatively using traditional measures of
    output and effect unless its goals for such things as environmental
    improvements or reductions in reporting burdens are made more specific
    and are quantified; and
•   good ideas on measures of success should be shared within the Initiative
    and with other reinvention efforts that are facing similar measurement
    challenges.

    To emphasize the importance of developing appropriate quantitative and
    qualitative measures for the Initiative and its individual projects, the study
    cautioned against generating numbers and statistics on activities that do
    not capture the Initiative’s important aspects. It noted, for example, that



    Page 36                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
                                Chapter 3
                                Results-Oriented Measures of Progress Are
                                Needed




                                counting the numbers of projects initiated or meetings held has some
                                limited utility but does not reflect the quality of the projects or their
                                appropriateness to an individual sector or to the Initiative as a whole.
                                Instead, the Initiative should measure its success in terms of results
                                achieved. According to the study, quantitative measures of success will be
                                most feasible at the project level. For example, some projects will be able
                                to estimate emission reductions, cost savings, or other measurable targets.
                                As discussed below, however, our detailed review of individual projects
                                found that few had such measures, either in place or included in their
                                design.


Success of Individual           The Initiative’s subcommittees are to carry out studies and projects to
Projects Is Difficult to        develop recommendations for administrative, regulatory, and statutory
Determine Because               changes. The recommendations and other ideas developed by these
                                subcommittees are to be presented to the Council, which in turn will
Projects Lack Performance       deliberate and provide advice directly to the EPA Administrator. While the
Measures                        subcommittees’ projects are typically designed to find cleaner and cheaper
                                approaches to environmental management, we found that the projects do
                                not typically establish or provide for performance measures to gauge their
                                progress toward achieving this goal.

                                We identified and reviewed the goals and objectives of 36 projects that
                                were ongoing as of September 1996. We further reviewed 15 of the 36
                                projects using a results-oriented performance measurement model
                                (discussed in app. I) to help determine whether the projects were designed
                                to (1) achieve the Initiative’s cleaner, cheaper environmental management
                                objectives and (2) measure progress toward achieving those objectives.
                                The projects we reviewed were typically designed to develop methods of
                                reducing pollution and/or reducing the cost and burden associated with
                                pollution prevention; however, the projects did not typically establish or
                                provide for performance measures to gauge their progress toward
                                achieving either their own objectives or the Initiative’s goal.

                                Minimizing and/or preventing pollution was a common goal for the
                                subcommittee projects we reviewed, and reducing or minimizing the costs
                                and burdens associated with preventing pollution was frequently cited as
                                an expected outcome, as in the following examples:

                            •   The iron and steel subcommittee’s multimedia permitting project is
                                expected to, among other things, develop a permitting system that will
                                create opportunities for preventing pollution, will be less costly than the



                                Page 37                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
                                Chapter 3
                                Results-Oriented Measures of Progress Are
                                Needed




                                individual medium-specific permits issued currently, and will reduce the
                                paperwork and administrative burdens imposed on facilities and
                                regulators by the current system.
                            •   The computers and electronics subcommittee’s consolidated emergency
                                response reporting project is expected to streamline the emergency
                                response planning process by consolidating the requirements for various
                                emergency response plans into one document that meets the needs of all
                                stakeholder groups (e.g. emergency responders, the community, facility
                                workers, and regulatory agencies).

                                These projects, like all of those we reviewed, were generally designed to
                                find ways to reduce or prevent pollution and/or reduce its associated
                                costs. However, the previously described projects were among those that
                                had not established performance measures to gauge the extent to which
                                they would reduce pollution and/or its costs. For example, the iron and
                                steel subcommittee’s project describes the advantages of a multimedia
                                permitting approach (e.g., time and cost savings, reduced paperwork and
                                administrative burden, and pollution prevention opportunities) but does
                                not provide for measuring the extent to which the project would reduce
                                pollution or its costs.


Few Projects Provide for        Of the 15 projects we reviewed in detail, 4 appeared to provide clearly for
Measuring Performance to        measuring the extent to which pollution would be decreased and/or its
Demonstrate Their Success       costs would be reduced. For example, the petroleum refining
                                subcommittee’s equipment leaks project is expected to result in the more
                                focused monitoring and control of the components that are most likely to
                                leak, as well as more cost-effective operations for individual refineries that
                                are given the flexibility to meet particular performance standards. This
                                project’s design includes the development of a performance-based audit
                                system for assessing whether the use of an alternative approach has
                                cost-effectively improved environmental performance and reduced
                                emissions. In addition, the subcommittee’s one-stop reporting and public
                                access project for developing a sector-based air emissions reporting
                                system includes in its design an assessment of the time and cost savings
                                expected from the revised system.

                                Although performance measures were not specifically built into another
                                project’s design, this project appeared to be establishing such measures.
                                The printing subcommittee’s New York education project, whose goal is to
                                incorporate the philosophy of pollution prevention into everyday work




                                Page 38                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
    Chapter 3
    Results-Oriented Measures of Progress Are
    Needed




    practices through education and outreach, currently has a “measurement
    team” working on how to measure the project’s success.

    A fourth project provides for measuring the cumulative performance of all
    of the metal finishing subcommittee’s projects. The subcommittee’s
    strategic goals initiative—essentially a strategic planning project to pull
    together the sector’s various activities and findings—was designed to
    establish performance measures to assess the cumulative effects of the
    subcommittee’s other ongoing projects. This project is examining methods
    for benchmarking performance for metal-finishing facilities and for the
    industrial sector. For example, measures planned for facilities include
    percentage improvements in resource utilization, percentage reductions in
    water and energy use, percentage reductions in organic and inorganic
    hazardous emissions, and reductions in compliance costs.

    While the metal finishing subcommittee is establishing performance
    measures to assess the cumulative effect of all of its projects through its
    strategic goals initiative, the subcommittee’s other projects apparently do
    not have performance measures to allow the subcommittee, and
    subsequently the Council, to gauge their individual success and assess
    whether a recommended change is warranted. Of the three other metal
    finishing subcommittee projects we reviewed, none specified individual
    performance measures for assessing whether the expected outcome was
    achieved.

•   The subcommittee expects that the widespread use of its metal finishing
    guidance manual will lead to improved compliance rates throughout the
    industry, particularly among smaller, information-poor shops. The project
    does not, however, provide for assessing whether the manual’s use has
    been widespread or has improved compliance rates throughout the
    industry and among specific industry segments.
•   The subcommittee’s promoting improved performance flexible track
    project is designed to promote improved performance through an
    alternative regulatory program for top-tier firms that might receive
    flexibility and incentives to seek ambitious environmental goals. The
    project is expected, among other things, to maximize environmental
    improvements and minimize costs. Although project data included some
    quantified “cleaner” and “cheaper” benefit goals, it was not clear how
    progress toward those goals would be measured. According to an EPA
    official, such measures will be part of test facilities’ project plans and
    memorandums of agreement that were not yet developed at the time of
    our review.



    Page 39                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
                              Chapter 3
                              Results-Oriented Measures of Progress Are
                              Needed




                          •   The subcommittee expects that the reengineered compliance reporting
                              process, developed through the regulatory information inventory team
                              evaluation project, will provide measurable benefits for regulators,
                              industry, and nongovernmental organizations. The expected benefits
                              include some quantified forms consolidation improvement goals, and the
                              project’s workgroup has identified the need to develop a benchmarking
                              mechanism for evaluating the project’s success. However, the project’s
                              design did not specify how progress toward the expected outcomes would
                              be assessed.


EPA Is Considering            As noted previously, EPA’s February 1997 contractor study concluded that
Performance Measurement       specific measures should be developed for evaluating the Initiative’s
Needs                         success at all levels of activity, including the project level—which should
                              lend itself best to the use of quantitative measures for assessing emission
                              reductions, cost savings, or other measurable targets. Such measures will
                              likely be needed for projects to demonstrate that an expected cleaner
                              and/or cheaper outcome has been achieved and that a recommended
                              change is needed. Currently, although the subcommittees’ projects are
                              generally designed to achieve cleaner, cheaper environmental
                              performance and to lead to recommendations for administrative,
                              regulatory, or statutory change, the projects’ success will be difficult to
                              demonstrate without appropriate performance measures. Consequently,
                              the projects that do not have appropriate performance measures and
                              cannot otherwise demonstrate the achievement of their expected
                              outcomes will not likely have a sufficient basis for recommending a
                              change in environmental approach.

                              At the time of our review, EPA officials and other Initiative stakeholders
                              were considering the February 1997 study’s findings and conclusions and
                              the results of the Council’s mid-February 1997 meeting, which focused on
                              opportunities for strengthening and improving the Initiative’s process and
                              the Council’s role. In addition, EPA has asked an advisory committee11 to
                              identify criteria for evaluating the progress and success of the agency’s
                              various reinvention efforts—including the Initiative. The Council’s next
                              meeting is scheduled for July 1997, and it is not clear what changes will be
                              forthcoming.

                              11
                                The committee is part of EPA’s National Advisory Council for Environmental Policy and Technology.
                              The Council is a federal advisory committee organized under the Federal Advisory Committee Act to
                              provide information and advice to the EPA Administrator and other EPA officials on policies for
                              managing the environment. The Council’s members include senior-level representatives of a wide
                              range of EPA’s constituents, including business and industry; academic, educational, and training
                              institutions; federal, state, and local government agencies and international organizations;
                              environmental groups; and nonprofit entities.



                              Page 40                                     GAO/RCED-97-164 EPA’s Common Sense Initiatives
                 Chapter 3
                 Results-Oriented Measures of Progress Are
                 Needed




                 While EPA’s Common Sense Initiative has made some progress in bringing
Conclusions      stakeholders together to work collaboratively on the Council and its
                 subcommittees, the agency does not have results-oriented performance
                 measures for assessing whether or to what extent the Council’s or
                 subcommittees’ activities have reduced or prevented pollution at less cost
                 to industry and the public through expected fundamental changes in EPA’s
                 regulatory approach. In the absence of such measures, much of the
                 success that EPA claims for the program is being measured by the
                 completion of activities or products that are a part of the Initiative’s
                 process for achieving desired goals or desired outcomes but are not
                 focused on outcomes. However, the February 1997 EPA contractor study of
                 the Initiative recognizes the limited usefulness of such measures for
                 assessing progress. Although the measures have value and involve
                 important elements, they focus on the means to the outcome that the
                 Initiative is trying to achieve and not on the “cleaner, cheaper, smarter”
                 environmental results that are its goal. In addition, EPA itself acknowledges
                 the importance of outcome-based performance measures for assessing
                 program progress, consistent with GPRA’s emphasis on the need for
                 agencies to focus on and achieve measurable program results.

                 Although EPA recognizes the need for outcome-based performance
                 measures, the agency has not developed such measures, in part because of
                 the Initiative’s complexity and the need for a range of measures to gauge
                 performance. While we agree that the program’s complexity makes
                 establishing performance measures difficult, in our opinion,
                 results-oriented performance measures focused on the program’s
                 expected outcomes are essential for EPA and others to assess the progress
                 of the Initiative toward its goal, at both the Council and the subcommittee
                 levels, and to determine whether subcommittee projects have achieved
                 their expected outcomes. Such outcomes are intended to provide a basis
                 for subcommittees, and subsequently the Council, to recommend that an
                 administrative, regulatory, or statutory change is warranted.


                 To provide a basis for evaluating the progress of the Initiative in
Recommendation   cost-effectively reducing or preventing pollution, GAO recommends that the
                 Administrator, EPA, require the development of results-oriented
                 performance measures for assessing the extent to which the Council’s and
                 subcommittees’ actions have produced real, measurable environmental
                 improvements at less cost to industry and the public.




                 Page 41                                GAO/RCED-97-164 EPA’s Common Sense Initiatives
Appendix I

GAO’s Performance Measurement Model and
the Projects Reviewed

             GAO’s performance measurement model, shown in figure I.1, applies the
             concepts underlying the Government Performance and Results Act of
             1993, which focus on program outcomes—the results of a program activity
             compared to its intended purpose—rather than activity levels and tasks
             completed. GPRA also incorporates performance measurement as one of its
             most important features. The model provides the basis for assessing the
             extent to which a project is results-oriented and has a mechanism for
             measuring the degree to which an expected outcome has been achieved.

             In applying our model, we answered the following questions for each of
             the Initiative’s 15 projects (see pp. 43-45) that we reviewed:

             1. To which of the Initiative’s program element(s) has the project been
             linked and what is the program element’s stated purpose? (Program
             elements are discussed on p. 13.)

             2. What is (are) the project’s stated objective(s)?

             3. Is there a clear link between the project’s stated objective(s) and the
             program element’s stated purpose?

             4. What is the project’s design and status of completion?

             5. What product or service will result (e.g., guidance manual, modified
             reporting process, streamlined permitting)?

             6. What is the project’s expected outcome (i.e., result, effect, impact)—as
             distinguished from its product or service?

             7. Does the project’s design include a results-oriented mechanism to
             measure whether and to what extent the project has achieved its expected
             outcome(s)?

             8. Is there a clear link between outcome(s) of the project and the program
             element(s)? Does the project’s design provide a mechanism for assessing
             the degree of such linkage or agreement?

             9. Will the project result in fundamental change (e.g., regulatory
             reinvention, legislative reform) or a redesigned process (e.g., streamlined
             permitting or reporting)?




             Page 42                            GAO/RCED-97-164 EPA’s Common Sense Initiatives
                                       Appendix I
                                       GAO’s Performance Measurement Model
                                       and the Projects Reviewed




                                       10.Does the project measure whether the outcome reduced or prevented
                                       pollution at less cost to industry and the taxpayer?

                                       11.Is the project linked to any other Initiative or non-Initiative project(s)?


Figure I.1: GAO’s Performance Measurement Model




                                                                                                   Outcome
                                                                                                    Results
                                                                                                    Effect
                                                                                                    Impact
                                                             Product
          Program                     Process
          purpose                      steps
                                                                    Service

                                                                                                   Outcome
                                                                                                    Customer
                                Recordkeeping                                                       satisfaction
              Regulation                               Permitting               Other
                                 and reporting


                        Pollution        Compliance and      Environmental
                       prevention         enforcement         technology


                                          Sector project




                                       We applied our model to the following 15 projects, which are grouped
                                       according to the subcommittee sponsoring each project:


Automobile Manufacturing               Alternative Sector Regulatory System/Community Technical Assistance
Subcommittee                           Project to develop alternative regulatory system principles that will
                                       provide the basis for plant-specific implementation and to encourage
                                       better understanding of and greater participation in environmental quality




                                       Page 43                               GAO/RCED-97-164 EPA’s Common Sense Initiatives
                            Appendix I
                            GAO’s Performance Measurement Model
                            and the Projects Reviewed




                            and economic development issues by the potentially affected
                            communities.

                            Regulatory Project Team Area/VOCs Mini-Group Project to determine
                            whether a more flexible and easier-to-implement surface area approach
                            for calculating and establishing emission limits for the volatile organic
                            compounds (VOC) associated with specific processes (e.g., painting
                            operations) used in the automobile manufacturing industry could replace
                            or enhance the existing method for calculating and establishing those
                            limits.


Computers and Electronics   Consolidated Emergency Response Reporting to streamline the emergency
Subcommittee                response planning process by consolidating various emergency response
                            plan requirements so that only one document will meet the needs of all
                            stakeholder groups (including emergency responders, the community,
                            facility workers, and regulatory agencies).

                            Consolidated Uniform Report for the Environment to design and test a
                            new comprehensive environmental report and to provide streamlined and
                            consolidated reporting while providing for electronic reporting and
                            increased public access to information.


Iron and Steel              Brownfields Demonstration Project to develop, pilot test, and document a
Subcommittee                process for redeveloping iron and steel brownfields sites.

                            Multimedia Permitting Project to develop a multimedia permitting process
                            covering air, water, and waste for a steel mini-mill.

                            Spent Pickle Liquor Workshop to convene a 1-day workshop at which
                            panelists and technical consultants will inform all stakeholders about the
                            problems (technical and regulatory) associated with spent pickle liquor.


Metal Finishing             Promoting Improved Performance (Metal Finishing 2000) Flexible Track
Subcommittee                Project to promote improved performance through an alternative
                            regulatory program for top-tier firms (i.e., tier 1 and tier 2 industry
                            performance leaders) that might receive flexibility and incentives to seek
                            ambitious environmental goals.




                            Page 44                               GAO/RCED-97-164 EPA’s Common Sense Initiatives
                        Appendix I
                        GAO’s Performance Measurement Model
                        and the Projects Reviewed




                        Metal Finishing Guidance Manual Project to create a shop floor “how to”
                        tool for facilities to maintain compliance and pursue pollution prevention.

                        Regulatory Information Inventory Team Evaluation Project to examine
                        federal, state, and local reporting requirements for metal finishers across
                        all environmental media and to explore ways to reduce paperwork burden,
                        improve public access to data, and promote better environmental
                        performance.

                        Strategic Goals Initiative Project to develop a set of national performance
                        goals along with an implementation plan for the metal finishing industry
                        and the government and public entities that interact with the industry.


Petroleum Refining      One-Stop Reporting and Public Access Project to examine federal and
Subcommittee            state air emission reporting requirements for petroleum refiners to identify
                        and recommend modifications to duplicative and/or obsolete requirements
                        and improve community access to and understanding of reported data.

                        Equipment Leaks Project to reduce the loss of process fluids/vapors
                        through equipment leaks more efficiently.


Printing Subcommittee   New York Education Project to achieve fundamental change within the
                        printing sector to incorporate the philosophy of pollution prevention into
                        everyday work practices through education and outreach.

                        Multimedia Flexible Permitting Pollution Prevention Project to develop a
                        permit system applicable to printers that (1) allows for operational
                        flexibility; pollution reduction across all media; and improved protection
                        of the environment, workplace, and community and (2) is simpler to
                        implement and manage for regulatory agencies and businesses alike to
                        ensure compliance and access to the public.




                        Page 45                               GAO/RCED-97-164 EPA’s Common Sense Initiatives
Appendix II

Comments From the Environmental
Protection Agency




              Page 46    GAO/RCED-97-164 EPA’s Common Sense Initiatives
Appendix II
Comments From the Environmental
Protection Agency




Page 47                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
Appendix II
Comments From the Environmental
Protection Agency




Page 48                           GAO/RCED-97-164 EPA’s Common Sense Initiatives
Appendix III

Major Contributors to This Report


                        Susan D. Kladiva
Resources,              J. Kenneth McDowell
Community, and          Raymond M. Ridgeway
Economic                William H. Roach, Jr.
                        Elizabeth R. Eisenstadt
Development             Martin H. Emmrich
Division, Washington,
D.C.
                        Karen K. Keegan
Office of General
Counsel
                        Curtis W. Copeland
General Government      Edward G. Joseph
Division, Washington,
D.C.




(160361)                Page 49                   GAO/RCED-97-164 EPA’s Common Sense Initiatives
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested