oversight

Results Act: Observations on Federal Science Agencies

Published by the Government Accountability Office on 1997-07-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                    United States General Accounting Office

GAO                 Testimony
                    Before the Committee on Science
                    House of Representatives




For Release
on Delivery
Expected at
                    RESULTS ACT
10:00 a.m. EDT
Wednesday
July 30, 1997
                    Observations on Federal
                    Science Agencies
                    Statement by Susan Kladiva
                    Associate Director, Energy, Resources, and Science Issues
                    Resources, Community, and Economic
                    Development Division




GAO/T-RCED-97-220
Mr. Chairman and Members of the Committee:

We are pleased to be here today to discuss the implementation of the
Government Performance and Results Act, often referred to as the Results
Act or GPRA, in federal science agencies. A focus on results, as intended by
the Results Act, is aimed at improving the efficiency and effectiveness of
the federal government. In the science and technology area, where more
than 20 agencies spent $60 billion in fiscal year 1996, the potential for
unnecessary overlap is particularly high and close coordination is
essential. While some shared responsibilities are fine, uncoordinated
program efforts can waste scarce funds, confuse and frustrate program
customers, and limit the overall effectiveness of the federal effort.
Coordination among federal programs contributing to the same or similar
results can ensure that goals are consistent and, as appropriate, program
efforts are mutually reinforcing.

As agreed with the Committee, my statement will focus on six agencies’
fulfillment of the requirements of the Results Act and the interagency
coordination of crosscutting science programs, activities, or functions that
are similar or complementary to those of other federal agencies. Regarding
the requirements of the Results Act, we focused on two
elements—(1) agencywide goals and objectives and (2) past and future
program evaluations. The six agencies I will discuss include the
Departments of Commerce, Energy, and Transportation, as well as the
Environmental Protection Agency (EPA), the National Science Foundation
(NSF), and the National Aeronautics and Space Administration (NASA). My
statement is based on our review of the agencies’ draft strategic plans,
discussions with agency officials, and our general knowledge of the
science agencies.

In summary, Mr. Chairman:

Overall, the draft strategic plans show progress toward meeting the
purposes of the Act; however, only one of the six agencies’ plans contains
all six of the Act’s critical elements. In addition, some of the completed
elements were insufficient. For example, the goals and objectives were
frequently results-oriented, but it was unclear in all of the plans how some
of the goals would be measured. Developing effective performance
measures for these program goals will be a major challenge for science
agencies. Furthermore, five of the six plans did not include information on
past and future program evaluations and the one inclusion could be
improved. Because the draft plans do not contain all six elements, the



Page 1                                                     GAO/T-RCED-97-220
             Congress is missing critical pieces of information for its consultations with
             the agencies.

             Under Office of Management and Budget (OMB) guidance, the agencies’
             final submission should include a summary of consultation efforts,
             including crosscutting activities. Currently, the agencies’ draft plans
             generally do not address how crosscutting activities correspond with
             those of other agencies. In addition, the plans generally do not address
             whether such shared responsibilities were coordinated in the development
             of the draft plans. However, some of the agencies’ missions and goals do
             involve or overlap those of other agencies. Despite the fact that the draft
             plans do not reflect coordination activities, several agencies have initiated
             efforts to coordinate crosscutting research programs governmentwide. But
             coordination has occurred primarily at the program level rather than at the
             senior management level which is necessary to ensure consistency of
             program objectives among agencies. In our opinion, recognition of such
             coordination activities as part of the final submission will be useful to the
             Congress in making funding decisions that involve similar or
             complementary science programs.

             Now, I will briefly discuss these issues within the context of each of the
             six agencies. More details are provided in our reports on each of these
             agencies’ implementation of the Results Act, as listed in appendix I to my
             statement.


             The Results Act is intended to improve the efficiency and effectiveness of
Background   federal programs by establishing a process to set goals for program
             performance and to measure results. As a starting point, the Act requires
             virtually every executive agency to develop a strategic plan covering at
             least 5 years. It also requires that an agency’s strategic plan contain the
             following six critical elements: a mission statement; agencywide goals and
             objectives; the strategies and resources needed to achieve the goals and
             objectives; the relationship between the long-term goals and objectives
             and the annual performance goals; the key external factors that could
             affect the achievement of goals; and a description of how program
             evaluations were and will be used to establish or revise strategic goals.

             When developing a strategic plan, the Results Act requires that the agency
             shall consult with the Congress and shall solicit and consider the views
             and suggestions of those entities potentially affected by or interested in
             such a plan. Furthermore, guidance from OMB states that when general



             Page 2                                                     GAO/T-RCED-97-220
                   goals and objectives have crosscutting functions, programs, or activities,
                   agencies may have a shared responsibility for defining and achieving them.
                   Thus, agencies should ensure that appropriate and timely consultation
                   occurs with other agencies during the development of strategic plans with
                   crosscutting goals and objectives. Though the Act does not include a
                   requirement that the draft strategic plans should contain a description of
                   how the activities of an agency relate and will be coordinated with the
                   activities of other agencies with similar programs, OMB guidance does
                   provide that the letter transmitting the strategic plan include a summary of
                   the general scope and nature of the consultation and the types of entities
                   consulted. In addition, OMB guidance on preparation of annual
                   performance plans beginning in fiscal year 1999, states that agencies
                   should indicate those goals and indicators that are being mutually
                   undertaken in support of programs or activities of an interagency,
                   crosscutting nature.

                   It is important to recognize that the final strategic plans are not due to the
                   Congress and OMB until September 1997. Furthermore, the Results Act
                   anticipated that it may take several planning cycles to perfect the process
                   and that the final plans would be continually refined as future planning
                   cycles occur. Thus, our comments reflect a snapshot of the plans when
                   they were submitted to the Congress, between May and July, and are
                   intended to provide insights that may help this Committee and the
                   agencies work together to successfully implement the Results Act. We also
                   recognize that the agencies are continuing to revise their strategic plans
                   with input from OMB, congressional staff, and other stakeholders.


                   While NSF addresses five of the six required elements of the Results Act, at
National Science   least four of them need further development, and the sixth element—key
Foundation         external factors—is not included in its draft strategic plan.1 Although NSF
                   defines goals and objectives in the draft plan, many of these goals are not
                   expressed in a measurable form, making it unclear whether the
                   Foundation and the Congress will be able to assess whether the goals are
                   achieved. NSF’s draft plan also does not discuss how the agency used
                   specific program evaluations to develop its strategic goals or the other
                   components of the plan. Further details are needed on a schedule for
                   future evaluations, the scope of and methodology for future evaluations,
                   and how the findings could be useful in assessing NSF’s goals and
                   performance plans.


                   1
                    We reviewed the draft strategic plan dated June 9, 1997.



                   Page 3                                                      GAO/T-RCED-97-220
                       The draft plan acknowledges the crosscutting nature of NSF’s work but
                       does not show evidence of interagency coordination. It emphasizes the
                       importance of NSF’s many partners in the research and education
                       enterprise but does not identify these partners or provide sufficient
                       information to determine the extent to which NSF and its partners’
                       functions are duplicative or overlapping. However, we identified
                       similarities of mission in the draft plans developed by NSF, Energy, and
                       Commerce. For example, the Foundation’s mission includes promoting the
                       progress of science, and one of its overarching goals is to enable the
                       United States to uphold a position of world leadership in all aspects of
                       science, mathematics, and engineering. Similarly, Energy’s science mission
                       is to ensure that the United States retains its leadership in science and
                       technology. Also comparable is Commerce’s mission, which includes
                       keeping America competitive with cutting-edge science and technology.

                       Although not identified in NSF’s draft plan, agency officials cite efforts to
                       coordinate the crosscutting areas. For example, NSF has shared its draft
                       plan on the Internet. Furthermore, according to NSF’s Assistant to the
                       Director for Science Policy and Planning, the Foundation participates in a
                       number of groups such as the National Science and Technology Council,
                       the Committee on Fundamental Science, and the Research Roundtable.
                       While one purpose of these groups is to coordinate, NSF’s draft plan does
                       not mention these groups or the frequency of their discussions. We have
                       found that although these agency forums have provided an important
                       opportunity for agencies to work together to address common concerns in
                       goal setting and performance measurement, they have not generally
                       attempted to coordinate crosscutting program efforts.


                       Energy’s draft plan fully addresses only two of the six required
Department of Energy   elements—the mission statement and the goals and objectives—and
                       partially addresses a third—the strategies element.2 We found that the
                       goals and objectives cover the agency’s major functions and operations,
                       and that the goals are generally results-oriented. However, in reviewing
                       the multiple objectives for each goal, we identified several objectives that
                       were stated in ways that will make it difficult to measure whether they are
                       being achieved. Furthermore, Energy did not complete the element related
                       to the impact of program evaluations on the development of strategic
                       goals. The Department is finalizing these elements and expects to include
                       them in the final plan.


                       2
                        We reviewed the draft strategic plan dated June 16, 1997.



                       Page 4                                                       GAO/T-RCED-97-220
                Although Energy does not mention crosscutting programs or coordination
                efforts in its draft plan, the Department is sharing its draft plan with other
                federal agencies. Energy’s draft plan does not identify programs and
                activities that are crosscutting or similar to those of other federal agencies,
                primarily because the Department believes its functions are unique. On the
                basis of our work, however, we believe that Energy’s broad missions do
                involve or overlap those of other agencies. As previously discussed, we
                have identified the potentially overlapping missions of Energy, Commerce,
                and NSF. Another example of potential overlap is in science education.
                Energy’s draft strategic plan states that it will use its laboratories and the
                nation’s universities to contribute to the nation’s science and mathematics
                education. Similarly, NSF’s authorizing legislation directs it to initiate and
                support science and engineering education programs at all levels and in all
                fields of science and engineering.

                Environmental quality is another area of potential overlap. The science
                and technology portion of Energy’s draft plan contains a strategy to
                conduct high-quality research on environmental quality. Similarly, one of
                EPA’s goals is to develop and apply the best available science for
                addressing current and future environmental hazards as well as new
                approaches toward improving environmental protection.


                Commerce’s draft strategic plan contains four of the six critical elements,
Department of   but each of the four, including the goals and objectives element, has
Commerce        weaknesses.3 While there are linkages among themes, goals, objectives,
                and responsible components, the goals and objectives are not as
                results-oriented as they could be. For example, the goal to support
                restructuring export controls for the 21st century could be made more
                results-oriented by identifying the purpose of the restructuring. Moreover,
                the draft plan does not explicitly discuss the program evaluations used to
                establish general goals and objectives and has no schedule for future
                program evaluations.

                Commerce’s draft plan also does not identify crosscutting programs and
                activities or whether such shared responsibilities were coordinated in the
                development of the draft plan. The draft plan does describe, in very
                general terms, some of the existing partnerships between Commerce
                agencies and various public and private entities. However, as stated
                earlier, we identified mission overlap between NSF, Energy, and
                Commerce. Another example of potential overlap occurs in Commerce’s

                3
                 We reviewed the draft strategic plan dated June 1997.



                Page 5                                                       GAO/T-RCED-97-220
                       grant program to promote the use of advanced telecommunications in the
                       public and nonprofit sectors. A number of other federal agencies,
                       including NSF, also support telecommunications projects for similar
                       constituencies.


                       Of the six elements required by the Results Act, NASA includes four in its
National Aeronautics   draft strategic plan. The two that are not explained in enough detail are
and Space              the relationship between NASA’s long-term goals and annual performance
Administration         goals and a description of program evaluations.4 While the goals outlined
                       by NASA appear to meet the Results Act’s requirements, progress toward
                       some of the goals, such as the goal to acquire and encourage knowledge
                       and technologies that promote our quality of life, may prove difficult to
                       assess. As we recently reported, it is inherently difficult to measure the
                       performance of research and development programs because the results
                       are typically not apparent until many years later.5 The draft does not
                       explicitly discuss or demonstrate how program evaluations were used in
                       establishing or revising agency goals and objectives, nor does it provide a
                       schedule for future evaluations.

                       NASA’s draft plan also does not identify specific programs and activities
                       that are crosscutting or similar to those of other federal agencies, and the
                       plan does not address interagency coordination. Many of NASA’s objectives,
                       however, are shared with other agencies. For example, one of NASA’s
                       objectives related to long-term climate and ozone research involves NSF,
                       the Department of Energy, and Commerce’s National Oceanic and
                       Atmospheric Administration. Moreover, like NSF’s mission of advancing
                       scientific knowledge, one of NASA’s missions is to advance and
                       communicate scientific knowledge and understanding of the Earth, the
                       solar system, and the environment of space for research.

                       The draft plan does not address what interagency coordination occurred
                       to address the shared responsibilities. However, the draft plan does note
                       the importance of working with other agencies in achieving its objectives,
                       and NASA officials stated that coordination has occurred at the program
                       level. NASA officials stated that each strategic enterprise coordinated its
                       objectives with the relevant agencies at the program level. They noted that
                       NASA officials participate in many crosscutting groups, like the Research
                       Roundtable, where programmatic objectives are discussed. They also

                       4
                        We reviewed the draft strategic plan dated May 1997.
                       5
                        Measuring Performance: Strengths and Limitations of Research Indicators (GAO/RCED-97-91, Mar. 21,
                       1997).



                       Page 6                                                                      GAO/T-RCED-97-220
                    noted that for the last few years, the agency has shared its strategic plan
                    with other federal agencies.


                    EPA’s draft plan contains four of the six elements required by the Results
Environmental       Act, certain aspects of which could be improved.6 Although the goals and
Protection Agency   objectives—one of the completed elements—are generally results-oriented
                    and measurable, some do not clearly define the expected results, and it is
                    unclear how EPA would assess progress toward achieving them.
                    Furthermore, the large number of goals (10), objectives (45), and
                    subobjectives (200) may make it difficult for the Congress and others to
                    discern EPA’s priorities—what will be most important to the agency over
                    the next several years. The two missing elements are (1) the relationship
                    between EPA’s long-term goals and the annual performance goals and
                    (2) program evaluations used in developing the plan and a schedule for
                    future evaluations.

                    EPA’s draft plan does not discuss the agency’s programs and activities that
                    are crosscutting or similar to those of other federal agencies, but it does
                    address the need for coordination with its stakeholders, which include
                    federal entities. Our past work has found that EPA—as the central federal
                    agency responsible for safeguarding the environment—carries out a
                    number of mission-related activities that are crosscutting or similar to
                    those of other federal agencies. For example, as we discussed previously,
                    EPA and Energy both conduct research on environmental quality. In
                    addition, EPA shares responsibilities with other agencies for collecting and
                    managing the data needed to perform environmental assessments. Thus,
                    EPA must coordinate data for health assessments with a number of
                    different agencies, including NSF.

                    EPA’s draft plan makes broad reference to the need for coordination with
                    federal agencies as well as other stakeholders to accomplish its mission.
                    As such, EPA is currently taking steps to coordinate its plan with other
                    agencies to address crosscutting programs and activities. To further refine
                    its plan and determine areas of potential overlap between EPA and federal
                    agencies with related responsibilities, EPA identified 16 federal agencies
                    with crosscutting or similar functions and sent each of them a draft outline
                    for the strategic plan in May 1997, and the full draft in early July 1997, for
                    their review and comment. In addition, EPA is reviewing these agencies’
                    draft plans to identify areas in which duplication of activities exists and
                    further coordination is warranted. According to EPA officials, the agency is

                    6
                     We reviewed the draft strategic plan dated July 1997.



                    Page 7                                                      GAO/T-RCED-97-220
                 including in its coordination all of the other five science agencies
                 discussed in this testimony.


                 Although Transportation’s draft plan includes all six critical elements, we
Department of    believe that only three of the six meet the Act’s requirements, including
Transportation   the mission statement, long-term goals, and a description of program
                 evaluations.7 The five long-term goals encompass the Department’s major
                 functions and operations and are within the Department’s span of
                 influence. However, they could be improved by stating all goals in a
                 measurable form to allow for future assessments of their achievement.
                 Furthermore, they are stated in general terms whereas sufficiently precise
                 goals can better direct and guide agency staff toward fulfilling the agency’s
                 mission. The program evaluation information is insufficient to determine
                 the scope and methodology or the key issues to be addressed. Without this
                 information, it is difficult to determine how or if scheduled evaluations
                 relate to Transportation’s goals.

                 Transportation’s draft plan does not identify the crosscutting activities nor
                 show evidence of interagency coordination. Supporting documents that
                 Transportation used to prepare its draft plan indicate that the Department
                 considered a number of crosscutting issues but did not include the
                 information in the plan. Again, however, one of the science-related
                 strategies outlined in Transportation’s draft strategic plan focuses on
                 research and technology to foster economic growth and enable research
                 and education. Transportation’s draft plan recognizes that there are other
                 federal stakeholders and provides for establishing partnerships. However,
                 it is silent on whether the Department coordinated with other federal
                 agencies that have programs and activities that are crosscutting or similar
                 to Transportation’s. According to a senior Transportation official,
                 coordination is an ongoing activity and no specific coordination was done
                 for the purposes of preparing the draft plan.


                 In conclusion, the Results Act process provides an opportunity for
                 agencies to begin improving the coordination and execution of
                 crosscutting science issues across the federal government. The lack of
                 coordinated performance goals that are results-oriented may be
                 problematic as the Congress begins to evaluate programs and activities
                 that are crosscutting among the science agencies. In addition, fully
                 resolving the problem of potential overlap and duplication will take time

                 7
                  We reviewed the draft strategic plan provided to congressional committees on July 2, 1997.



                 Page 8                                                                         GAO/T-RCED-97-220
and require sustained leadership by this Committee, OMB, and agency
senior management to ensure that science agencies coordinate their
efforts and develop coordinated goals among crosscutting programs and
activities. Congressional consultations on agencies’ strategic plans provide
an ongoing opportunity for the Congress and the executive branch to work
together to minimize the extent and potential consequences of overlap and
fragmentation in federal program efforts. Special attention devoted to
crosscutting issues and coordination activities in the strategic plans will
improve this consultation process. Congressional oversight, such as is
occurring here today, is also key.

This concludes my statement. I would be pleased to respond to any
questions you or the Members of the Committee may have.




Page 9                                                     GAO/T-RCED-97-220
Page 10   GAO/T-RCED-97-220
Page 11   GAO/T-RCED-97-220
Related GAO Reports


              Results Act: Observations on the National Science Foundation’s Draft
              Strategic Plan (GAO/RCED-97-203R, July 11, 1997).

              Results Act: Observations on the Department of Energy’s Draft Strategic
              Plan (GAO/RCED-97-199R, July 11, 1997).

              The Results Act: Observations on Commerce’s June 1997 Draft Strategic
              Plan (GAO/GGD-97-152R, July 14, 1997).

              Results Act: Observations on NASA’s May 1997 Draft Strategic Plan
              (GAO/NSIAD-97-205R, July 22, 1997).

              Results Act: Observations on Environmental Protection Agency’s Draft
              Strategic Plan (GAO/RCED-97-209R, July 30, 1997).

              Results Act: Observations on the Department of Transportation’s Draft
              Strategic Plan (GAO/RCED-97-208R, July 30, 1997).




(141090)      Page 12                                                  GAO/T-RCED-97-220
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested