oversight

Observations on the National Science Foundation's Fiscal Year 2000 Performance Plan

Published by the Government Accountability Office on 1999-07-20.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States General Accounting Office                                                           Resources, Community, and
Washington, D.C. 20548                                                                        Economic Development Division




                 B-282729

                 July 20, 1999

                 The Honorable Dick Armey
                 Majority Leader
                 House of Representatives

                 The Honorable Dan Burton
                 Chairman, Committee on Government Reform
                 House of Representatives

                 The Honorable Fred Thompson
                 Chairman, Committee on Governmental Affairs
                 United States Senate

                 Subject: Observations on the National Science Foundation’s Fiscal Year 2000 Performance
                 Plan

                 As you requested, we have reviewed and evaluated the fiscal year 2000 performance plans for
                 the 24 Chief Financial Officers (CFO) Act agencies that were submitted to Congress as
                 required by the Government Performance and Results Act of 1993 (Results Act). Enclosure I
                 to this letter provides our observations on the fiscal year 2000 performance plan for the
                 National Science Foundation (NSF). Enclosure II lists management challenges NSF’s Office
                 of the Inspector General (OIG) identified that face the agency and the applicable goals and
                 measures in the fiscal year 2000 annual performance plan.

                 Our objectives were to (1) assess the usefulness of the agency’s plan for decisionmaking and
                 (2) identify the degree of improvement the agency’s fiscal year 2000 performance plan
                 represents over the fiscal year 1999 plan. Our observations were generally based on the
                 requirements of the Results Act, guidance to agencies from the Office of Management and
                 Budget (OMB) for developing the plan (OMB Circular A-11, Part 2), our knowledge of NSF’s
                 operations and programs, and our observations on NSF’s fiscal year 1999 performance plan.
                 Our summary report on the CFO Act agencies’ fiscal year 2000 plans contains a complete
                                                                        1
                 discussion of our objectives, scope, and methodology.


                 1
                  Managing for Results: Opportunities for Continued Improvements in Agencies’ Performance Plans (GAO/GGD/AIMD-99-215,
                 July 20, 1999).




                 Page 1                                                 GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
B-282729


As agreed, unless you announce the contents of this letter earlier, we plan no further
distribution until 30 days from the date of the letter. The major contributors to this report
were Robin M. Nazzaro and Diane B. Raynes. Please call me on (202) 512-3841 if you or your
staff have any questions.




Victor S. Rezendes
Director, Energy,
  Resources, and Science

Enclosures – 2




Page 2                                      GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I

Observations on the National Science
Foundation's Performance Plan for
Fiscal Year 2000
The National Science Foundation’s (NSF) fiscal year 2000 annual performance plan will be of
general usefulness to decisionmakers. The plan provides a general picture of intended
performance across the agency, a general discussion of the strategies and resources the
agency will use to achieve its goals, and limited confidence that agency’s performance
information will be credible. While the plan identifies crosscutting efforts with other
agencies, it does not provide clear information on the linkages between the NSF budget and
its performance goals, which will be key for congressional reviewers. NSF provides a matrix
documenting the relative extent to which NSF functions, such as “research project support”
and “education and training,” support its goals such as the “connections between discoveries
and their use in service to society.” But there is no direct linkage between specific budget
activities such as “U.S. Polar Research Program” or “graduate education” and NSF’s
performance goals. Figure 1 highlights the plan’s major strengths and key weaknesses to be
noted as NSF seeks to make additional improvements to its plan.

Figure 1: Major Strengths and Key Weaknesses of NSF’s Fiscal Year 2000 Annual
Performance Plan

Major Strengths
•Uses an alternative format to describe type and level of performance
• Provides additional outcome oriented goals
• Links strategies to specific program goals and describes how strategies contribute to the
achievement of those goals
• Identifies crosscutting efforts with other related federal programs

Key Weaknesses
• Provides limited discussion of capital, human and financial resources
• Lacks clear linkages between the budget and performance goals
• Provides limited confidence in the validation and verification of data

The fiscal year 2000 performance plan indicates moderate progress in addressing the
weaknesses that we identified in our assessment of the fiscal year 1999 performance plan. In
reviewing the fiscal year 1999 plan, we observed that there was insufficient detail on
crosscutting programs, external factors, strategies and resources needed to achieve goals,
and data verification and validation. Among the improvements in the fiscal year 2000 plan are
additional information on crosscutting efforts and external factors. Regarding crosscutting
efforts, NSF describes both formal and informal agreements with other agencies. For
example, under its goal of “discoveries at and across the frontier of science and engineering,”
NSF supports research activities with the Department of Energy at the large hadron collider
in Switzerland and with the National Aeronautics and Space Administration at its space-based
and ground-based astronomy facilities. NSF also improved its plan by describing external
factors that could affect performance. For example, NSF describes the necessary




Page 3                                      GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




commitment on the part of school districts, schools, and their faculty to modifying their
approaches to education in order to enhance achievement for the NSF performance goal of
“improved achievement in mathematics and science skills needed by all Americans.”
Furthermore, NSF officials told us that if they believe work funded through a grant cannot be
reliably completed, they may stop funding for the award. While this may not improve
performance, it may mitigate the continued use of funds for unproductive activities.
Improvements that still need to be made to the performance plan are more detailed
discussions of the resources needed to achieve goals and further elaboration on the
procedures to assess the reliability and validity of data used to assess goal achievement.

The Agency’s Performance Plan Provides a General Picture of
Intended Performance Across the Agency
While NSF’s performance plan provides a general picture of intended performance across the
agency, there are still inconsistencies in the information supporting each performance goal,
and weaknesses that we identified in the fiscal year 1999 plan remain. The plan does a
reasonable job of using descriptive statements to express its annual performance goals for
scientific research and education and of using primarily quantifiable performance goals for
management and investment process activities. Also, the plan effectively links performance
goals with the agency’s mission and strategic goals.

For its scientific research and education activities, NSF established annual performance goals
in the form of statements that describe “successful” and “minimally effective” performance,
an alternative format allowed by the Results Act and the Office of Management and Budget
(OMB). For example, NSF believes its performance will be rated successful in meeting its
strategic goal of promoting connections between discoveries and their use in service to
society if the results of NSF awards are rapidly and readily available and, as appropriate, feed
into education, policy development, or work of other federal agencies or the private sector.
But, NSF’s performance will be rated only minimally effective if the results of its grant
awards show only the potential for use in service to society. The descriptive statements
developed by NSF reasonably define the type and level of annual performance that the
agency expects for these activities.

NSF’s use of the alternative format performance goals for outcomes in research and
education allows for expert judgment, considering both quantitative and qualitative
information on performance. There is strong consensus among science agencies that expert
review is the most rigorous and effective tool for evaluating basic and applied research and
that the practical outcomes of basic research cannot be captured by quantitative measures
alone.




Page 4                                              GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




To evaluate the results under this alternative format, NSF will use expert peer review. NSF is
relying on external experts who constitute Committees of Visitors who make qualitative
judgments of many aspects of the agency’s performance and who, starting in fiscal year 2000,
will include the Results Act performance goals in this process. This is the first time that
experts have been asked to pass judgment on the results of NSF’s research investments. NSF
has Guidelines for Committees of Visitors that describe the processes in place for collecting
results information used to assess NSF’s performance. The guidelines provide information on
how phrases such as “important discoveries” and “steady stream of outputs of good scientific
quality” are to be interpreted by reviewers and decisionmakers. They also identify procedures
and processes in place to ensure reliable and accurate qualitative data for assessment. In
essence, the experts provide the standards and must justify their assessment of NSF’s
performance with specific examples.

Decisionmakers within NSF, OMB, and the Congress who will use performance information
may need to understand the basis for the reviewers’ evaluations. It may therefore become
necessary to share some of the written justifications supporting programs’ ratings. NSF
recognizes that the elements of the external expert’s quality determination will vary in
character from year to year and from program to program. This variance may lead to
difficulties in evaluating results over time. Nonetheless, until the NSF guidelines for
evaluating performance results are implemented, it will be difficult to assess whether they are
providing enough guidance to aid reviewers.

The fiscal year 2000 performance plan corrects some of the weaknesses that we identified in
our assessment of the fiscal year 1999 performance plan by providing a clear picture of
intended performance across the agency and making some specific commitments to address
those weaknesses. In reviewing the fiscal year 1999 plan, we observed that definitions of
expected performance were not provided with the alternative format. In addition,
performance was defined by using output-oriented rather than outcome-oriented goals for its
management and other activities. The improvement shown in the fiscal year 2000 plan
includes the addition of outcome-oriented goals to the management and investment activity
sections, as well as details on crosscutting efforts with other federal agencies.

Other opportunities for strengthening NSF’s performance plan include attention to
intermediate goals and to the management challenges identified by the NSF Office of the
Inspector General (OIG). (See enclosure II for a list of the management challenges.) Given
the long-term nature of many of NSF’s performance goals, multiyear goals may be useful for
conveying what a program is expected to achieve for that year and in the long-term. In
addition, performance goals or strategies to resolve mission-critical management problems,
such as the development and implementation of an effective system for cost sharing, could
enhance the plan.




Page 5                                              GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




The Agency’s Performance Plan Provides a General Discussion
of the Strategies and Resources the Agency Will Use to Achieve
Its Goals
The plan provides a general discussion of strategies and resources the agency will use to
achieve performance goals and includes information, for most of the goals, on external
factors and crosscutting areas with other agencies. The plan discusses strategies for NSF’s
scientific research and education goals as well as some description of strategies the agency
will use to achieve its goals for management and investment process activities. The plan
discusses external factors and, in some instances, the actions to mitigate their impact on
performance. However, the plan only partially discusses resources that NSF will use to
achieve performance goals, primarily discussing how budgetary resources relate to the
achievement of performance goals.

Connecting Resources to Strategies
The plan does not clearly show how budgetary resources relate to NSF’s performance goals.
After arraying the funding requested for NSF’s research program activities, education and
training, and administration and management, the plan presents a second analysis that shows
which of the functional areas support one or more of the agency’s five major scientific
research and education goals. The plan provides financial information by discussing the funds
that NSF requested in its fiscal year 2000 budget request to achieve scientific research and
education goals. However, this second analysis does not indicate what level of funding is
associated with each of these goals. Specifically, the plan does not show how program
activities and related funding will be allocated to NSF’s performance goals.

Documenting how the funding from NSF’s activities will be allocated to its performance goals
remains challenging for NSF. For example, NSF’s research grants in support of graduate
students produce outcomes for more than one performance goal. According to NSF,
programs associated with these grants produce results for each of the four performance
goals: (1) discoveries at and across the frontier of science; (2) connections between
discoveries and their use in service to society; (3) diverse, globally oriented science and
engineering workforce; and (4) improved math and science skills needed by all Americans.
Therefore, associating the funds for one project to only one goal underestimates the funds in
support of other goals. Because of this complexity, it may be helpful for NSF to document the
various connections between program activities and performance goals and to consider some
variations in its program activity structure. In our 1999 report issued on the subject of agency
performance plans, we reported on other federal agencies’ efforts to align their budgets with




Page 6                                              GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




                              1
their performance goals. For example, the Department of Health and Human Services’ Indian
Health Service included an effective crosswalk that aligns its budget activities with its
performance in aggregated program activities. The Environmental Protection Agency
proposed changes to its program activity structure across all of its accounts in which each
program activity represents one of the agency’s strategic goals. Using this realigned program
activity structure in its annual plan, the agency showed the funding it requested in order to
achieve each strategic objective and the supporting annual performance goals.

Decisionmakers recognize that funding from one budget activity may support more than one
performance goal and the Results Act gives agencies the flexibility to aggregate, disaggregate,
or consolidate the budget’s program activities so that they align with the performance goals.
While the level of detail in the plan depends on the needs of congressional committees, better
performance plans not only describe the resources needed but also provide a rationale for
how the resources will contribute to accomplishing the expected level of performance.

NSF officials told us that the interdependence of programs and goals is the principal
challenge they face in linking budget and performance goals. NSF pointed out that its
investments generally work toward more than one of the performance goals simultaneously.
NSF’s discussions with OMB about modifying its budget lines to align with performance goals
are ongoing. Nonetheless, we believe further effort is needed to meet the needs of
congressional reviewers who will be evaluating budgetary resources based on achievement of
performance goals.

Connecting Strategies to Results
NSF’s performance plan presents clear and reasonable strategies that NSF will use to achieve
its fiscal year 2000 performance goals. Specifically, the plan describes general strategies that
NSF intends to use to achieve its five primary goals for scientific research and education and
then discusses the strategies that the agency will use to achieve each of its performance
goals, most of its performance goals for the NSF investment process, and each of its
performance goals for management. For example, NSF will use a competitive merit-based
review process with peer evaluations to identify the most promising ideas from the strongest
researchers and educators and will integrate research and education activities to strengthen
both activities. Overall, the strategies describe reasonable approaches for achieving NSF’s
fiscal year 2000 goals for scientific research and education. For its management and
investment process activities, NSF established 19 performance goals. These include such
elements as completing the processing of proposals electronically, constructing research


1
 Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers (GAO/GGD/AIMD-99-69,
Feb. 26, 1999).




Page 7                                                GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




facilities on a timely basis, addressing integration of research and education, and maintaining
the participation of students and staff from underrepresented groups in science and
engineering.

Information on means and strategies for achieving these goals, crosscutting activities with
other agencies, external factors that could affect performance, and capital assets and
mission-critical management systems is provided for some but not all of the management and
investment process performance goals. Moreover, these elements are often incomplete. For
example, in the performance area of facilities oversight, NSF’s performance goals include
keeping construction and upgrades within the annual expenditure plan by not exceeding 110
percent of estimates and keeping operating time lost due to unscheduled downtime to less
than 10 percent of the total scheduled operating time. External factors such as extremely
adverse weather or the failure of partners to act as planned are identified as issues that could
have a significant effect on NSF’s construction projects and operating plans. However, NSF
did not describe strategies to mitigate the effects of external factors on the accomplishment
of performance goals. The failure to complete construction of facilities could affect the
volume and quality of research or affect NSF’s costs for supporting research. The plan does
not indicate the efforts that the agency will make to monitor the extent to which unforeseen
circumstances could affect the agency’s fiscal year 2000 performance.

NSF told us that to the extent that there are major changes that might have an impact on NSF
performance, the agency would work to smooth the transition to a new “steady state,” while
optimizing performance under the circumstances. Furthermore, if the agency believes the
work cannot be reliably completed, it may cancel the funding for an award, which may not
improve performance but at least may mitigate the continued use of funds for unproductive
activities.

Comparison to Fiscal Year 1999 Performance Plan
The fiscal year 2000 performance plan recognizes the weaknesses that we identified in our
assessment of the fiscal year 1999 performance plan in terms of providing a complete
discussion of the strategies and resources the agency will use to achieve performance goals
and makes specific commitments or shows actual attempts to address those weaknesses. In
reviewing the fiscal year 1999 plan, we observed that the plan did not provide a linkage
between performance goals and the agency’s budget; clearly describe the strategies the
agency will use to achieve its goals for management and other activities; address external
factors that are likely to affect its performance; or discuss resources that it will use to achieve
performance goals. Among the improvements in the fiscal year 2000 plan is the inclusion of
some information on the means and strategies for achieving goals, crosscutting activities with
other agencies, external factors that could affect performance, and information resources
addressing management and other activities. Although these elements are often incomplete,
the additional detail demonstrates an improvement over last year’s plan. For example, while




Page 8                                              GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




the plan provides some additional discussion of the means and strategies supporting NSF’s
investment in new and emerging technologies, such as electronic proposal processing, the
plan lacks specific information on the capital, human, or information resources that will be
needed.

The Agency’s Performance Plan Provides Limited Confidence
That Agency Performance Information Will Be Credible
The plan provides limited confidence that the agency’s performance information will be
credible. NSF’s performance plan provides separate verification and validation plans for its
research and education goals and its management and investment activity goals. These plans
provide an overview of the information sources and the primary information systems that
NSF plans to use to assess achievement of its fiscal year 2000 performance goals. The plan
identifies four information systems that will store, process, analyze, and report performance
measurement data, but does not describe any standards or procedures that it will use to
assess the reliability of the systems. Moreover, significant limitations are not described, and
details of assessment or correction plans are not provided, making it impossible to assess
these data limitations.

NSF officials told us that they have procedures in place and are developing processes to
assess the reliability and validity of data used to assess goal achievement in partnership with
the OIG. For example, the OIG plans to selectively test systems presently in use and under
development that produce data on the performance goals for their accuracy and reliability
and to determine that they work correctly. The systems used for performance measurement
having the greatest risk for inaccuracy will be tested for reliability of quantitative data
supporting the performance measures. In addition, the adequacy of internal controls over
automated data information systems will be reviewed by the OIG. It is too soon to know
whether the data entering the information systems are valid and reliable because NSF did not
present information regarding reliability and validity.

NSF officials told us that many of the specifics on data systems are provided in the agency’s
internal documents. Explaining that they did not want to overwhelm the performance plan
with detail on how the information will be collected and assessed for its reliability, officials
told us that the information is available in other sources. Nonetheless, the plan’s lack of
specificity does not provide any reasonable assurance that the performance information will
be of sufficient reliability and validity to credibly assess NSF’s fiscal year 2000 goal
achievement. In summary, NSF’s plan fails to describe specific and credible procedures to (1)
verify and validate performance information and associated information systems required for
assessment of fiscal year 2000 goal achievement; (2) determine the soundness of the review,
analysis, and synthesis process used to assess performance; and (3) identify significant data




Page 9                                              GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




and/or information system limitations, their implications for assessment of performance goal
achievement, or any actions designed to improve recognized problems.

The fiscal year 2000 performance plan does not appear to recognize the weaknesses that we
identified in our assessment of the fiscal year 1999 performance plan in terms of providing
sufficient confidence that the agency’s performance information will be credible. In reviewing
the fiscal year 1999 plan, we observed that NSF’s performance plan fell short of identifying
significant limitations with performance data and the potential implication of the limitations
for assessing the achievement of performance goals. These concerns remain with the fiscal
year 2000 performance plan.

Other Observations on the National Science Foundation’s
Implementation of Performance-Based Management
For some crosscutting activities, a coordinating entity oversees the contributions of
participating agencies, independent of the Results Act’s implementation. For example, the
National Science and Technology Council establishes national goals for federal science and
technology investments for the U.S. Global Change Research Program, among others. NSF is
one of several agencies that support or conduct research and education activities in support
of this program. One desirable result of this performance management is increased teamwork
among agencies as well as improved communication between research agencies and
oversight entities.

While NSF was effective in identifying programs that contribute to the same or similar results
of many of its programs in its fiscal year 2000 performance plan, the plan’s usefulness to
congressional decisionmakers could be enhanced further. Identifying the results-oriented
performance goals that involve other agencies and by setting intermediate goals that clarify
the specific contribution the agency makes to the common results would make the plan more
informative. However, because NSF addresses many small parts of several crosscutting
programs, adding too much information could skew the level of detail in this area relative to
the rest of the plan.

For significant crosscutting projects, such as the U.S. Global Change Research Program, the
National Science and Technology Council developed an implementation plan that addresses
the achievement of performance goals and objectives by several participating agencies. Plans
such as these are specific to a particular goal and may be more useful to decisionmakers than
brief discussions in multiple agency plans. To encourage such efforts, a coordinating agency
could act as a focal point for other areas of significant crosscutting endeavors. Because it is
common and valuable for multiple agencies to approach similar fields of research from
different perspectives, better communication among agencies would enhance the
opportunities for collaboration, help to keep important questions from being overlooked, and




Page 10                                             GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the National Science Foundation's Performance Plan for Fiscal Year 2000




reduce instances of inefficient duplication of effort. Information on support for a particular
field of research could be provided to all agencies involved in it so that they can adjust their
efforts to ensure that the field is appropriately covered.

Agency Comments
On April 21, 1999, we obtained comments from NSF officials, including the Deputy Director,
on a draft of our analysis of the agency’s fiscal year 2000 annual performance plan. These
officials generally agreed with the observations made in the draft. They provided clarification
on several points about the linkages between performance and resources and about issues
concerning measurement and data verification and validation. We incorporated this
information in the report as appropriate. NSF officials pointed out that the Foundation is one
of the only agencies using the qualitative method to assess performance in research and
education by using the alternative format. To test this approach, officials are using its
Committees of Visitors process to assess performance for NSF’s first performance report for
March 2000. This point was incorporated in the body of the report.




Page 11                                             GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure II

Management Challenges


The following table identifies management challenges confronting the National Science
Foundation, as identified by the NSF Office of Inspector General (OIG).

Table II.1: Management Challenges in NSF’s Fiscal Year 2000 Performance Plan
Inspector General’s areas of concern                    Applicable goals and measures in the fiscal year
                                                        2000 annual performance plan
Managing an effective merit review system               Among the annual performance goals for NSF’s
                                                        investment process, NSF has two goals related to
                                                        merit review: (1) At least 90 percent of NSF funds will
                                                        be allocated to projects reviewed by appropriate peers
                                                        external to NSF and selected through a merit-based
                                                        competitive process, maintaining the fiscal year 1998
                                                        baseline and fiscal year 1999 goal of 90 percent, and
                                                        (2) NSF’s performance in implementation of the new
                                                        merit review criteria will be successful when reviewers
                                                        address the elements of both generic review criteria
                                                        appropriate to the proposal at hand and when program
                                                        officers take the information provided into account in
                                                        their decisions on awards; performance will be
                                                        minimally effective when reviewers consistently use
                                                        only a few of the suggested elements of the generic
                                                        review criteria, although others might be applicable.

                                                        In addition, according to NSF, the performance goal of
                                                        funding a percentage of new investigators is directly
                                                        responsive to the concern about the fair evaluation of
                                                        proposals from “individuals or institutions that do not
                                                        already receive NSF funding.” As part of the
                                                        performance evaluation of results, NSF will ask the
                                                        directorate advisory committees to examine the fiscal
                                                        year 2000 portfolio to characterize high-risk,
                                                        multidisciplinary, or innovative projects. This
                                                        addresses the OIG’s concerns about proposals with
                                                        novel ideas.

Capitalizing on NSF strengths when responding to        None. The key issues here are (1) the use of
increased expectations                                  competitive processes in merit review (addressed in
                                                        the performance goals for use of merit review and use
                                                        of merit review criteria) and (2) the emphasis on
                                                        fundamental research in emerging areas of
                                                        opportunity linked with societal interest (results-
                                                        oriented goals for discoveries and connections).

                                                        The OIG cited information technology and the
                                                        environment as areas where increasing NSF’s
                                                        leadership role is a challenge. An example of NSF’s
                                                        response to the challenge is the new emphasis in
                                                        biocomplexity. In the performance plan, NSF says that
                                                        relevant advisory committees will be asked to address
                                                        previous NSF efforts in the biocomplexity area. This is
                                                        an example of special emphasis areas under the
                                                        performance goals for discoveries and connections.




Page 12                                            GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure II
Management Challenges




Inspector General’s areas of concern                    Applicable goals and measures in the fiscal year
                                                        2000 annual performance plan
Using the Government Performance and Results Act        None. According to NSF, there are three important
                                                        concepts that NSF has incorporated in its plan to
                                                        implement the Results Act: (1) using the Results Act
                                                        as a NSF management tool for improving performance
                                                        (this is the underlying purpose for all of the NSF
                                                        performance goals); (2) minimizing burdens on NSF
                                                        staff and the external expert community (NSF has
                                                        worked to do this by using standard data systems that
                                                        are already part of NSF processes, linking external
                                                        expert assessment of NSF performance to preexisting
                                                        oversight functions, and developing an electronic
                                                        project reporting system designed to capture the
                                                        information needed for the assessment of results, and
                                                        to replace previous hard-copy systems); (3) preserving
                                                        the traditional roles of advisory committees in
                                                        providing advice and guidance (NSF is in the process
                                                        of testing an expert assessment process with advisory
                                                        committees and will fold its concerns about the
                                                        process into NSF’s continuing planning).
Responding to the Chief Financial Officer’s Act         None. According to NSF, the OIG’s discussion focuses
                                                        on one aspect of financial statements on which the
                                                        auditors could not express an opinion. The auditors
                                                        expressed an unqualified opinion on the fiscal year
                                                        1998 financial statements. There was not need to
                                                        address this for fiscal year 2000 because systems that
                                                        have been put in place are being maintained that
                                                        address this issue identified by audits in earlier years.
                                                        (Note: NSF was one of two agencies to receive an “A”
                                                        rating by the Congress for its fiscal year 1998 annual
                                                        financial report.)

Implementing FastLane: a new electronic proposal        FastLane is an electronic processing system that has
and award data information system                       been in development for several years. According to
                                                        NSF, FastLane will allow implementation of the final
                                                        project reporting system, one of NSF’s goals intended
                                                        to generate improved outcome data. NSF pointed out
                                                        that even when complete, it will continue to evolve to
                                                        reflect new technologies. Among NSF’s annual
                                                        performance goals for management are three that
                                                        relate to electronic proposal processing: (1) NSF will
                                                        receive at least 35 percent of full proposal
                                                        submissions electronically through FastLane,
                                                        improving on the fiscal year 1998 baseline of 17
                                                        percent; (2) by the end of fiscal year 2000, NSF will
                                                        have the technological capability of taking competitive
                                                        proposals submitted electronically through the entire
                                                        proposal and award/declination process without
                                                        generating paper within NSF; and (3) by the end of
                                                        fiscal year 2000, all relevant staff will receive
                                                        preliminary training on use of electronic
                                                        proposal/award jackets.




Page 13                                            GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan
Enclosure II
Management Challenges




Inspector General’s areas of concern                  Applicable goals and measures in the fiscal year
                                                      2000 annual performance plan
Managing the Antarctic program                        None, however, the Antarctic program challenge falls
                                                      under the broad outcome goals of NSF’s performance
                                                      plan. The facilities goals are also relevant, with the key
                                                      issue being support for facilities and logistics. Both are
                                                      addressed as part of NSF’s performance goals for
                                                      facilities construction and operation under the
                                                      investment goals.
Sustaining high scholarship and integrity             None. However, according to NSF, the agency has
                                                      systems in place to deal with the integrity issues
                                                      addressed and is responsive to the recommendations
                                                      of the OIG as reflected in the semiannual reports by
                                                      the OIG to the Congress. The OIG recommendation is
                                                      that the agency remain vigilant, which is its intention.
Spending funds effectively and efficiently            This challenge relates to developing partnerships
                                                      between NSF management and the OIG. As part of
                                                      the partnering effort, an Audit Coordination Committee
                                                      was implemented with the concurrence of the National
                                                      Science Board in fiscal year 1998. The data
                                                      verification sections indicate that this partnership is
                                                      working with regard to performance measurement.
                                                      Numerous indications of progress in this area are also
                                                      mentioned in the OIG’s semiannual reports.
Managing an effective system for cost-sharing         None. However, NSF is developing new policy in this
                                                      area. The draft policy has been shared with the
                                                      appropriate committee of the National Science Board
                                                      and will be on the agenda for approval at the Board’s
                                                      May meeting. NSF may consider developing a
                                                      performance goal related to the policy’s
                                                      implementation once it is finalized.
Managing salaries and administrative resources        None. NSF has proposed some budget changes that
                                                      could ameliorate the issues raised by the OIG. NSF’s
                                                      plan addresses staff recruitment and diversity in an
                                                      effort to be responsive to the OIG’s comments. The
                                                      performance plan does not address staff travel,
                                                      program oversight, and workload, except insofar as it
                                                      expresses the hope that electronic innovation will help
                                                      the agency address this. The fiscal year 2000 budget
                                                      request does include an increase in travel funds for
                                                      fiscal year 2000 to continue to support the merit review
                                                      process while increasing NSF’s capacity to do
                                                      oversight and outreach travel. According to NSF, this
                                                      increase will help to ensure both a reliable merit
                                                      review process and the oversight recommended by
                                                      the OIG.




(141303)




Page 14                                          GAO/RCED-99-206R NSF’s Fiscal Year 2000 Performance Plan