oversight

Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers

Published by the Government Accountability Office on 1999-02-26.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                     United States General Accounting Office

GAO                  Report to the Chairman, Committee on
                     Governmental Affairs, U.S. Senate



February 1999
                     AGENCY
                     PERFORMANCE PLANS
                     Examples of Practices
                     That Can Improve
                     Usefulness to
                     Decisionmakers




GAO/GGD/AIMD-99-69
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      General Government Division

      B-279902

      February 26, 1999

      The Honorable Fred Thompson
      Chairman, Committee on Governmental Affairs
      United States Senate

      Dear Mr. Chairman:

      The Government Performance and Results Act of 1993 seeks to improve
      the effectiveness, efficiency, and accountability of federal programs by
      establishing a system for agencies to set goals for program performance
      and to measure results. Under the Act, agencies are to prepare annual
      performance plans to systematically provide decisionmakers with
      information on the results to be achieved for a proposed level of
      resources. These plans are to reinforce the connection between the
      long-term strategic goals outlined in agencies’ strategic plans and the
      day-to-day activities of their program managers and staff. In so doing, the
      annual performance plans provide a basis for establishing accountability
      for results.

      Fiscal year 1999 marks the first time agencies across the federal
      government were to prepare annual performance plans covering each
      program activity in their budgets.1 As requested by several members of the
      congressional leadership, we reviewed the performance plans of the 24
      agencies covered by the Chief Financial Officers (CFO) Act. After briefing
      congressional staff, we issued separate reports on each of the 24 agencies’
      fiscal year 1999 performance plans.2 In September 1998, we issued a report
      that summarized our observations on the agencies’ plans and noted that
      substantial further development is needed for these plans to be useful in a
      significant way to congressional and other decisionmakers.3 That report
      identified opportunities to improve the usefulness of future performance
      plans. These opportunities include (1) better articulating a results
      orientation, (2) coordinating crosscutting programs, (3) clearly showing
      how strategies will be used to achieve goals, (4) showing performance
      consequences of budget decisions, and (5) building capacity within
      agencies to gather and use performance information.



      1
       The term “program activity” refers to the listing of projects and activities in the appendix portion of
      the Budget of the United States Government. Program activity structures are intended to provide a
      meaningful representation of the operations financed by a specific budget account.
      2
       See appendix I for the list of reports on fiscal year 1999 performance plans.
      3
       Managing for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance Plans
      (GAO/GGD/AIMD-98-228, Sept. 8, 1998).



      Page 1                                           GAO/GGD/AIMD-99-69 Agency Performance Plans
          B-279902




          As you requested, this report builds on those improvement opportunities
          by (1) identifying and describing practices that were applied in some plans
          that might, if consistently applied, improve the usefulness of all agencies’
          annual performance plans; and (2) providing examples from agencies’
          fiscal year 1999 performance plans that illustrate each practice. A
          forthcoming report that you also requested will provide additional
          information and examples related to one of those practices—showing the
          relationship of budgetary resources to performance goals.


          Taken as a whole, some of the fiscal year 1999 annual performance plans
Results   contain practices that, if consistently applied, should help improve the
          usefulness of subsequent agency performance plans to congressional and
          other decisionmakers. These practices are shown in table 1.




          Page 2                             GAO/GGD/AIMD-99-69 Agency Performance Plans
                                              B-279902




Table 1: Summary of Opportunities to Improve the Usefulness of Annual Plans and Applied Practices
Opportunities to improve the usefulness
of annual plans                           Applied practices
Better articulate a results orientation.      1. Create a set of performance goals and measures that addresses important
                                              dimensions of program performance and balances competing priorities.

                                              2. Use intermediate goals and measures to show progress or contribution to intended
                                              results.

                                              3. Include explanatory information on the goals and measures.

                                              4. Develop performance goals to address mission-critical management problems.

                                              5. Show baseline and trend data for past performance.

                                              6. Identify projected target levels of performance for multiyear goals.

                                              7. Link the goals of component organizations to departmental strategic goals.
Coordinate crosscutting programs.             8. Identify programs that contribute to the same or similar results.

                                              9. Set complementary performance goals to show how differing program strategies are
                                              mutually reinforcing and establish common or complementary performance measures,
                                              as appropriate.

                                              10. Describe—briefly or refer to a separate document—planned coordination strategies.
Show how strategies will be used to achieve   11. Link strategies and programs to specific performance goals and describe how they
goals.                                        will contribute to the achievement of those goals.

                                              12. Describe strategies to leverage or mitigate the effects of external factors on the
                                              accomplishment of performance goals.

                                              13. Discuss strategies to resolve mission-critical management problems.

                                              14. Discuss—briefly or refer to a separate document—plans to ensure that
                                              mission-critical processes and information systems function properly and are secure.
Show performance consequences of budget 15. Show how budgetary resources relate to the achievement of performance goals.
and other resource decisions.
                                        16. Discuss—briefly and refer to the agency capital plan—how proposed capital assets
                                        (specifically information technology investments) will contribute to achieving
                                        performance goals.

                                              17. Discuss—briefly or refer to a separate plan—how the agency will use its human
                                              capital to help achieve performance goals.
Build the capacity to gather and use          18. Identify internal and external sources for data.
performance information.
                                              19. Describe efforts to verify and validate performance data.

                                              20. Identify actions to compensate for unavailable or low-quality data.

                                              21. Discuss implications of data limitations for assessing performance.
                                              Source: GAO analysis of agency performance plans.




                                              Page 3                                     GAO/GGD/AIMD-99-69 Agency Performance Plans
                     B-279902




                     The applied practices and examples illustrating how these practices were
                     incorporated into certain agencies’ fiscal year 1999 performance plans are
                     discussed in the individual sections accompanying this report.


                     At your request, the objectives of this review were to (1) identify and
Objectives, Scope,   describe practices to improve the usefulness of agencies’ annual
and Methodology      performance plans and (2) provide examples from agencies’ fiscal year
                     1999 performance plans that illustrate each practice.

                     To meet both of our objectives, we contacted selected House and Senate
                     congressional staff—representing appropriation, authorization, budget,
                     and oversight committees that reviewed the 24 CFO Act agencies’ annual
                     plans—to solicit their views on the features they found to be particularly
                     useful in the plans. We also reviewed our individual and summary reports
                     on agencies’ fiscal year 1999 plans; the Results Act requirements for
                     agency annual performance plans; guidelines contained in the Office of
                     Management and Budget (OMB) Circular No. A-11, Part 2; fiscal year 1999
                     performance plans of the 24 CFO Act agencies; our guides to assist
                     agencies and Congress to effectively implement the Results Act;4 and our
                     work over the last several years examining agencies’ efforts to implement
                     the Results Act. Appendix II provides references to the Results Act; OMB
                     Circular No. A-11, Part 2; and our recently issued related products and is
                     arranged by the sections discussed in this report.

                     We did our work from March 1998 to January 1999 in Washington, D.C., in
                     accordance with generally accepted government auditing standards. We
                     did not seek agency comments on this report because much of the analysis
                     was drawn from our review of the fiscal year 1999 performance plans on
                     which the agencies had already commented. However, we asked officials
                     in each of the agencies profiled to verify the accuracy of the information
                     presented on their respective agencies’ fiscal year 1999 performance plans,
                     and we incorporated their comments where applicable. We did not
                     independently verify the accuracy of the information contained in the
                     agencies’ fiscal year 1999 performance plans.



                     4
                      The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance Plans, Version 1
                     (GAO/GGD-10.1.20, Apr. 1998); Agencies’ Annual Performance Plans Under the Results Act: An
                     Assessment Guide to Facilitate Congressional Decisionmaking, Version 1 (GAO/GGD/AIMD-10.1.18,
                     Feb. 1998); Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review,
                     Version 1 (GAO/GGD-10.1.16, May 1997); and Executive Guide: Effectively Implementing the
                     Government Performance and Results Act (GAO/GGD-96-118, June 1996). See also GPRA Performance
                     Reports (GAO/GGD-96-66R, Feb. 14, 1996).



                     Page 4                                        GAO/GGD/AIMD-99-69 Agency Performance Plans
B-279902




We are sending copies of this report to the Ranking Minority Member of
the Senate Committee on Governmental Affairs; the House Majority
Leader; the Chair and Ranking Minority Member of the House Committee
on Government Reform; other interested congressional committees; and to
the Director, Office of Management and Budget. We will also make copies
available to others on request.

The major contributors to this report are listed in appendix III. If you have
any questions concerning this report, please contact me on (202) 512-8676.

Sincerely yours,




J. Christopher Mihm
Associate Director, Federal Management and Workforce Issues




Page 5                             GAO/GGD/AIMD-99-69 Agency Performance Plans
Contents



Letter                                                                    1


Section 1                                                                10
Better Articulate a
Results Orientation
Section 2                                                                19
Coordinate
Crosscutting
Programs
Section 3                                                                24
Show How Strategies
Will Be Used to
Achieve Goals
Section 4                                                                29
Show Performance
Consequences of
Budget and Other
Resource Decisions
Section 5                                                                38
Build the Capacity to
Gather and Use
Performance
Information




                        Page 6   GAO/GGD/AIMD-99-69 Agency Performance Plans
                      Contents




Appendix I                                                                                     42
                      Departments                                                              42
GAO Products on       Independent Agencies                                                     43
Fiscal Year 1999
Performance Plans
Appendix II                                                                                    45
                      Section 1: Better Articulate a Results Orientation                       45
Related Guidance on   Section 2: Coordinate Crosscutting Programs                              46
Annual Performance    Section 3: Show How Strategies Will Be Used to Achieve Goals             46
                      Section 4: Show Performance Consequences of Budget and Other             48
Plans                   Resource Decisions
                      Section 5: Build the Capacity to Gather and Use Performance              50
                        Information


Appendix III                                                                                   51
Major Contributors
Tables                Table 1: Summary of Opportunities to Improve the Usefulness of            3
                        Annual Plans and Applied Practices
                      Table 1.1: Applied Practices to Address a Results Orientation            10
                      Table 2.1: Applied Practices to Address Crosscutting Programs            19
                      Table 3.1: Applied Practices to Address Goal Achievement                 24
                        Strategies
                      Table 4.1: Applied Practices to Address Budget and Resource              29
                        Decisions
                      Table 5.1: Applied Practices to Address PerformanceInformation           38
                        Capacity


Figures               Figure 1.1: Excerpt From the Department of Commerce’s Fiscal             12
                        Year 1999 Annual Performance Plan Featuring the National
                        Oceanic and Atmospheric Administration
                      Figure 1.2: Excerpt From the Department of the Treasury’s Fiscal         16
                        Year 1999 Annual Performance Plan Featuring the U.S. Customs
                        Service
                      Figure 1.3: Excerpt From the Department of Veterans Affairs’             17
                        Fiscal Year 1999 Annual Performance Plan
                      Figure 2.1: Excerpt From the Department of Commerce’s Fiscal             20
                        Year 1999 Annual Performance Plan




                      Page 7                           GAO/GGD/AIMD-99-69 Agency Performance Plans
Contents




Figure 2.2: Excerpt From the Department of Education’s Fiscal             23
  Year 1999 Annual Performance Plan
Figure 4.1: Excerpt From the Department of Health and Human               31
  Services’ Fiscal Year 1999 Annual Performance Plan Featuring
  the Indian Health Service
Figure 4.2: Excerpt From the Environmental Protection Agency’s            33
  Fiscal Year 1999 Annual Performance Plan
Figure 4.3: Excerpt From the Department of the Treasury’s Fiscal          35
  Year 1999 Annual Performance Plan Featuring the Internal
  Revenue Service




Abbreviations

CFO Act    Chief Financial Officers Act
GPRA       Government Performance and Results Act
OMB        Office of Management and Budget
ONDCP      Office of National Drug Control Policy
OPM        Office of Personnel Management


Page 8                            GAO/GGD/AIMD-99-69 Agency Performance Plans
Page 9   GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1

Better Articulate a Results Orientation


                                  Annual performance plans could be more useful documents in helping to
                                  guide decisions and, subsequently, assess actual performance if the plans
                                  consistently showed how the agency was adopting a results orientation.
                                  Table 1.1 identifies practices that can help agencies articulate a results
                                  orientation.

Table 1.1: Applied Practices to
Address a Results Orientation
                                  A performance plan that better articulates 1. Create a set of performance goals and
                                  a results orientation includes the         measures that addresses important
                                  following practices:                       dimensions of program performance and
                                                                             balances competing priorities.

                                                                             2. Use intermediate goals and measures to
                                                                             show progress or contribution to intended
                                                                             results.

                                                                             3. Include explanatory information on the
                                                                             goals and measures.

                                                                             4. Develop performance goals to address
                                                                             mission-critical management problems.

                                                                             5. Show baseline and trend data for past
                                                                             performance.

                                                                             6. Identify projected target levels of
                                                                             performance for multiyear goals.

                                                                             7. Link the goals of component
                                                                             organizations to departmental strategic
                                                                             goals.

                                  1. Create a set of performance goals and measures that addresses
                                  important dimensions of program performance and balances
                                  competing priorities.

                                  Performance goals and measures that successfully address important and
                                  varied aspects of program performance are key aspects of a results
                                  orientation. To illustrate, the Department of Transportation’s general goal
                                  to advance highway safety was included in its performance plan and was
                                  accompanied by various goals and measures to depict the complex
                                  performance they are intended to assess. For example, although the
                                  department’s performance goals and measures on the highway-related
                                  fatality rate and highway-related injury rate are the primary gauges of
                                  safety, they are complemented by other goals and measures that
                                  contribute to those results. Specifically, the plan includes a goal to




                                  Page 10                                GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1
Better Articulate a Results Orientation




increase the percentage rate of seat belt usage by front seat occupants, a
key element in reducing overall highway-related fatalities and injuries.

Federal programs are designed and implemented in dynamic environments
where competing program priorities and stakeholders’ needs must be
balanced continuously and new needs must be addressed.1 As a result,
programs are often forced to strike difficult balances among priorities that
reflect competing demands, such as timeliness, service quality, customer
satisfaction, program cost, and other stakeholder concerns. Sets of
performance goals and measures could provide a balanced perspective of
the intended performance of a program’s multiple priorities. In its
performance plan, for example, the Department of Commerce’s National
Oceanic and Atmospheric Administration defined its expected
performance to improve the timeliness and accuracy of short-range
environmental predictions that have immediate impact on individuals and
many sectors of the economy. The relationship between “timeliness” and
“accuracy” can represent competing demands because the accuracy of
predictions could be improved as the weather event comes closer to a
vicinity. However, the closer the weather event to a vicinity, the shorter
and less timely the warning or lead time to the public in that vicinity. For
fiscal year 1999, the agency set a lead time of 11 minutes for tornado
warnings with 70-percent accuracy. The agency also reported that it set a
lead time of 10 minutes with 59-percent accuracy in fiscal year 1997 and 10
minutes with 65-percent accuracy in fiscal year 1998. (See fig. 1.1.)




1
Performance and Accountability Series: Major Management Challenges and Program Risks: A
Governmentwide Perspective (GAO/OCG-99-1, Jan. 1999).



Page 11                                     GAO/GGD/AIMD-99-69 Agency Performance Plans
                                         Section 1
                                         Better Articulate a Results Orientation




Figure 1.1: Excerpt From the Department of Commerce’s Fiscal Year 1999 Annual Performance Plan Featuring the National
Oceanic and Atmospheric Administration




                                         Program cost is another important factor to be considered. In crafting the
                                         Results Act, Congress expected that agencies, whenever possible, would
                                         develop performance measures that correlate the level of program activity
                                         with program costs, such as costs per unit of result, costs per unit of



                                         Page 12                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1
Better Articulate a Results Orientation




service, or costs per unit of output. Agencies were expected to assign a
high priority to developing these types of unit cost measures. The
successful implementation of the managerial cost accounting standards
recommended by the Federal Accounting Standards Advisory Board and
issued by OMB and GAO are vital to providing agencies the program cost
information needed to develop such performance measures. Effective for
fiscal year 1998, these standards are to provide decisionmakers with
information on the costs of all resources used and the costs of services
provided by others to support activities or programs, thus allowing
decisionmakers to compare the costs of various programs and activities
with their performance outputs and results.

2. Use intermediate goals and measures to show progress or
contribution to intended results.

Intermediate goals and measures, such as outputs or intermediate
outcomes, can be used to show progress or contribution to intended
results. For instance, when it may take years before an agency sees the
results of its programs, intermediate goals and measures can provide
information on interim results. Also, when program results could be
influenced by external factors, agencies can use intermediate goals and
measures to identify the programs’ discrete contribution to a specific
result.

As an example, the Department of Health and Human Services’ Centers for
Disease Control and Prevention reported in its performance plan that
heart disease is the nation’s number one killer among men and women of
all racial and ethnic groups. However, because of the long latency period
of chronic diseases, such as heart disease, health outcome measures do
not provide a complete picture of the agency’s annual performance. To
address the burden of heart disease, the Centers for Disease Control and
Prevention intends, among other things, to focus on the prevention of risk
factors, such as tobacco use. The plan explained that comprehensive state
programs have been shown to be effective in reducing the prevalence of
tobacco use. To show a nationwide reduction in morbidity and mortality
attributable to behavioral risk factors in tobacco use, the plan presented
an output measure to increase the number of state programs with tobacco
prevention capacities from 17 in 1996 to 30 in 1999. The Centers for
Disease Control and Prevention intends to fund programs in all 50 states.




Page 13                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1
Better Articulate a Results Orientation




3. Include explanatory information on the goals and measures.

Explanatory information in a performance plan can help to show the
relationship among results-oriented goals, measures, and program outputs
and services. Such information is particularly important when an agency
uses intermediate goals or has several goals that need to be balanced. In
its performance plan, the Centers for Disease Control and Prevention
linked its program objectives to its performance goals and measures and
to the Department of Health and Human Services’ strategic goals. Also
included was a rationale for the use of process and output measures to
inform the user of the plan about limitations of the data and associated
program constraints. As previously illustrated, the section in its
performance plan on chronic disease prevention noted that health
outcome measures for these programs have been difficult to define for a
number of reasons, including the long latency period of chronic diseases
like heart disease.

4. Develop performance goals to address mission-critical
management problems.

Weaknesses in internal management processes and systems undermine the
achievement of program results. Therefore, the value of performance
plans could be increased if they more fully included performance goals to
address mission-critical management problems that may exist. Discussing
the most critical management problems—such as those identified by the
administration as a priority management objective in the governmentwide
performance plan, the agency’s financial audit, and other agency
assessments made by external organizations, including our recent
performance and accountability and high-risk series2—ensures that those
management problems that would have the greatest impact on results
receive the most attention. A specific reference to those priority
management objectives identified in the governmentwide performance
plan can help show an integrated approach to their resolution.

The Department of Education’s annual performance plan referred to the
priority management objectives identified in the fiscal year 1999
governmentwide performance plan. Specifically, the plan acknowledged
that Education’s strategic objective on postsecondary student aid delivery
and program management relates to the governmentwide performance
plan’s priority management objective on student aid programs. Education
is responsible for the collection of more than $150 billion in outstanding
2
 Performance and Accountability Series: Major Management Challenges and Program Risks
(GAO/OCG-99-22SET, Jan. 1999) and High-Risk Series: An Update (GAO/HR-99-1, Jan. 1999).



Page 14                                      GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1
Better Articulate a Results Orientation




loans, and its data systems track 93 million student loans and 15 million
Pell Grants. One of Education’s measures for the Federal Family
Education Loan and Direct Loan programs was a decline in the percentage
of borrowers leaving school who default on their student loans within 2
years. For the Federal Family Education Loan program, the annual plan
provided baseline and trend data for the default rate and indicated that the
rate declined from 22.4 to 10.4 percent from fiscal years 1990 to 1995. For
fiscal year 1999, Education’s goal for this program is to reduce the default
rate to 10.1 percent of borrowers. According to the plan, future declines
are likely to be steady but smaller because of the large number of
high-default schools that have already been eliminated from the program.

5. Show baseline and trend data for past performance.

With baseline and trend data, the more useful performance plans provided
a context for drawing conclusions about whether performance goals are
reasonable and appropriate. Decisionmakers can use such information to
gauge how a program’s anticipated performance level compares with
improvements or declines in past performance.

In addition to baseline and trend data on past performance, the
Department of the Treasury’s U.S. Customs Service performance plan
showed both actual and planned levels of performance for fiscal year 1997.
For example, in presenting its goal to maximize trade compliance,
Customs reported the extent of actual and proposed compliance with
trade laws in key industries. Specifically, in the bearings industry, Customs
reported that in fiscal year 1997, it achieved an 85.6 percent level of
compliance compared to a planned level of 83 percent; in the automobile
and truck parts industry, it achieved an 82.5 percent compliance level
compared to a planned level of 86 percent. The proposed performance
goals for fiscal year 1999 in both industries are 89 percent. (See fig. 1.2.)




Page 15                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
                                          Section 1
                                          Better Articulate a Results Orientation




Figure 1.2: Excerpt From the Department of the Treasury’s Fiscal Year 1999 Annual Performance Plan Featuring the U.S.
Customs Service




                                          6. Identify projected target levels of performance for multiyear
                                          goals.

                                          Where appropriate, an agency can convey what it expects to achieve in the
                                          long term by including multiyear performance goals in its performance
                                          plan. Such information can provide congressional and other
                                          decisionmakers with an indication of the incremental progress the agency
                                          expects to make in achieving results. As a start, for example, the



                                          Page 16                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
                                          Section 1
                                          Better Articulate a Results Orientation




                                          Department of Veterans Affairs’ performance plan has an objective for
                                          fiscal year 2002 to increase by 20 percent the number of patients (as
                                          opposed to visits) of its healthcare system over its fiscal year 1997
                                          baseline. The plan shows that the department projects achieving a
                                          4.4 percent increase in patients treated in fiscal year 1998 and an
                                          8.6 percent increase in fiscal year 1999. (See fig. 1.3.)


Figure 1.3: Excerpt From the Department of Veterans Affairs’ Fiscal Year 1999 Annual Performance Plan




                                          7. Link the goals of component organizations to departmental
                                          strategic goals.

                                          Linking component performance to departmental goals can provide a
                                          clear, direct understanding of how the achievement of the components’
                                          annual goals will lead to the achievement of the agency’s strategic goals.
                                          The Centers for Disease Control and Prevention, a component of the
                                          Department of Health and Human Services, linked its annual performance
                                          goals to the departmentwide strategic goals. For example, the plan set an
                                          objective to reduce the case rate of tuberculosis—an infectious disease
                                          that was nearly eliminated in the United States in the mid-1980s, only to
                                          reemerge strongly in the 1990s with drug-resistant strains. The plan linked



                                          Page 17                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1
Better Articulate a Results Orientation




that objective to Health and Human Services’ departmentwide goals to
reduce major threats to the health and productivity of all Americans.
Specific performance goals and measures were included in the plan to
reduce the tuberculosis case rate, such as increase to 85 percent the
patients who will complete a course of curative treatment within 12
months of initiation of treatment.




Page 18                                   GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2

Coordinate Crosscutting Programs


                                  A focus on results implies that agencies will coordinate efforts in
                                  significant program areas where responsibility for achieving results is
                                  shared among agencies. Agencies’ annual performance plans can be useful
                                  tools for identifying such crosscutting programs, setting complementary
                                  performance goals, and describing planned coordination strategies. Table
                                  2.1 identifies applied practices that can help agencies address
                                  coordination of crosscutting programs.

Table 2.1: Applied Practices to
Address Crosscutting Programs
                                  A performance plan that addresses           8. Identify programs that contribute to the
                                  coordination of crosscutting programs       same or similar results.
                                  includes the following practices:
                                                                              9. Set complementary performance goals
                                                                              to show how differing program strategies
                                                                              are mutually reinforcing and establish
                                                                              common or complementary performance
                                                                              measures, as appropriate.

                                                                              10. Describe—briefly or refer to a separate
                                                                              document—planned coordination
                                                                              strategies.

                                  8. Identify programs that contribute to the same or similar results.

                                  Annual performance plans that include listings of programs and agencies
                                  that share common results are useful in that they begin to document the
                                  range and degree to which those federal programs have shared
                                  responsibilities for achieving results. These listings can serve as the
                                  prelude to the substantive work and fuller discussion of how agencies’
                                  contributions to the same or similar results are being coordinated. As a
                                  first step, the Department of Commerce included in its performance plan a
                                  table identifying interagency activities within which the department
                                  participates, such as formulating policy proposals. The three themes
                                  drawn from Commerce’s mission statement provide the organizational
                                  framework for the listing that includes crosscutting activities by bureaus
                                  between Commerce and other federal agencies. (See fig. 2.1.)




                                  Page 19                                 GAO/GGD/AIMD-99-69 Agency Performance Plans
                                        Section 2
                                        Coordinate Crosscutting Programs




Figure 2.1: Excerpt From the Department of Commerce’s Fiscal Year 1999 Annual Performance Plan




                                        9. Set complementary performance goals to show how differing
                                        program strategies are mutually reinforcing and establish common
                                        or complementary performance measures, as appropriate.

                                        After agencies have identified other agencies with which they have a
                                        shared responsibility, the next step is to include in their performance
                                        plans complementary performance goals to show how differing program
                                        strategies are mutually reinforcing and establish common or
                                        complementary performance measures, as appropriate. Taking this



                                        Page 20                              GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2
Coordinate Crosscutting Programs




further, the usefulness of a performance plan to congressional and other
decisionmakers could be enhanced if an agency identifies the
results-oriented performance goals that involve other agencies and sets
intermediate goals that clarify the specific contribution the agency makes
to the common result.

The Department of Health and Human Services’ Indian Health Service
associated its annual performance goals to the governmentwide goals of
Healthy People. Started in 1979, Healthy People is a series of
outcome-based public health objectives and measures developed and
updated each decade by the U.S. Public Health Service in consultation
with other federal agencies, state governments, and national organizations.
The objectives for 2000 succeeded the 1990 health objectives that were set
in 1980. The Public Health Service has started planning for the 2010 health
objectives, which are scheduled to be released in January 2000.

In developing its performance plan, the Indian Health Service reviewed the
appropriateness of the Healthy People objectives that focused on the most
significant health problems affecting American Indians and Alaska Natives
and the essential services that address them. The plan reported that
virtually all of the Service’s performance goals support the Healthy People
objectives for 2000. For example, the Indian Health Service’s fiscal year
1999 goal to ensure that the proportion of the American Indian and Alaska
Native female population over 40 years of age who have had screening
mammography is no lower than the fiscal year 1996 level relates to a
Healthy People objective on reducing the number of women who die from
breast cancer. The plan also described the approach for meeting this goal,
as well as the data sources and baseline data.

The performance plans for both the Department of Transportation and the
Department of Justice referenced the Office of National Drug Control
Policy (ONDCP) goal framework. Established in 1988, the principal purpose
of ONDCP is to establish policies, priorities, and objectives for the nation’s
drug control program. In conjunction with stakeholders from both the
public and private sectors, ONDCP has proposed a system to assess the
effectiveness of the national drug control strategy to reduce the use of
drugs, their availability, and the damaging consequences associated with
drug use and trafficking. The strategic goals and performance measures
form a framework that shows how the efforts of more than 50 federal drug
control agencies’ programs—along with state, local, and foreign
governments and the private sector—can contribute to the drug control
strategy.



Page 21                             GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2
Coordinate Crosscutting Programs




Transportation’s performance plan stated that the Coast Guard is working
with ONDCP to develop a performance goal and indicator measuring the
rate of illegal drug smuggling in noncommercial maritime routes. Justice’s
performance plan stated it will continue to work with ONDCP and other
agencies in developing and implementing an interagency performance
measurement system to help gauge the effectiveness of the national drug
control strategy.

10. Describe—briefly or refer to a separate document—planned
coordination strategies.

The more useful performance plans not only identified crosscutting
efforts, they also described how agencies expected to coordinate efforts
with other agencies that have similar responsibilities. When interagency
coordination is difficult because of disparate or even conflicting missions
across agencies, performance plans offer agencies an opportunity to
identify statutory or regulatory options for better aligning disparate
missions and for identifying initiatives to avoid or mitigate conflicts over
strategic and performance goals.

Because progress in coordinating crosscutting programs is still in its
infancy, discussions in performance plans of what is being done can
contribute to the development of a common base for understanding the
strengths and weaknesses of various approaches to coordination. The
discussion in the annual performance plans can describe opportunities for
interagency coordination that are identified in other agencies’ strategic
and performance plans, describe the status of coordination efforts, and
develop interagency strategies to achieve shared outcome goals in the
most efficient and effective manner. As a next step, annual performance
plans could discuss how agencies can cooperate in data collection and
program evaluation.

For example, the Department of Education’s performance plan discussed
how it plans to coordinate its programs that share common results and its
efforts to cooperate with others to collect data and conduct program
evaluations. For its strategic objective that schools are strong, safe,
disciplined, and drug-free, the performance plan indicated that the agency
will continue to pursue joint prevention activities, such as truancy and
hate crimes initiatives, with the Department of Justice. Education intends
to develop a database for the President’s Report Card on School Violence
with the Department of Justice and make maximum use of data from other
federal agencies. Education also plans to continue to cooperate on



Page 22                             GAO/GGD/AIMD-99-69 Agency Performance Plans
                                         Section 2
                                         Coordinate Crosscutting Programs




                                         evaluation projects with Justice and Health and Human Services. (See fig.
                                         2.2.)


Figure 2.2: Excerpt From the Department of Education’s Fiscal Year 1999 Annual Performance Plan




                                         Page 23                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3

Show How Strategies Will Be Used to
Achieve Goals

                                      The more useful annual performance plans discussed how specific
                                      strategies and programs will contribute to achieving performance goals.
                                      Agencies can also use their performance plans to articulate their strategies
                                      to leverage or mitigate key external factors that will affect program
                                      performance and strategies to address mission-critical management
                                      problems that threaten the achievement of program results. Table 3.1
                                      identifies applied practices that can help agencies show how strategies
                                      will be used to achieve goals.

Table 3.1: Applied Practices to
Address Goal Achievement Strategies
                                      A performance plan that shows how          11. Link strategies and programs to
                                      strategies will be used to achieve goals   specific performance goals and describe
                                      includes the following practices:          how they will contribute to the achievement
                                                                                 of those goals.

                                                                                 12. Describe strategies to leverage or
                                                                                 mitigate the effects of external factors on
                                                                                 the accomplishment of performance goals.

                                                                                 13. Discuss strategies to resolve
                                                                                 mission-critical management problems.

                                                                                 14. Discuss—briefly or refer to a separate
                                                                                 document—plans to ensure that
                                                                                 mission-critical processes and information
                                                                                 systems function properly and are secure.

                                      11. Link strategies and programs to specific performance goals and
                                      describe how they will contribute to the achievement of those
                                      goals.

                                      The more useful annual performance plans linked the agencies’ strategies
                                      and programs to specific performance goals and described how they will
                                      contribute to the achievement of those goals. The listings of current
                                      programs and initiatives that often were included in agencies’ fiscal year
                                      1999 plans were useful for providing an understanding of what agencies
                                      do. However, presentations that more directly explain how programs and
                                      initiatives achieve goals will be most helpful to congressional and other
                                      decisionmakers in assessing the degree to which strategies are appropriate
                                      and reasonable. For example, an agency might state in its performance
                                      plan that it will administer grants to states. In order to link this strategy to
                                      its intended results, the agency should explain how administering grants to
                                      states will help it achieve specific performance goals. Over time and as
                                      agencies gain experience in linking strategies to results, agencies will be in




                                      Page 24                                GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3
Show How Strategies Will Be Used to
Achieve Goals




a better position to understand and discuss alternative strategies and
program design and the relative contributions these alternatives could
make to results. A subsequent discussion of how different governing tools
(for example, intergovernmental partnerships, performance-based
contracts, financial grants and credits, and direct service provision by the
federal government) can be used in achieving goals could further enhance
the plans.

The Department of Agriculture’s Food Safety and Inspection Service
provided a detailed discussion of its strategy to achieve its performance
goal of reducing pathogens on raw meat and poultry products. The
performance plan proposed to expand in-plant inspections and laboratory
testing of meat and poultry samples taken from slaughtering plants.
Laboratory personnel will conduct expanded testing of pathogens of
samples taken from plants slaughtering cattle, swine, chicken, and turkeys
and use upgraded technology to conduct inspections including
stethoscopes, thermometers, and computer software programs that are
not currently available.

The Federal Emergency Management Agency identified in its performance
plan a number of strategies relating to its Project Impact program, which
is designed to promote predisaster mitigation. One performance goal for
this program is to increase the number of Project Impact communities in
each state by at least one. The strategies identified for achieving this goal
included working with states and federal agencies to identify candidate
communities, providing grants as seed funding, providing technical
information, and monitoring progress.

12. Describe strategies to leverage or mitigate the effects of
external factors on the accomplishment of performance goals.

The extent to which results are achieved is influenced by a number of
external factors, such as economic, social, and technological
developments; and the actions of numerous outside entities, including
state and local governments. The Results Act requires agencies to identify
key external factors in their strategic plans that would affect the degree to
which they achieve their strategic goals. The value of performance plans
can be augmented by including discussions on the strategies agencies plan
to take to leverage or mitigate the influence these factors can have on
achieving results. Such discussions can help congressional and other
decisionmakers determine if the agency has the best mix of program




Page 25                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3
Show How Strategies Will Be Used to
Achieve Goals




strategies in place to achieve its goals, or if additional agency or
congressional actions are needed to better meet those goals.

The Department of Agriculture’s Food and Nutrition Service discussed
external factors as well as strategies to mitigate conditions. The annual
performance plan included a performance goal to reduce food stamp
trafficking, which is the illegal sale of food stamps for cash. The plan cited
the agency’s dependence on grocery stores to properly handle food stamps
as an external factor that affects the agency’s ability to achieve this goal.
To mitigate this factor, the plan noted that it has promulgated rules that
participating stores must follow and instituted a system of sanctions that
may be applied to stores that violate the rules.

13. Discuss strategies to resolve mission-critical management
problems.

Performance plans that contain specific strategies to resolve
mission-critical management problems provide congressional and other
decisionmakers with an understanding of how the agency plans to
improve its management. By discussing such activities, plans also provide
a basis for accountability. As mentioned earlier, agencies should set goals
to address these management problems, such as those identified by the
administration as a priority management objective in the governmentwide
performance plan, the agency’s financial audit, and other agency
assessments by external organizations, including our performance and
accountability and high-risk series. Annual performance plans can lay out
concrete actions the agency plans to resolve mission-critical management
problems that threaten the achievement of major program goals.

As a first step, Commerce’s performance plan identified major
management challenges raised by the Inspector General, such as the
department’s financial management system, which is also included in our
performance and accountability series. The plan stated that the system,
the Commerce Administrative Management System, is substantially over
its original estimated cost and continues to experience performance
shortfalls and schedule delays. The plan identified actions the department
will take to reduce expenditures and impose greater control over the
project’s direction, such as implementing the system as a pilot at the
Census Bureau.




Page 26                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3
Show How Strategies Will Be Used to
Achieve Goals




14. Discuss—briefly or refer to a separate document—plans to
ensure that mission-critical processes and information systems
function properly and are secure.

By including information technology issues in annual performance plans,
agencies underscore the critical role that technology plays in achieving
results. Key information technology issues include the ability to ensure
that mission-critical information systems will function properly after the
year 1999 and are secure from intrusion and misuse—two of the issues we
have identified to be high risk for the federal government.1 Year
2000-induced failures of one or more mission-critical systems may have a
severe impact on agencies’ abilities to deliver critical services. Agencies
can use their fiscal year 2000 performance plans, which become effective
in October 1999, to discuss their final business continuity and contingency
plans to reduce the risk of Year 2000 business failures.

As a start, the Social Security Administration reported in its performance
plan that it expects that by January 1999 all of its mission-critical systems
will have the ability to operate properly in 2000. It reported that it is also
preparing a Year 2000 contingency plan that addresses how core business
processes will be supported if planned conversion activities experience
unforeseen disruptions.

Despite the sensitivity and criticality of federal information systems, they
are not being adequately protected from unauthorized access.2 Although
advances in the use of electronic data promise to streamline federal
operations and improve the delivery of federal services, security
weaknesses can create serious pervasive risks for the federal government,
such as potential disclosure of sensitive data, loss of assets worth billions
of dollars due to fraud, and disruption of critical operations. The annual
performance plan could discuss the agency’s ability to ensure the integrity
and availability of information resources supporting mission-critical
operations and the confidentiality of sensitive information.

As a first step, the Department of Transportation’s fiscal year 1999
performance plan stated that a key component of continued security of its
information system is the education of the workforce. By the end of fiscal
year 1998, the plan stated that the agency will develop information systems
security training for various elements of the workforce. A fiscal year 1999

1
 High-Risk Series: An Update (GAO/HR-99-1, Jan. 1999).
2
 Information Security: Serious Weaknesses Place Critical Federal Operations and Assets at Risk
(GAO/AIMD-98-92, Sept. 23, 1998).



Page 27                                       GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3
Show How Strategies Will Be Used to
Achieve Goals




milestone is that 100 percent of senior management, 75 percent of system
administrators, and 60 percent of end users will be trained.

In its performance plan, the Office of Personnel Management discussed
plans to enhance its information security program by conducting internal
and external evaluations of its systems, such as engaging the assistance of
the National Security Agency to review its security capabilities and
implementing appropriate recommendations to improve its security. In
addition to training staff, the agency will have in place a tested disaster
recovery capability for general support and major financial, benefits, and
workforce information application systems.




Page 28                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4

Show Performance Consequences of Budget
and Other Resource Decisions

                                  The more useful annual performance plans related the performance
                                  consequences of budget and other resource decisions by indicating how
                                  the funding from agencies’ program activities will be allocated to a
                                  discrete set of performance goals. Table 4.1 identifies applied practices
                                  that can help agencies show performance consequences of budget and
                                  other resource decisions.

Table 4.1: Applied Practices to
Address Budget and Resource
Decisions                         A performance plan that shows               15. Show how budgetary resources relate
                                  performance consequences of budget          to the achievement of performance goals.
                                  and other resource decisions includes
                                  the following practices:                    16. Discuss—briefly and refer to the
                                                                              agency capital plan—how proposed
                                                                              capital assets (specifically information
                                                                              technology investments) will contribute to
                                                                              achieving performance goals.

                                                                              17. Discuss—briefly or refer to a separate
                                                                              plan—how the agency will use its human
                                                                              capital to help achieve performance goals.

                                  15. Show how budgetary resources relate to the achievement of
                                  performance goals.

                                  A key expectation of the Results Act is that Congress will gain a clearer
                                  understanding of what is being achieved in relation to what is being spent.
                                  Therefore, plans are more useful when they convey the requested funding
                                  level associated with achieving a discrete set of performance goals and
                                  identify where that funding was included in the structure of agencies’
                                  budget requests. With such information, decisionmakers can better
                                  compare planned levels of accomplishments with the resources requested.
                                  Decisionmakers can then consider how significant changes from the
                                  requested amounts for these goals could affect performance. Our
                                  forthcoming report on this practice provides additional information on the
                                  variety of approaches agencies used to connect budget resources with
                                  results.

                                  The Results Act gives agencies the flexibility to aggregate, disaggregate, or
                                  consolidate the budget’s program activities so that they align with the
                                  performance goals. If an agency takes one of these approaches, its
                                  performance plan will be more useful if it includes a crosswalk that shows
                                  how the program activities in its budget request relate to the program
                                  activities in its annual performance plan. In addition to this flexibility, an




                                  Page 29                                 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4
Show Performance Consequences of Budget
and Other Resource Decisions




agency may propose a change to its budget account or program activity
structures to facilitate an understanding of performance.

The Department of Health and Human Services’ Indian Health Service has
taken the first step in its performance plan to aggregate program activities
for the purposes of performance planning. The plan includes a crosswalk
that aligns its budget activities with its performance in four areas
identified as Results Act (GPRA) aggregated program activities: Treatment;
Prevention; Capital Programming/Infrastructure; and Partnerships,
Consultation, Core Functions, and Advocacy. The crosswalk indicates that
from two separate budget accounts—Services and Facilities—the Indian
Health Service consolidated program activities (6, 7, 8, and 9 from the
Services budget account and 21 and 22 from the Facilities budget account)
in the area of Prevention. The crosswalk also shows that budget program
activities 21 and 22 support GPRA program activities 2, 3, and 4 (Prevention;
Capital Programming/Infrastructure; and Partnerships, Consultation, Core
Functions, and Advocacy). (See fig. 4.1.) The Indian Health Service’s
intended performance in the Prevention area includes such things as
increasing childhood immunization rates, reducing deaths by unintentional
injuries, and reducing childhood obesity.




Page 30                               GAO/GGD/AIMD-99-69 Agency Performance Plans
                                         Section 4
                                         Show Performance Consequences of Budget
                                         and Other Resource Decisions




Figure 4.1: Excerpt From the Department of Health and Human Services’ Fiscal Year 1999 Annual Performance Plan
Featuring the Indian Health Service




                                         Page 31                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4
Show Performance Consequences of Budget
and Other Resource Decisions




The Environmental Protection Agency proposed changes to its program
activity structure to better align its budget resources with performance
goals. The agency proposed a uniform program activity structure across all
of its accounts in which each program activity represents one of the
agency’s strategic goals. Using this realigned program activity structure in
its annual plan, the agency showed, by account, the funding it requested in
order to achieve each strategic objective and the supporting annual
performance goals. For example, it requested almost $22 million in fiscal
year 1999 to meet its strategic objective related to the reduction of acid
rain. This request is associated with performance goals relating to
emissions of sulfur dioxide (SO2) and nitrogen oxides (NOx) that react in
the atmosphere and fall to earth as acid rain. The request for the acid rain
strategic objective was related to three accounts from the “clean air”
program activity: Environmental Program and Management, Science and
Technology, and State and Tribal Assistance Grants. (See fig. 4.2.)




Page 32                               GAO/GGD/AIMD-99-69 Agency Performance Plans
                                         Section 4
                                         Show Performance Consequences of Budget
                                         and Other Resource Decisions




Figure 4.2: Excerpt From the Environmental Protection Agency’s Fiscal Year 1999 Annual Performance Plan




                                         Page 33                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4
Show Performance Consequences of Budget
and Other Resource Decisions




In addition to aligning budget resources with performance goals and
including trend data and target levels of performance, the Department of
the Treasury’s Internal Revenue Service displayed in its performance plan
the budget amounts that corresponded with the performance actually
achieved in a given fiscal year. Specifically, the agency showed that for
fiscal year 1997, it received budget authority of almost $795 million in its
Submission Processing Activity to achieve a set of performance goals:
Improve Customer Service, Increase Compliance, and Increased
Productivity. The performance level that it achieved in fiscal year 1997 to
Improve Customer Service included, among other things, a timeliness
refund rate of 14.5 days for electronic filings, an improvement over the
rates in previous years of 15.5 days in fiscal year 1996 and 21 days in fiscal
year 1995. (See fig. 4.3.)




Page 34                               GAO/GGD/AIMD-99-69 Agency Performance Plans
                                          Section 4
                                          Show Performance Consequences of Budget
                                          and Other Resource Decisions




Figure 4.3: Excerpt From the Department of the Treasury’s Fiscal Year 1999 Annual Performance Plan Featuring the
Internal Revenue Service




                                          Page 35                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4
Show Performance Consequences of Budget
and Other Resource Decisions




16. Discuss—briefly and refer to the agency capital plan—how
proposed capital assets (specifically information technology
investments) will contribute to achieving performance goals.

The federal government’s spending on major physical capital investment is
projected to total over $68 billion in fiscal year 1999. Capital assets include
information technology; equipment; land; structures; and intellectual
property, such as software. With federal agencies facing increasing
demands to improve performance, the importance of making the most
effective capital acquisition choices—choices that are linked to
results—will also intensify. A number of laws are beginning to propel
agencies toward improving their capital decisionmaking practices. For
example, Title V of the Federal Acquisition Streamlining Act of 1994 was
designed to foster the development of measurable cost, schedule, and
performance goals and incentives for acquisition personnel to reach these
goals. The Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of
1996 put in place statutory requirements for agencies to better link their
information technology plans and decisions to their missions and
programmatic goals.

OMB  has taken an active leadership role in seeking to link capital
decisionmaking to program results. OMB developed a Capital Programming
Guide that provides agencies with the key elements for producing effective
plans and investments.1 OMB’s Guide drew on our work on best practices
used by leading private sector and state and local governments, which was
subsequently published.2 Consistent with these best practices, OMB has
required agencies to submit 5-year capital spending plans and
justifications—thus encouraging a longer-term consideration of agency
capital needs and alternatives for addressing them. OMB’s Guide provides a
basic reference on principles and techniques, including appropriate
strategies for analyzing benefits and costs, preparing budget justifications,
and managing capital assets once they are in place. In addition, OMB has
worked closely with the President’s Commission to Study Capital
Budgeting, which is expected to issue its report and recommendations
soon.

In this regard, a more useful performance plan describes the processes,
technologies, and types of resources, including capital, that are needed to
achieve performance goals. Further, performance plans provide agencies
with the opportunity to show how a proposed capital asset will, for

1
 Capital Programming Guide, Version 1, July 1997 (Executive Office of the President, OMB).
2
 Executive Guide: Leading Practices in Capital Decision-Making (GAO/AIMD-99-32, Dec. 1998).



Page 36                                       GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4
Show Performance Consequences of Budget
and Other Resource Decisions




example, decrease costs, improve service quality, or increase productivity.
For example, the Department of the Treasury’s U.S. Customs Service
performance plan explained that continuing the acquisition and
installation of the Land Border automation equipment is needed to allow
inspectors to perform more careful screening and questioning of vehicle
occupants, which should help to achieve Customs’ goal of improving its
efficiency at targeting arriving vehicles for enforcement purposes.

17. Discuss—briefly or refer to a separate plan—how the agency
will use its human capital to help achieve performance goals.

In an era that demands improved performance at reduced costs, an
agency’s success depends upon its ability to assemble the human capital
with the right blend of talents and skills. Thus, human capital planning
must be an integral part of an organization’s strategic and program
planning; human capital itself should be thought of not as a cost to be
minimized but as an asset to be enhanced. To show that the vital
integration of human capital and program planning is in place, the more
useful annual performance plans could discuss the size and
composition—in terms of knowledge, skills, and abilities—of the human
capital needed to support the achievement of performance goals and
explain the workforce planning methods by which these needs were
determined. Also, the more useful plans could describe how strategies—in
such areas as recruitment and hiring, retention and separation, training
and career development, and employee incentives and accountability
systems—could meet workforce needs and support the achievement of
annual performance goals.

The U.S. Agency for International Development stated in its performance
plan that its direct-hire workforce has been trimmed by hiring freezes,
early retirements, and a reduction-in-force in 1996. In the wake of these
events, the plan recognized that the agency must identify critical skills
needed to achieve its goals. Its Office of Human Resources is examining
issues and changes in workforce planning and is focusing, among other
things, on improving basic personnel operations, such as assignments and
employee evaluations.




Page 37                               GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5

Build the Capacity to Gather and Use
Performance Information

                                  In order to successfully measure progress toward intended results,
                                  agencies need to build the capacity to gather and use performance
                                  information. Under the Results Act, agencies are to discuss in their annual
                                  performance plans how they will verify and validate the performance
                                  information that they plan to use to show whether goals are being met.
                                  The usefulness of agencies’ plans could increase if they identify data
                                  sources, describe efforts to verify and validate data, identify actions to
                                  compensate for unavailable data or low-quality data, and discuss
                                  implications of data limitations for assessing performance. Table 5.1
                                  identifies applied practices that can help agencies describe their capacity
                                  to gather and use performance information.

Table 5.1: Applied Practices to
Address Performanceinformation
Capacity                          A performance plan that describes an       18. Identify internal and external sources
                                  agency’s capacity to gather and use        for data.
                                  performance information includes the
                                  following practices:                       19. Describe efforts to verify and validate
                                                                             performance data.

                                                                             20. Identify actions to compensate for
                                                                             unavailable or low-quality data.

                                                                             21. Discuss implications of data limitations
                                                                             for assessing performance.

                                  18. Identify internal and external sources for data.

                                  The usefulness of performance data ultimately depends on the degree of
                                  confidence that users have in that data. The more informative annual
                                  performance plans can provide decisionmakers with confidence in the
                                  agency’s ability to report on its performance goals and measures by
                                  identifying existing internal and external data sources. In those instances
                                  when performance data do not exist, a description in the performance
                                  plan of efforts by the agency to identify relevant data sources could inform
                                  decisionmakers of actions the agency is taking in order to measure
                                  performance.

                                  The Department of Justice identified in its performance plan specific
                                  sources of performance data for many of its performance measures.
                                  According to its plan, data are already collected and reported through
                                  existing statistical series and internal Justice data systems, such as the
                                  department’s Bureau of Justice Statistics—the primary source of justice




                                  Page 38                                GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5
Build the Capacity to Gather and Use
Performance Information




statistics on crime, criminal offenders, victims of crime, and the operation
of the justice system at all levels of government.

As another example, the Small Business Administration recognized in its
performance plan its need for data from external sources, such as the
Federal Procurement Data System. This system provides information
relating to the agency’s goal to help increase the share of federal
procurement dollars awarded to small firms. Specifically, the Federal
Procurement Data System provides the agency with data on the
percentage of federal procurement dollars awarded to small businesses,
women-owned businesses, and small disadvantaged businesses.

19. Describe efforts to verify and validate performance data.

Performance plans are to include descriptions of the procedures that will
be used to verify and validate the measured values of actual performance.
The procedures should be credible and specific to ensure that
performance information is sufficiently complete, accurate, and consistent
to document performance and support decisions on how best to manage
programs. Information in the performance plan about the procedures to be
used to verify and validate the data—such as scope, methodology, and
timing—can reinforce an agency’s ability to generate reliable and timely
performance data.

To ensure the accuracy and reliability of reported data, the Department of
Justice stated in its performance plan that it will (1) examine and analyze
data to identify anomalies or inconsistencies, (2) make comparisons to
other data series measuring the same or similar variables, (3) require the
submitting entities to certify the accuracy and completeness of data and to
identify any data limitations, (4) institute appropriate quality control
checks, (5) conduct periodic data audits, and (6) obtain independent
audits of its financial statements. Justice expects that its component
organizations will independently pursue several of these initiatives,
consistent with their individual data requirements.

As part of its efforts to underscore that program managers are to use
performance data to make decisions, the Department of Education has
decided to hold program managers accountable for the quality of the
performance data related to their programs. In its performance plan, the
Department of Education established accountability for data quality with
its program managers as one of its strategies for ensuring high-quality
information. Specifically, program managers will attest that the data used



Page 39                                GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5
Build the Capacity to Gather and Use
Performance Information




for their program’s performance measurement are reliable and valid, or
they will submit plans for data improvement. To validate program
managers’ assertions to the quality of their data, Inspector General audits
and program evaluations will be used. Education will support this strategy
through guidance to managers on developing and monitoring quality data
systems and using data to manage program performance.

In addition to the procedures used to verify and validate performance
data, the use of program evaluations is important for assessing how well a
program is working, identifying the factors affecting performance, and
pinpointing improvement opportunities. In its performance plan, the
Department of Transportation recognized that program evaluation will
help it set meaningful performance goals by quantifying the relationship
between program efforts and levels of outcomes. The plan further
identified specific program evaluations planned for fiscal years 1998 and
1999. The department also noted that it is developing a method to both
plan and track program evaluations, as well as to develop the skills and
resources sufficient to conduct evaluations throughout its organization.

20. Identify actions to compensate for unavailable or low-quality
data.

When data are unavailable or existing data contain significant limitations,
providing credible data may require new information systems or major
changes to existing systems. We recently reported, for example, that some
agencies developed or adopted standardized data collection systems to
obtain data not readily available to them or to fill information gaps.1
However, an agency’s ability to collect and generate data often needs to be
weighed against several factors, such as cost or the burden on
respondents. The more informative annual performance plans provide a
vehicle for discussing where the agency struck such a balance and the
implications for the quality of the data—including accuracy, reliability, and
timeliness—and the cost of collection.

The Department of Education recognized that to effectively report the
performance of the department in achieving its strategic plan goals and
objectives requires developing some new data systems and fixing old ones.
For example, Education stated in its performance plan that its student aid
delivery system has suffered from significant data quality problems. The
plan outlined several steps that Education plans to take to address these

1
 Managing for Results: Measuring Program Results That Are Under Limited Federal Control
(GAO/GGD-99-16, Dec. 11, 1998).



Page 40                                      GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5
Build the Capacity to Gather and Use
Performance Information




problems, including improving data accuracy by establishing industrywide
standards for data exchanges, receiving individual student loan data
directly from lenders, expanding efforts to verify data reported to the
National Student Loan Data System, and preparing a systems architecture
for the delivery of federal student aid.

21. Discuss implications of data limitations for assessing
performance.

Performance plans could be more useful if they recognize and identify
significant data limitations and their implications for assessing
performance. Plans can identify known data limitations from internal and
external sources, such as quality or timeliness of the data; indicate what
actions will be taken to compensate for the limitations; and report when
those actions will be taken. The more informative performance plans also
identify implications for decisionmaking arising from data limitations on
reported results.

The Department of Transportation stated in its performance plan that one
of the most significant limitations of both internal and external data is
timeliness. One way the department will deal with this limitation is to
compile preliminary estimates from the portion of data that is available in
time to report on the performance measures. According to the plan,
fatality data from the first 6 months of the year could be compared with
data from the first 6 months of the previous year for an initial performance
measurement.




Page 41                                GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix I

GAO Products on Fiscal Year 1999
Performance Plans

              Managing for Results: An Agenda to Improve the Usefulness of Agencies’
              Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

              The Results Act: Assessment of the Governmentwide Performance Plan
              for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

              Managing for Results: Measuring Program Results That Are Under Limited
              Federal Control (GAO/GGD-99-16, Dec. 11, 1998).


              Results Act: Observations on the U.S. Department of Agriculture’s Annual
Departments   Performance Plan for Fiscal Year 1999 (GAO/RCED-98-212R, June 11, 1998).

              Results Act: Observations on the Department of Commerce’s Annual
              Performance Plan for Fiscal Year 1999 (GAO/GGD-98-135R, June 24, 1998).

              Results Act: DOD’s Annual Performance Plan for Fiscal Year 1999
              (GAO/NSIAD-98-188R, June 5, 1998).

              The Results Act: Observations on the Department of Education’s Fiscal
              Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8, 1998).

              Results Act: Observations on DOE’s Annual Performance Plan for Fiscal
              Year 1999 (GAO/RCED-98-194R, May 28, 1998).

              The Results Act: Observations on the Department of Health and Human
              Services’ Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-180R,
              June 17, 1998).

              Results Act: Observations on the Department of Housing and Urban
              Development’s Fiscal Year 1999 Annual Performance Plan
              (GAO/RCED-98-159R, June 5, 1998).

              Results Act: Department of the Interior’s Annual Performance Plan for
              Fiscal Year 1999 (GAO/RCED-98-206R, May 28, 1998).

              Observations on the Department of Justice’s Fiscal Year 1999 Performance
              Plan (GAO/GGD-98-134R, May 29, 1998).

              Results Act: Observations on Labor’s Fiscal Year 1999 Performance Plan
              (GAO/HEHS-98-175R, June 4, 1998).




              Page 42                            GAO/GGD/AIMD-99-69 Agency Performance Plans
                       Appendix I
                       GAO Products on Fiscal Year 1999
                       Performance Plans




                       The Results Act: Observations on the Department of State’s Fiscal Year
                       1999 Annual Performance Plan (GAO/NSIAD-98-210R, June 17, 1998).

                       Results Act: Observations on the Department of Transportation’s Annual
                       Performance Plan for Fiscal Year 1999 (GAO/RCED-98-180R, May 12, 1998).

                       Results Act: Observations on Treasury’s Fiscal Year 1999 Annual
                       Performance Plan (GAO/GGD-98-149, June 30, 1998).

                       Results Act: Observations on VA’s Fiscal Year 1999 Performance Plan
                       (GAO/HEHS-98-181R, June 10, 1998).


                       Results Act: EPA’s Annual Performance Plan for Fiscal Year 1999
Independent Agencies   (GAO/RCED-98-166R, Apr. 28, 1998).

                       Results Act: Observations on the Federal Emergency Management
                       Agency’s Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-207R,
                       June 1, 1998).

                       Results Act: Observations on the General Services Administration’s Annual
                       Performance Plan (GAO/GGD-98-110, May 11, 1998).

                       Managing for Results: Observations on NASA’s Fiscal Year 1999
                       Performance Plan (GAO/NSIAD-98-181, June 5, 1998).

                       Results Act: NSF’s Annual Performance Plan for Fiscal Year 1999
                       (GAO/RCED-98-192R, May 19, 1998).

                       Results Act: NRC’s Annual Performance Plan for Fiscal Year 1999
                       (GAO/RCED-98-195R, May 27, 1998).

                       Results Act: Observations on the Office of Personnel Management’s
                       Annual Performance Plan (GAO/GGD-98-130, July 28, 1998).

                       Results Act: Observations on the Small Business Administration’s Fiscal
                       Year 1999 Annual Performance Plan (GAO/RCED-98-200R, May 28, 1998).

                       The Results Act: Observations on the Social Security Administration’s
                       Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R, June 9, 1998).




                       Page 43                            GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix I
GAO Products on Fiscal Year 1999
Performance Plans




The Results Act: Observations on USAID’s Fiscal Year 1999 Annual
Performance Plan (GAO/NSIAD-98-194R, June 25, 1998).




Page 44                            GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II

Related Guidance on Annual Performance
Plans

                       This is a compilation of related guidance, including the Results Act, OMB
                       Circular A-11, and GAO reports and is arranged by the sections discussed in
                       this report.


                       Government Performance and Results Act (Results Act), 5 U.S.C. 306(c),
Section 1: Better      31 U.S.C. 1115(a)(1), 1115(a)(2), 1115(a)(4), and 1115(a)(5).
Articulate a Results
Orientation            Senate Committee on Governmental Affairs Report accompanying the
                       Results Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance
                       Plans”; pp. 27-28, “Federal Government Performance Plan”; p. 28, “Agency
                       Performance Plans”; p. 29, “Performance Goals”; and pp. 29-30,
                       “Performance Indicators.”

                       OMB Circular A-11(1998), secs. 210.4, 220.1, 220.3, 220.4, 220.5(a), 220.5(b),
                       220.5(c), 220.5(f), 220.6(a), 220.6(b), 220.6(d), 220.7(a), 220.7(b), 220.7(c),
                       220.7(d), 220.7(f), 220.9(a), 220.9(b), 220.9(c), 220.9(d), 220.9(e), 220.13,
                       220.15, 220.16, 221.1(a), 221.1(b), 221.1(c), 221.2(a), 221.2(b), 221.2(c),
                       221.2(d), 221.2(e), and 221.3.

                       Managing for Results: Measuring Program Results That Are Under Limited
                       Federal Control (GAO/GGD-99-16, Dec. 11, 1998).

                       Managing for Results: An Agenda to Improve the Usefulness of Agencies’
                       Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

                       The Results Act: Assessment of the Governmentwide Performance Plan
                       for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

                       The Results Act: An Evaluator’s Guide to Assessing Agency Annual
                       Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

                       Agencies’ Annual Performance Plans Under the Results Act: An
                       Assessment Guide to Facilitate Congressional Decisionmaking, Version 1
                       (GAO/GGD/AIMD-10.1.18, Feb. 1998).

                       The Government Performance and Results Act: 1997 Governmentwide
                       Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

                       Managing for Results: Analytic Challenges in Measuring Performance
                       (GAO/HEHS/GGD-97-138, May 30, 1997).




                       Page 45                              GAO/GGD/AIMD-99-69 Agency Performance Plans
                        Appendix II
                        Related Guidance on Annual Performance
                        Plans




                        Measuring Performance: Strengths and Limitations of Research Indicators
                        (GAO/RCED-97-91, Mar. 21, 1997).

                        Executive Guide: Effectively Implementing the Government Performance
                        and Results Act (GAO/GGD-96-118, June 1996).

                        GPRA   Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

                        Implementation of the Government Performance and Results Act (GPRA), A
                        Report on the Chief Financial Officer’s Role and Other Issues Critical to
                        the Governmentwide Success of GPRA, Chief Financial Officers Council,
                        GPRA Implementation Committee, May 1995.



                        OMB   Circular A-11(1998), secs. 220.7(e) and 220.9(f).
Section 2: Coordinate
Crosscutting            Managing for Results: An Agenda to Improve the Usefulness of Agencies’
Programs                Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

                        The Results Act: Assessment of the Governmentwide Performance Plan
                        for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

                        The Results Act: An Evaluator’s Guide to Assessing Agency Annual
                        Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

                        Agencies’ Annual Performance Plans Under the Results Act: An
                        Assessment Guide to Facilitate Congressional Decisionmaking, Version 1
                        (GAO/GGD/AIMD-10.1.18, Feb. 1998).

                        Managing for Results: Using the Results Act to Address Mission
                        Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).

                        The Government Performance and Results Act: 1997 Governmentwide
                        Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).


                        Results Act, 31 U.S.C. 9703.
Section 3: Show How
Strategies Will Be      Senate Committee on Governmental Affairs Report accompanying Results
Used to Achieve Goals   Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance Plans”;
                        pp. 17-18, “Managerial Flexibility Waivers”; and pp. 34-36, “Section 5.
                        Managerial Accountability and Flexibility.”



                        Page 46                                  GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II
Related Guidance on Annual Performance
Plans




OMB Circular A-11 (1998), secs. 220.5(d), 220.5(e), 220.9(b), 220.9(d),
220.11(a), 220.11(b), 220.11(c), 220.11(d), and 220.11(e).

Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, Nov. 1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’
Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: Assessment of the Governmentwide Performance Plan
for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

Year 2000 Computing Crisis: Business Continuity and Contingency
Planning (GAO/AIMD-10.1.19, Aug. 1998).

Executive Guide: Information Security Management (GAO/AIMD-98-68,
May 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual
Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Executive Guide: Measuring Performance and Demonstrating Results of
Information Technology Investments (GAO/AIMD-98-89, Mar. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An
Assessment Guide to Facilitate Congressional Decisionmaking, Version 1
(GAO/GGD/AIMD-10.1.18, Feb. 1998).

Year 2000 Computing Crisis: An Assessment Guide (GAO/AIMD-10.1.14, Sept.
1997).

The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Managing for Results: Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997).

Business Process Reengineering Assessment Guide, Version 3
(GAO/AIMD-10.1.15, Apr. 1997).

Privatization: Lessons Learned by State and Local Governments
(GAO/GGD-97-48, Mar. 14, 1997).




Page 47                                  GAO/GGD/AIMD-99-69 Agency Performance Plans
                     Appendix II
                     Related Guidance on Annual Performance
                     Plans




                     Executive Guide: Effectively Implementing the Government Performance
                     and Results Act (GAO/GGD-96-118, June 1996).


                     Results Act, 31 U.S.C. 1115(a)(3) and 1115(c).
Section 4: Show
Performance          Senate Committee on Governmental Affairs Report accompanying Results
Consequences of      Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance Plans”;
                     pp. 18-19, “Performance Budgeting”; pp. 29-30, “Performance Indicators”;
Budget and Other     and p. 31, “Coverage of Program Activities.”
Resource Decisions
                     OMB Circular A-11 (1998), secs. 220.1, 220.8(a), 220.8(b), 220.8(c), 220.8(d),
                     220.8(e), 220.9(d), 220.10(a), 220.10(b), 220.10(c), 220.11(a), 220.11(d),
                     221.1(a), 221.1(b), 221.1(c), and Part 3.

                     OMB   Capital Programming Guide, v. 1.0 (July 1997).

                     Executive Guide: Leading Practices in Capital Decision-Making
                     (GAO/AIMD-99-32, Dec. 1998).

                     Managing for Results: An Agenda to Improve the Usefulness of Agencies’
                     Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

                     Performance Management: Aligning Employee Performance With Agency
                     Goals at Six Results Act Pilots (GAO/GGD-98-162, Sept. 4, 1998).

                     The Results Act: An Evaluator’s Guide to Assessing Agency Annual
                     Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

                     Acquisition Reform: Implementation of Key Aspects of the Federal
                     Acquisition Streamlining Act of 1994 (GAO/NSIAD-98-81, Mar. 9, 1998).

                     Budget Issues: Budgeting for Capital (GAO/T-AIMD-98-99, Mar. 6, 1998).

                     Executive Guide: Measuring Performance and Demonstrating Results of
                     Information Technology Investments (GAO/AIMD-98-89, Mar. 1998).

                     Agencies’ Annual Performance Plans Under the Results Act: An
                     Assessment Guide to Facilitate Congressional Decisionmaking, Version 1
                     (GAO/GGD/AIMD-10.1.18, Feb. 1998).




                     Page 48                                  GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II
Related Guidance on Annual Performance
Plans




The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Performance Budgeting: Past Initiatives Offer Insights for GPRA
Implementation (GAO/AIMD-97-46, Mar. 27, 1997).

Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ IT
Investment Decision-making, Version 1 (GAO/AIMD-10.1.13, Feb. 1997).

Information Technology Investment: Agencies Can Improve Performance,
Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept. 30, 1996).

Executive Guide: Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996).

GPRA   Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Transforming the Civil Service: Building the Workforce of the
Future—Results of a GAO-Sponsored Symposium (GAO/GGD-96-35, Dec. 26,
1995).

Integrating Performance Measurement into the Budget Process, Chief
Financial Officers Council, GPRA Implementation Committee
Subcommittee Project, September 22, 1997.

Federal Accounting Standards Advisory Board (FASAB) Volume 1 Original
Statements: Statements of Federal Financial Accounting Concepts and
Standards, Statement of Federal Financial Accounting Standards No. 1,
Objectives of Federal Financial Reporting (GAO/AIMD-21.1.1, Mar. 1997).

FASAB Volume 1 Original Statements: Statements of Federal Financial
Accounting Concepts and Standards, Statement of Federal Financial
Accounting Standards No. 4, Managerial Cost Accounting Standards
(GAO/AIMD-21.1.1, Mar. 1997).

Budget and Financial Management: Progress and Agenda for the Future
(GAO/T-AIMD-96-80, Apr. 23, 1996).




Page 49                                  GAO/GGD/AIMD-99-69 Agency Performance Plans
                       Appendix II
                       Related Guidance on Annual Performance
                       Plans




                       Results Act 31 U.S.C. 1115 (a)(6).
Section 5: Build the
Capacity to Gather     Senate Committee on Governmental Affairs Report accompanying Results
and Use Performance    Act (Senate Report 103-58, June 16, 1993), p. 30, “Verification and
                       Validation.”
Information
                       OMB   Circular A-11(1998), secs. 220.6(a), 220.12(a), and 220.12(b).

                       Managing for Results: Measuring Program Results That Are Under Limited
                       Federal Control (GAO/GGD-99-16, Dec. 11, 1998).

                       Managing for Results: An Agenda to Improve the Usefulness of Agencies’
                       Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

                       The Results Act: An Evaluator’s Guide to Assessing Agency Annual
                       Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

                       Agencies’ Annual Performance Plans Under the Results Act: An
                       Assessment Guide to Facilitate Congressional Decisionmaking, Version 1
                       (GAO/GGD/AIMD-10.1.18, Feb. 1998).

                       Managing for Results: Regulatory Agencies Identified Significant Barriers
                       to Focusing on Results (GAO/GGD-97-83, June 24, 1997).

                       The Government Performance and Results Act: 1997 Governmentwide
                       Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

                       Managing for Results: Analytic Challenges in Measuring Performance
                       (GAO/HEHS/GGD-97-138, May 30, 1997).

                       Measuring Performance: Strengths and Limitations of Research Indicators
                       (GAO/RCED-97-91, Mar. 21, 1997).

                       Executive Guide: Effectively Implementing the Government Performance
                       and Results Act (GAO/GGD-96-118, June 1996).

                       GPRA   Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).




                       Page 50                                  GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix III

Major Contributors


                     Alan M. Stapleton, Assistant Director
General Government   Lisa R. Shames, Project Manager
Division             Dorothy L. Self, Senior Evaluator


                     The examples used in this report are drawn from the assessments of the
Acknowledgments      individual agency annual performance plans that were done by staff across
                     GAO. Thus, in addition to the individuals noted above, the staff who worked
                     on the individual agency plan assessments and summary assessment also
                     made important contributions to this report. These individuals are
                     identified in the separate reports on agency fiscal year 1999 plans.




(410311)             Page 51                           GAO/GGD/AIMD-99-69 Agency Performance Plans
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested