oversight

Performance Budgeting: Initial Experiences Under the Results Act in Linking Plans With Budgets

Published by the Government Accountability Office on 1999-04-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                     United States General Accounting Office

GAO                  Report to the Chairman, Committee on
                     Governmental Affairs,
                     U.S. Senate


April 1999
                     PERFORMANCE
                     BUDGETING

                     Initial Experiences
                     Under the Results Act
                     in Linking Plans With
                     Budgets




GAO/AIMD/GGD-99-67
      United States
GAO   General Accounting Office
      Washington, D.C. 20548                                                                                Leter




      Accounting and Information
      Management Division                                                                                   Leter




      B-282035                                                                                       Letter

      April 12, 1999

      The Honorable Fred Thompson
      Chairman
      Committee on Governmental Affairs
      United States Senate

      Dear Mr. Chairman:

      The Government Performance and Results Act of 1993 (the Results Act)
      seeks to strengthen federal decision-making and accountability by focusing
      on the results of federal activities and spending. A key expectation is that
      the Congress will gain a clearer understanding of what is being achieved in
      relation to what is being spent. To accomplish this, the act requires that,
      beginning for fiscal year 1999, agencies prepare annual performance plans
      containing annual performance goals covering the program activities in
      agencies’ budget requests. In addition, Office of Management and Budget
      (OMB) guidance states that agency performance plans should display, by
      program activity, the funding level being applied to achieve performance
      goals.1 Plans that meet these expectations can provide the Congress with
      useful information on the performance consequences of budget decisions.

      Our assessment of fiscal year 1999 performance plans found that agencies
      generally covered the program activities in their budgets, but most plans
      did not identify how the funding for those program activities would be
      allocated to performance goals.2 To enhance the Congress’ understanding
      of issues that may affect agencies’ abilities to relate budgetary resources
      and results, you requested that we review selected fiscal year 1999
      performance plans to (1) describe agencies’ approaches to linking
      performance goals and budgetary resources, (2) examine characteristics
      that might be associated with different approaches to linking performance
      goals and budgetary resources, and (3) identify implications for future
      efforts to clarify the relationship between budgetary resources and results.
      More clearly describing and analyzing the approaches agencies developed
      to link funding requests with performance expectations can be an



      1OMB   Circular A-11, Preparation and Submission of Budget Estimates, Sec. 220.9(e), June 23, 1997.

      2
       Managing for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance Plans
      (GAO/GGD/AIMD-98-228, September 8, 1998). For a discussion of key practices for improving the use-
      fulness of annual performance plans, see Agency Performance Plans: Examples of Practices That Can
      Improve Usefulness to Decisionmakers (GAO/GGD/AIMD-99-69, February 26, 1999).




      Page 1                                             GAO/AIMD/GGD-99-67 Performance Budgeting
                   B-282035




                   important foundation as the Congress reviews agencies’ fiscal year 2000
                   performance plans.

                   To respond to your request, we selected 35 fiscal year 1999 performance
                   plans for review that covered entire agencies or large subordinate bureaus,
                   services, or administrations.3 We developed a methodology to characterize
                   each plan’s structure and method of linking program activities and
                   performance goals, and we gathered information on each agency’s budget
                   structure. We also identified agency plans in which program activity
                   funding was allocated to performance goals—a key criterion for useful
                   plans.4 We analyzed how characteristics differed among agencies that
                   made such allocations and those that did not to determine which, if any, of
                   the characteristics were statistically significant in making these
                   allocations. We provided a draft of this report to OMB for comment and
                   also discussed selected aspects of this report with staff from the Senate
                   and House appropriations committees. See appendix I for a detailed
                   discussion of our objectives, scope, and methodology.



Results in Brief   In their first Results Act performance plans, agencies experimented with a
                   variety of approaches to connect budget requests with anticipated results.
                   Although most agencies reviewed (30 of 35) defined some type of
                   relationship between the program activities of their proposed budgets and
                   the performance goals of their plans, far fewer (14 or 40 percent of the
                   plans reviewed) translated these relationships into budgetary terms—that
                   is, most plans did not explain how funding would be allocated to achieve
                   performance goals. Such allocations are a critical first step in defining the
                   performance consequences of budgetary decisions. We found that
                   agencies with budget and planning structures of widely varying complexity
                   made these allocations, but some common approaches were used.
                   Agencies were significantly more likely to have allocated funding to
                   program activities if they (1) showed simple, clear relationships between
                   program activities and performance goals (as illustrated by eight agencies


                   3We generally focused on bureau-level plans for each department but limited our review to the three
                   largest bureaus with discretionary spending over $1 billion and/or the two largest bureaus. For a list of
                   plans reviewed, see table I.1 in appendix I.

                   4
                     While an allocation of funding requests to performance expectations is important, plans must also
                   have other attributes to be fully useful. In addition to showing the performance consequences of budget
                   decisions, useful plans need to contain results-oriented goals, clear strategies, and credible
                   performance information. See GAO/GGD/AIMD-98-228, September 8, 1998 and GAO/GGD/AIMD-99-69,
                   February 26, 1999.




                   Page 2                                              GAO/AIMD/GGD-99-67 Performance Budgeting
                      B-282035




                      in our review); (2) fully integrated performance plans into congressional
                      budget justifications (as illustrated by five agencies); or (3) had changed
                      their program activity structures to reflect their goal structures (as
                      illustrated by three agencies).

                      Agencies’ first-year experiences show progress in bringing planning and
                      budgeting structures and presentations closer together, but much remains
                      to be done if performance information is to be more useful for budget
                      decision-making. Continued efforts are needed to (1) clarify and
                      strengthen links between planning and budgeting structures and
                      presentations and (2) address persistent challenges in performance
                      planning and measurement and cost accounting. The progress that has
                      been made, the challenges that persist—including the indefinite delay in
                      the performance budgeting pilots called for by the act—and the Congress’
                      interest in having credible, results-oriented information underscore the
                      importance of developing an agenda to ensure continued improvement in
                      showing the performance consequences of budgetary decisions.
                      Therefore, we recommend that the Director of OMB assess the linkages
                      between performance goals and program activities presented in the fiscal
                      year 2000 plans and develop a constructive and practical agenda to further
                      clarify the relationship between budgetary resources and results.



The Results Act and   The Government Performance and Results Act of 1993 is a key component
                      of a statutory framework that the Congress put in place during the 1990s to
Performance           promote a new focus on results.5 Finding that waste and inefficiency in
Budgeting             federal programs were undermining confidence in government, the
                      Congress sought to hold federal agencies accountable for the results of
                      federal spending through regular and systematic performance planning,
                      measurement, and reporting. Among its several purposes, the act is
                      designed to improve congressional decision-making by providing more
                      objective information on the relative effectiveness and efficiency of federal
                      programs and spending. That is, with regard to spending decisions, the act
                      aims for a closer and clearer link between the process of allocating
                      resources and the expected results to be achieved with those resources.




                      5
                       For a fuller discussion of this framework, see Managing for Results: The Statutory Framework for
                      Performance-Based Management and Accountability (GAO/GGD/AIMD-98-52, January 28, 1998).




                      Page 3                                            GAO/AIMD/GGD-99-67 Performance Budgeting
                        B-282035




The Concept of          The concept of linking performance information with the budget is
Performance Budgeting   commonly known as performance budgeting.6 Within the past 50 years,
                        initiatives at the local, state, and federal levels of government as well as in
                        other nations have sought to link performance expectations with specific
                        budget amounts. In essence, the concept of performance budgeting
                        assumes that a systematic presentation of performance information
                        alongside budget amounts will improve budget decision-making by
                        focusing funding choices on program results. Specifically, performance
                        budgeting seeks to shift the focus of attention from detailed items of
                        expense—such as salaries and travel—to the allocation of resources based
                        on program goals and measured results. In this sense, the Results Act is the
                        most recent of a series of federal initiatives embodying concepts of
                        performance budgeting.

                        At the federal level and elsewhere, performance budgeting initiatives have
                        encountered many challenges.7 Key challenges include a lack of credible
                        and useful performance information, difficulties in achieving consensus on
                        goals and measures, dissimilarities in program and fund reporting
                        structures, and limitations of information and accounting systems. For
                        example, some prior initiatives used new and unfamiliar formats that were
                        layered onto existing budget and appropriations processes, compromising
                        the goal of integrating performance information into the budget process.
                        Specifically in the federal government, past performance budgeting
                        initiatives resulted in unique and often voluminous presentations
                        unconnected to the structures and processes used in congressional budget
                        decision-making.8

                        When viewed collectively, these past initiatives suggest three common
                        themes. First, any effort to link plans and budgets must explicitly involve
                        the executive and legislative branches of our government. Past initiatives
                        often faltered because the executive branch developed plans and
                        performance measures in isolation from congressional oversight and


                        6
                         In this report, we use the term “performance budgeting” to refer generally to the process of linking
                        expected results to budget levels, but not to any particular approach.

                        7See Performance Budgeting: State Experiences and Implications for the Federal Government
                        (GAO/AFMD-93-41, February 17, 1993). For a discussion of past federal initiatives—including the
                        Planning-Programming-Budgeting-System (PPBS) and Zero-Based Budgeting (ZBB)—and the evolution
                        of the concept and techniques of performance budgeting, see Performance Budgeting: Past Initiatives
                        Offer Insights for GPRA Implementation (GAO/AIMD-97-46, March 27, 1997).

                        8
                            See GAO/AIMD-97-46, March 27, 1997.




                        Page 4                                              GAO/AIMD/GGD-99-67 Performance Budgeting
                        B-282035




                        resource allocation processes. Second, the concept of performance
                        budgeting will likely continue to evolve. Past initiatives demonstrated that
                        there is no single definition of performance budgeting that encompasses
                        the range of needs and interests of federal decisionmakers. Third and
                        perhaps most importantly, past initiatives showed that performance
                        budgeting cannot be viewed in simplistic terms—that is, resource
                        allocation cannot be mechanically linked to results. The process of
                        budgeting is inherently an exercise in political choice—allocating scarce
                        resources among competing needs—in which performance information
                        can be one, but not the only, factor underlying decisions. Ultimately, the
                        promise of any performance budgeting initiative, including the Results Act,
                        lies in its potential to more explicitly infuse performance information into
                        budgetary deliberations, thereby changing the terms of debate from simple
                        inputs to expected and actual results.


Performance Budgeting   The Results Act differs from earlier performance budgeting initiatives in
Under the Results Act   several key respects but can be viewed as a continued evolution of the
                        concept.9 At its most basic level, the act requires agencies’ annual
                        performance plans to directly link performance goals and the program
                        activities of their budget requests. Testifying on the Results Act before its
                        passage, the Director of OMB characterized this linkage as a “limited—but
                        very useful—form of performance budgeting . . . .”10 The act also requires
                        that another form of performance budgeting be tested during performance
                        budgeting pilots.

                        The Results Act requires an agency’s annual performance plan to cover
                        each program activity in the President’s budget request for that agency.
                        Subject to clearance by OMB and generally resulting from negotiations
                        between agencies and appropriations subcommittees, program activity
                        structures are intended to provide a meaningful representation of the
                        operations financed by a specific budget account. Typically, the President’s
                        annual budget submission encompasses over 1,000 accounts and over 3,000
                        program activities. As the committee report accompanying the act noted,
                        however, the program activity structure is not consistent across the federal




                        9See   GAO/AIMD-97-46, March 27, 1997.

                        10
                           Government Performance and Results Act of 1993, Committee on Governmental Affairs, United
                        States Senate, S. Rpt. No. 58, 103d Cong. 1st Sess., p. 19 (1993).




                        Page 5                                          GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




government but rather is tailored to individual accounts.11 The committee
report further cautioned that agencies’ annual plans should not be
voluminous presentations that overwhelm rather than inform the reader.
Accordingly, the act gives agencies the flexibility to consolidate, aggregate,
or disaggregate program activities, as illustrated in figure 1, so long as no
major function or operation of the agency is omitted or minimized. In
addition to this flexibility, agencies also have the option to propose
changing their budget structures, subject to concurrence from OMB and
the Congress.12



Figure 1: Using the Results Act’s Flexibility to Align the Annual Performance Plan
With the Budget Account and Program Activity Structure

       %XGJHW DFFRXQWV DQG                                                    $QQXDO
        SURJUDP DFWLYLWLHV                                               SHUIRUPDQFH SODQ

      Budget account 1

         Program activity 1
                                             Aggregation                    Performance goal 1
         Program activity 2

         Program activity 3
                                                                            Performance goal 2
             Activity 3.1 a
                                           Disaggregation
             Activity 3.2a

                                                                            Performance goal 3
         Program activity 4

      Budget account 2                     Consolidation                    Performance goal 4
          Program activity 1

a
 These activities are created through disaggregation and would not necessarily appear in the list of
program activities presented in the President’s budget.




11S.   Rpt. No. 103-58, p. 31.

12
     OMB Circular A-11, Sec. 11.6(c), June 23, 1997.




Page 6                                                 GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




OMB’s guidance regarding this provision of the act set forth an additional
criterion: plans should display, generally by program activity, the funding
level being applied to achieve performance goals.13 That is, OMB expected
performance plans to show how amounts from the agency’s budget request
would be allocated to the performance goals displayed in the plan.14

In addition to mandating a linkage between budget requests and
performance plans, the act required that pilot projects be used to test
another approach to performance budgeting. OMB, in consultation with
the head of each agency, was required to designate for fiscal years 1998 and
1999 at least five agencies to prepare budgets that “present, for one or more
of the major functions and operations of the agency, the varying levels of
performance, including outcome-related performance, that would result
from different budgeted amounts.”15 While the act required agencies to
define goals consistent with the level of funding requested in the
President’s budget, the act’s pilots would also show how performance
would change if the agency received more or less than requested. OMB
was to include these pilot performance budgets as an alternative
presentation in the President’s budget for fiscal year 1999 and to transmit a
report to the President and to the Congress no later than March 31, 2001, on
the feasibility and advisability of including a performance budget as part of
the President’s budget. This report is also to recommend whether
legislation requiring performance budgets should be proposed.

The performance budgeting pilots were scheduled to start in fiscal year
1998 “so that they would begin only after agencies had sufficient
experience in preparing strategic and performance plans, and several years
of collecting performance data.”16 In this context, and recognizing the
importance of concentrating on governmentwide implementation in 1998,
OMB announced on May 20, 1997, that the pilots would be delayed for at
least a year. OMB stated that the performance budgeting pilots would
require the ability to calculate the effects on performance of marginal
changes in cost and funding. According to OMB, very few agencies had this
capability, and the delay would give time for its development. In


13
     OMB Circular A-11, Sec. 220.9(e), June 23, 1997.
14
  Subsequently, in its guidance on fiscal year 2000 plans, OMB noted that it expected to see “significant
progress in associating funding with specific performance goals or sets of goals” in agencies’ plans.

1531    U.S.C. 1119(b).

16
     S. Rpt No. 103-58, p. 38.




Page 7                                                  GAO/AIMD/GGD-99-67 Performance Budgeting
                           B-282035




                           September 1998, OMB solicited agencies’ comments on these pilots, but no
                           agencies have been designated as pilots. At present, OMB has no definite
                           plans for proceeding with the performance budgeting pilots.



Agencies Linked            The complexities that are the hallmark of today’s federal budget account
                           structure and the diverse planning structures associated with broad federal
Complex Planning and       missions were reflected in the approaches used to link budget and planning
Budgeting Structures       structures. The fiscal year 1999 performance plans we reviewed frequently
                           depicted complex and imprecise relationships between these structures.
Using Many                 Using the Results Act’s flexibility to aggregate, consolidate, and
Approaches                 disaggregate existing program activities, most agencies generally linked
                           their program activities to some level of a frequently complex planning
                           hierarchy. These linkages were often presented in a performance plan that
                           was separable from the agency’s budget justification. As a result, plans
                           frequently depicted a relationship between program activities and
                           performance goals that, while consistent with the act’s charge to cover
                           program activities, proved difficult to translate into budgetary terms.
                           However, 14, or 40 percent of the agencies we reviewed, built on these
                           relationships to show how the funding from program activities would be
                           allocated to achieve discrete sets of performance goals. These agencies, in
                           effect, took the first step toward defining the performance consequences of
                           budget decisions.


The Complexity of Budget   As we have previously reported, the federal government’s budget account
and Planning Structures    structure was not created as a single, integrated framework but rather
                           generally developed over time to respond to specific needs.17 As a result,
                           budget accounts and program activities within the accounts vary from
                           agency to agency. This complexity was evident for agencies included in
                           this review. For example, the number of budget accounts associated with a
                           given agency’s performance plan ranged from a low of 1 to a high of 118 for
                           the agencies we reviewed, while the number of program activities to be
                           covered by the plan ranged from 6 to 465. The median number of accounts
                           for agencies that we reviewed was 9, and the median number of program
                           activities was 32. Typically, program activity structures were unique not
                           only to each of the agencies in our review, but also to each of the budget
                           accounts within an agency. In only 2 agencies—EPA and the Department


                           17
                                Budget Account Structure: A Descriptive Overview (GAO/AIMD-95-179, September 18, 1995).




                           Page 8                                             GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




of Defense—were the same program activity titles repeated across all or
groups of an agency’s budget accounts.

Agencies’ planning structures were similarly complex, comprised of widely
varying numbers and layers of goals. The Results Act and OMB guidance
give agencies flexibility in structuring strategic and annual performance
goals. Annual performance goals are expected to measure progress in
achieving longer term strategic goals. There is no required format or
structure and no limitation on the number of goals that can be included in a
performance plan. Our sample of performance plans presented a variety of
planning structures and many different terms to describe those structures.
Twenty-one of the 35 agencies we reviewed used a cascading hierarchy of
goals to put their fiscal year 1999 performance in context—that is, these
plans placed one or more layers of goals between their strategic and annual
performance goals. For example, the Bureau of Land Management (BLM)
plan contained a complex hierarchy of goals: 5 “General Goals,” supported
by 17 “Strategic Goals,” that were supported by 43 “Performance Goals,”
47 “Long-Term Goals,” and, finally, 64 fiscal year 1999 “Annual Goals.”
Figure 2 shows the numbers and layers of goals associated with one of
BLM’s five general goals. The remaining 14 agencies, however, had fewer
layers of goals. For example, the U.S. Agency for International
Development’s (USAID) six strategic goals were supported directly by
performance goals with fiscal year 1999 targets. Figure 3 presents the
performance goals associated with one of USAID’s strategic goals.




Page 9                                GAO/AIMD/GGD-99-67 Performance Budgeting
                                         B-282035




Figure 2: BLM’s Fiscal Year 1999 Performance Plan Presents Many Goal Layers


                                           BLM mission: To sustain the health, diversity, and productivity of the
                                         public lands for the use and enjoyment of present and future generations




                                       1. Serve current
  General goals                                                     2            3               4            5            6
                                      and future publics




                                      1.01. Provide opportunities for
  Strategic goals                      environmentally responsible              1.02            1.03         1.04          1.05
                                                recreation




                                      1.01.01. Manage outdoor recreation
  Performance goals                 activities to achieve and maintain public               1.01.02     1.01.03          1.01.04
                                               land health standards




                                    1.01.01.01. Encourage outdoor recreational users to
  Long-Term goals                    be better stewards for protecting BLM public lands                     1.01.01.02
                                       and waters by using multimedia environmental
                                          education and interpretative information




                                      1. By Sept. 30, 1999, increase
  Annual goals                                                                         2                3                  4
                                    number of recreation sites in “good
                                    condition” by 20 percent over 1997

                                         Source: GAO analysis based on BLM’s fiscal year 1999 performance plan.




                                         Page 10                                           GAO/AIMD/GGD-99-67 Performance Budgeting
                                          B-282035




Figure 3: USAID’s Fiscal Year 1999 Performance Plan Links Strategic Goals Directly to Performance Goals

                             USAID mission: To contribute to the U.S. national interests by supporting the people of
                             developing and transitional countries in their efforts to achieve enduring economic and
                                 social progress and to participate more fully in resolving the problems of their
                                                            countries and the world.




                             1. Broad-based economic
Strategic goals                                                           2                                4               5                  6
                              growth and agricultural                                     3
                             development encouraged




                            1. Average annual growth
Performance goals
                              rates in real per capita                        2                 3                    4                    5
                             income above 1 percent



                                          Source: GAO analysis based on USAID’s fiscal year 1999 performance plan.


                                          To analyze these disparate planning structures, we developed and used a
                                          common framework that defined two layers of goals above the annual
                                          performance goals stated in the performance plans: (1) strategic goals, to
                                          reflect the first goal layer under an agency’s mission statement and (2)
                                          strategic objectives, to reflect the next subordinate level of goals under
                                          strategic goals.18 We found that, quantifying just these two layers, most
                                          agency plans involved relatively complex presentations. The number of
                                          strategic goals in plans we reviewed ranged from 1 to 47, with a median of
                                          5. Just over half of the agencies in our review placed an intervening layer
                                          of strategic objectives between strategic and performance goals. In plans


                                          18
                                            For our definitions of strategic goal and strategic objective, see characteristics six and seven in table
                                          I.2 of appendix I.




                                          Page 11                                              GAO/AIMD/GGD-99-67 Performance Budgeting
                           B-282035




                           where strategic objectives were used, the number of strategic objectives
                           ranged from 5 to 122. Across our sample of agency plans, the median
                           number of strategic objectives used was nine.


Variety of Approaches to   The variety and complexity of agencies’ planning and budget structures
Linking Budget and         necessarily resulted in a range of approaches to present and link this
                           information. However, across this range, two approaches predominated:
Planning Structures
                           29 performance plans were presented separately from congressional
                           budget requests, and 22 plans established complex linkages of multiple
                           program activities related to multiple performance goals.

                           Although performance plans and budget requests—commonly referred to
                           as justifications of estimates—are both transmitted to the Congress
                           following the President’s budget submission in February, most agencies
                           kept their plans physically separated from their budget submissions, either
                           as entirely separate documents or as separate components appended to the
                           justifications.19 Only 6 of the 35 plans we reviewed had been fully
                           integrated into the agency’s budget justification (see appendix II).

                           In addition to separating plans and budgets, almost all agencies retained
                           the budget structure they had used in previous budget submissions. Only
                           three agencies in our review—EPA, the Customs Service, and the Nuclear
                           Regulatory Commission (NRC)—substantially changed their program
                           activity structures, and all cited the Results Act as a factor in this change.
                           Some others—such as the Bureau of Indian Affairs (BIA) and the
                           Department of Veterans Affairs (VA)—noted that such changes were under
                           consideration but proposed no changes in fiscal year 1999. For example,
                           BIA stated that it would work with OMB and congressional committees
                           to “simplify the budget format to mirror the strategic plan.”20 However,
                           agencies very frequently took advantage of the act’s flexibility to
                           modify—that is, to aggregate, consolidate, or disaggregate—program
                           activities in order to show linkages with performance goals. The extent to
                           which this flexibility was used varied greatly. For example, figure 4
                           illustrates how the Administration for Children and Families (ACF)
                           consolidated program activities before relating them to performance goals.



                           19Seeappendix I for a discussion of how we categorized the degree of integration between agencies’
                           budget justifications and performance plans.

                           20
                                BIA fiscal year 1999 performance plan, p. 2.




                           Page 12                                             GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 5 illustrates how the Health Resources and Services Administration
(HRSA) linked some performance goals to disaggregated program
activities.



Figure 4: The Administration for Children and Families Uses Consolidation to Link
Program Activities to Strategic Objectives


        $&) EXGJHW DFFRXQWV DQG                        $&) VWUDWHJLF JRDOV VWUDWHJLF
             SURJUDP DFWLYLWLHV                      REMHFWLYHV DQG SHUIRUPDQFH JRDOV




   Family support payments to states                Strategic goal:
   account                                          Increase economic independence and
       1. State child support administrative        productivity for families
       costs ($2,749)
       2. Federal incentive/ hold harmless            Strategic objective:
       payments to states ($469)
                                                      Increase parental responsibility ($3,257)
       3. Access and visitation grants ($10)
       4. Payments to territories                     Performance goals:
                                                        • Increase the paternity establishment percentage
       5. Repatriation                                  among children born out-of-wedlock to 96 percent
       6. Aid to families with dependent children       • Increase the percentage of cases having child
       benefit payments                                 support orders to 74 percent
       7. Emergency assistance                          • Increase the collection rate for current support
                                                        to 70 percent
   Children’s research and technical                    • Increase the percentage of paying cases among
   assistance account                                   arrearage cases to 46 percent
                                                        • Increase the cost-effectiveness ratio (total dollars
       1. Federal parent locator service ($30)
                                                        collected per dollar of expenditures) to $5.00
       2. Training and technical assistance
       3. Child welfare study
       4. Welfare research
       5. Evaluation of welfare to work
       6. Evaluation of abstinence education


Note: Dollars in millions. Numbers may not total due to rounding.
Source: GAO analysis based on Administration for Children and Families fiscal year 1999 performance
plan and Budget of the United States Government Fiscal Year 1999—Appendix.




Page 13                                             GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 5: The Health Resources and Services Administration Uses Disaggregation
to Link Program Activities to Sets of Performance Goals



          +56$ EXGJHW DFFRXQWV                             +56$ VWUDWHJLF JRDOV DQG
          DQG SURJUDP DFWLYLWLHV                                SHUIRUPDQFH JRDOV



       Health resources and                         Strategic goals:
       services account                              1. Eliminate health care disparities
                                                     2. Eliminate barriers to care
                                                     3. Assure quality

       20 program activities including:
       HIV/AIDS                                     Performance goals:

                                                          • Increase the number of clients served
               Activity 1: AIDS emergency
                                                          by Title I grant programs by 7.5 percent
               relief grants ($490)
                                                          • 3 other performance goals


                                                         • Increase the number of clients receiving
               Activity 2: HIV care grants to            anti-retro viral therapy to 57,500
               states ($670)
                                                         • 5 other performance goals


                                                          • Increase by 5 percent the number of
               Activity 3: HIV early                      clients receiving primary care services
               intervention services ($86)
                                                          • 4 other performance goals



               4 other activities




Note: Dollars in millions.
Source: GAO analysis based on Health Resources and Services Administration’s fiscal year 1999
performance plan and Budget of the United States Government Fiscal Year 1999—Appendix.




Page 14                                         GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




In addition to modifying or proposing changes to program activities,
agencies in our review used three principal strategies to meet the Results
Act’s expectation that annual plans would “establish performance goals to
define the level of performance to be achieved by a program activity.” 21

• About half (13) of the agencies that made linkages22 between
  performance goals and program activities in their performance plans
  established this connection at the relatively high level of strategic goals
  or objectives. For example, as shown in figure 4, ACF linked
  consolidated program activities to strategic objectives, which, in turn,
  were subsequently associated with performance goals.
• About the same number of agencies (14) defined direct linkages to
  performance goals. Figure 5 illustrates how HRSA linked disaggregated
  program activities directly to its performance goals.
• Three agencies—the National Aeronautics and Space Administration,
  the National Science Foundation, and VA—linked program activities to
  something other than a statement of strategic goals, strategic objectives,
  or performance goals. For example, the VA plan contained 17 strategic
  goals and 25 strategic objectives. However, instead of linking program
  activities directly to these goals and objectives, VA linked both its
  program activities and performance goals to its 10 business lines. The
  business lines generally represented different agency functions such as
  medical care and education.

As the above discussion indicates, regardless of the strategy used, most
agencies ended with the same basic relationship—many program activities
were related to many performance goals (see appendix II). These
imprecise, “many-to-many” relationships frequently resulted from agencies’
linking aggregated or consolidated program activities with strategic goals
or other groupings of performance goals. For example, in figure 4, ACF
consolidated program activities and linked them to the group of
performance goals associated with a strategic objective, creating a many-
to-many relationship between program activities and performance goals.
In contrast, eight plans used a more direct and simple approach—typically
linking a single program activity with multiple performance goals. Figure 5


2131   U.S.C. 1115(a)(1).

22
  In addition, we could not determine linkages between program activities and performance goals from
information provided in the performance plans of the following 5 agencies: the Department of Defense,
the Immigration and Naturalization Service, the National Institutes of Health, the Rural Housing Ser-
vice, and the Social Security Administration.




Page 15                                           GAO/AIMD/GGD-99-67 Performance Budgeting
                          B-282035




                          shows that HRSA linked a single disaggregated program activity to multiple
                          performance goals.

                          While 30 of 35 plans defined some relationship between program activities
                          and performance goals, 16 of these did not build on that relationship by
                          showing how the funding from program activities would be allocated to
                          discrete sets of performance goals.

                          • Ten of these 16 plans did not present funding levels for any set of
                            performance goals in their performance plans.
                          • Six of these 16 plans presented funding levels for some set of
                            performance goals without explaining how those funding levels had
                            been derived from program activities in their budget requests.

                          However, 14 agency plans, or 40 percent of those included in our review,
                          both identified the funding for program activities and explained how that
                          funding would be allocated to a discrete set of performance goals.23 In
                          effect, these 14 plans, which will be discussed more fully in the next
                          section, took the first step in defining the performance consequences of
                          budget decisions.



Funding Was Allocated     Our review of selected fiscal year 1999 performance plans indicated that
                          agencies with budget and planning structures of widely varying complexity
to Performance Under      were able to develop approaches toward achieving a fundamental purpose
a Variety of Conditions   of the Results Act—clarifying the relationship between resources and
                          results. Figure 6 lists the 14 agencies in our review that allocated program
But Using Some            activity funding to performance goals in their performance plans. We
Common Approaches         found that some of the approaches that these agencies used, alone or in
                          combination, were more frequently associated with plans that linked
                          program activity funding to performance goals. For example, agencies that
                          (1) established simple relationships between program activities and
                          performance goals or (2) fully integrated budget justifications and
                          performance plans were significantly24 more likely to allocate program
                          activity funding to performance goals. In addition, each of the three


                          23The set of goals ranged in size and scope. For example, some of these plans presented allocations of
                          funding to strategic goals or objectives, which represented discrete sets of performance goals.

                          24
                            When “significant” is used in this report, we are referring to statistical significance at or around the 95
                          percent confidence level. This significance means that we can be 95 percent certain that the associa-
                          tion we find in our sample is not due to chance or random variation.




                          Page 16                                               GAO/AIMD/GGD-99-67 Performance Budgeting
                               B-282035




                               agencies that changed their budget structures to align them with their
                               planning structures made these allocations. Conversely, plans
                               characterized by imprecise, many-to-many relationships between
                               performance goals and program activities and plans presented separately
                               from budget justifications generally did not present such allocations.



                               Figure 6: Agencies Allocating Program Activity Funding to Performance Goals in
                               Fiscal Year 1999 Annual Performance Plans

                                 • Department of Energy              • Administration for Children and Families
                                 • Customs Service                   • Employment and Training Administration
                                 • Environmental Protection Agency   • Federal Bureau of Investigation
                                 • Food and Nutrition Service        • Health Resources and Services Administration
                                 • Internal Revenue Service          • National Aeronautics and Space Administration
                                 • Nuclear Regulatory Commission     • Office of Personnel Management
                                 • Small Business Administration     • U. S. Agency for International Development


                               Source: GAO analysis.




Allocations Were Made          Our review generally found few differences in the budget structures of
Within a Variety of Planning   agencies that allocated program activity funding to performance goals and
                               those that did not. For example, the number of an agency’s accounts or
and Budgeting Structures
                               program activities was not significantly related to whether a plan presented
But Alignment of These         funding allocations for performance goals. As table 1 shows, for example,
Structures Was Significant     the median number of accounts and program activities was nearly the same
                               for agencies that did allocate program activity funding to performance
                               goals and for those that did not.




                               Page 17                                 GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Table 1: Complexity of Budget Structure Was Not Significant in Allocating Funding
to Performance Goals
                                                        Low       High     Median
Agencies that allocated funding
     Number of accounts                                    2        33              7
     Number of program activities                          6       119         32
Agencies that did not allocate funding
     Number of accounts                                    1       118         10
     Number of program activities                          9       465         32

Source: GAO analysis.

We also considered whether plans that allocated program activity funding
to performance goals were more frequently associated with agencies
having spending concentrated in one account. To determine if
concentration of spending was a significant factor, we calculated the
number of agencies for which 75 percent or more of requested spending
was associated with a single account. Again, no significant relationship
was observed (see figure 7). Four, or 36 percent, of the agencies with
spending concentrated in one account allocated program activity funding
to performance goals. Ten, or 42 percent, of the agencies with spending
concentrated in multiple accounts allocated program activity funding to
performance goals.




Page 18                                  GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 7: Concentration of Requested Spending Was Not Significant in Allocating
Funding to Performance Goals
Number of agencies
                                              14
14


12

                                                        10
10


 8
             7

 6

                        4
 4


 2


 0
            co ted




                                               u n in
                t
              un
          ac ra




                                             co d
                                                  ts
                                           ac te
         e nt




                                         le tra
       on ce




                                      tip en
     in con




                                    ul c
                                 m con
      ng
    di




                                    ng
  en




                                 di
Sp




                              en
                            Sp




                     Did not allocate funding

                     Allocated funding

Source: GAO analysis.


Similarly, the complexity of an agency’s program activity structure was not
a significant factor in allocating funding to performance goals. We
considered whether a simpler program activity structure—that is, one in
which program activity titles were repeated across budget accounts—was
more frequently associated with allocating funding to performance goals.
We found no significant difference in the number of agencies allocating



Page 19                                                 GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




program activity funding to performance goals. Thirteen, or 39 percent of
the 33 agencies without common program activity structures, allocated
program activity funding to performance goals (see figure 8). Although
only two agencies—EPA and the Department of Defense—exhibited
common program activity structures, EPA presented such an allocation
and DOD did not.



Figure 8: Commonalities Within Agencies’ Program Activity Structures Were Not
Significant in Allocating Funding to Performance Goals
Number of agencies

             20
20
18

16
14                     13
12
10
 8

 6
 4

 2                                        1       1
 0
                                             ct m
                   ct m




                                           ru ra
                                                 e
                 ru ra
                       e




                                               ur
                                         st og
                     ur
               st og




                                      ity pr
            ity pr




                                   tiv on
         tiv on




                                 ac m
      ac m




                                     m
           m




                                 Co
        co
     Un




                       Did not allocate funding
                      Allocated funding

Source: GAO analysis.


Although a particular account and program activity structure was generally
not associated with allocating funding to performance goals, there was one
significant but perhaps unsurprising exception. All three of the agencies




Page 20                                               GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




that proposed changing their program activities to be consistent with their
planning structures—EPA, NRC, and the Customs Service—allocated
funding to performance goals (see figure 9). For example, EPA proposed a
uniform program activity structure across all of its accounts in which each
program activity represented one of its strategic goals. Figure 10 illustrates
how EPA used consolidation to allocate program activity funding to
strategic objectives and their supporting performance goals.



Figure 9: Alignment of Budget and Planning Structures Was Significant for
Allocating Funding to Performance Goals
Number of agencies
 25

                21

 20



 15

                          11

 10



  5                                                3

                                            0
  0
                ct ar




                                           ct d
                                         ru ge
              ru e
                    e




                                               e
            st al y
                  ur




                                             ur
                                       st an
                                         Ch
        98 isc
      19 d f
          e
       Us




                        Did not allocate funding
                        Allocated funding

Source: GAO analysis.




Page 21                                            GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 10: EPA Proposed Aligning Budget and Planning Structures


            (3$ EXGJHW DFFRXQWV DQG                                         (3$ VWUDWHJLF JRDOV
                   SURJUDP DFWLYLWLHV                                    VWUDWHJLF REMHFWLYHV DQG
                                                                             SHUIRUPDQFH JRDOV

      Science and technology account
             1. Clean air ($137)
                                                                       Strategic goal: clean air
                                                  $4
             2.   Clean water
             3.   Safe food                                            Strategic objective: acid rain ($22)
             4.   Preventing pollution
             5.   Waste management                                     Performance goals:
             6.   Global and cross border
                                                                            • Maintain 4 million tons of sulfur
             7.   Right to know
                                                                            dioxide reductions from utility sources
             8.   Sound science
             9.   Credible deterrent                                        • Maintain 300,000 tons of nitrogen
                                                                            oxides reductions from coal-fired utility
      Environmental programs and                                            sources
      management account                                                    • Launch the nitrogen oxides Emissions
                                                                            and Allowance Tracking System for the
             1. Clean air ($169)                  $13
                                                                            Ozone Transport Region
             • Other program activities corresponding to
             EPA’s other strategic goals (similar to above)

      State and tribal assistance grants
      account
             1. Clean air ($201)                   $5
             • Other program activities corresponding to
             EPA’s other strategic goals (similar to above)




Note: Dollars in millions.
Source: GAO analysis based on EPA’s fiscal year 1999 performance plan and Budget of the United
States Government Fiscal Year 1999—Appendix.


Fourteen agencies in our review that showed how program activities were
allocated to performance goals did not appear to have any common
structural elements in their performance plans. Differences in plan
complexity—defined in terms of number and layers of goals—were not
significant between agencies that allocated program activity funding to
performance goals and those that did not. As indicated in table 2, the
median number of strategic goals and objectives was the same or similar
between these two groups of plans.




Page 22                                                       GAO/AIMD/GGD-99-67 Performance Budgeting
                              B-282035




                              Table 2: Planning Structure Was Not a Significant Factor for Agencies That
                              Allocated Program Activity Funding to Performance Goals
                                                                                       Low        High     Median
                              Agencies that allocated funding
                                    Number of strategic goals                             3         10          5
                                    Number of strategic objectives                        0         55          7
                              Agencies that did not allocate funding
                                    Number of strategic goals                             1         47          5
                                    Number of strategic objectives                        0        122         10
                              Source: GAO analysis.




Two Approaches to Linking     Although the complexity of an agency’s budget and planning structures
Budget and Planning           generally was not significantly related to whether it linked program activity
                              funding to performance goals, two approaches to linking these structures
Structures Were Significant
                              were. These approaches were (1) establishing simpler relationships
                              between program activities and performance goals and (2) fully integrating
                              budget justifications and performance plans. Whether used alone or in
                              combination, these approaches were more frequently associated with
                              agencies that were able to show the performance consequences of budget
                              decisions.

                              A significant difference in allocating program activity funding to
                              performance goals existed between agencies in which the performance
                              plan was integrated with the agency’s budget justification and agencies in
                              which the performance plan was not integrated. Agencies whose plans
                              were fully integrated with the budget justification nearly always (five of six
                              such plans) allocated program activity funding to performance goals (see
                              figure 11). Conversely, about two-thirds, or 20 of the 29 agencies whose
                              plans could be physically separated from the budget justification, did not
                              allocate program activity funding to performance goals. For example,
                              figure 12 shows how information traditionally contained in a budget
                              justification, such as descriptions of accounts and their funding, was
                              combined with performance information in the Internal Revenue Service’s
                              (IRS) integrated budget justification and performance plan.




                              Page 23                                  GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 11: Integrating Budget and Planning Documents Was Significant for
Allocating Funding to Performance Goals
Number of agencies
           20
20
18
16
14
12
10                    9
 8
 6                                              5
 4
 2                                       1
 0
            dg le




                                        dg d
          bu rab




                                      bu rate
              et




                                          et
      th p a




                                  th e g
    m se




                               th int
        e
 fro lan




                                    e
                             wi lly
                                 fu
    P




                              an
                           Pl




                    Did not allocate funding
                    Allocated funding

Source: GAO analysis.




Page 24                                             GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 12: The Internal Revenue Service Integrates Its Budget Justification and
Performance Plan

                                    3URFHVVLQJ $VVLVWDQFH DQG
                                   0DQDJHPHQW $FFRXQW 

       Program activity: Functions: This activity provides for the salaries, benefits, and related
  submission processing costs to process tax returns and supplemental documents, account for tax
                 ($888) revenues, issue refunds and tax notices, develop and print tax returns and
                                   publications . . . . Also included are resources to: process information
                                   returns such as wage, dividend, and interest statements; provide for
                                   payment of refunds . . . identification of possible non-filers for investigation;
                                   and, provide tax returns for audits . . . .

         Performance goal #1:               Performance goal #2:               Performance goal #3:
       Improve customer service             Increase compliance                Increase productivity

       Performance measure:                Performance measure:                 Performance measure:
       Number of individual                211.8 million primary returns        19.5 percent of individual
       refunds issued will equal           processeda                           returns filed electronically
       93.3 milliona
                                                                                Performance measure:
       Performance measure:
                                                                                78.2 percent of dollars
       Refund timeliness--paper
                                                                                received electronically
       40 days

       5 other performance                                                      Performance measure:
       measures                                                                 70.9 percent dollars
                                                                                received via third party
                                                                                processors


aIRS   noted that this is a projection for budget purposes and is not used in the agency’s business review.
Note: Dollars in millions.
Source: GAO analysis based on the Internal Revenue Service’s fiscal year 1999 performance plan.


We also found that plans showing a simpler relationship between program
activities and performance goals were significantly more likely to show
how funding was allocated to performance goals (see figure 13). That is, in
all agencies in which the relationship between program activities and
performance goals could be characterized as one-to-many or many-to-one,
program activity funding was allocated to performance goals. For
example, figure 14 shows that the allocation of funding to performance
goals in the NRC plan was essentially automatic because each of the
agency’s program activities generally align with a strategic goal and its
supporting performance goals. However, where relationships were less
precise—that is, when multiple program activities were related to multiple
performance goals—allocations of program activity funding to
performance goals were less common.




Page 25                                                     GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 13: Simple Relationships Between Program Activities and Performance
Goals Were Significant for Allocating Funding to Performance Goals
Number of agencies

16          16


14

12

10

                                                      8
8

                       6
6

4

2

                                             0
0
                                             ns ple
             ns lex




                                                  p
                  p
          tio p




                                          tio m
                                               hi
               hi
        la m




                                        la Si
      re Co




                                      re




                      Did not allocate funding
                      Allocated funding

Source: GAO analysis.




Page 26                                               GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Figure 14: The Nuclear Regulatory Commission Aligns Budget and Planning
Structures to Create a Simple Relationship Between Program Activities and
Performance Goals

   15& EXGJHW DFFRXQWV DQG                               15& VWUDWHJLF JRDOV DQG
        SURJUDP DFWLYLWLHV                                    SHUIRUPDQFH JRDOV



   Salaries and expenses account               Strategic goal: nuclear reactor safety ($211)
    1. Nuclear reactor safety ($211)           Performance goals:
                                                     • Goal I.A - zero civilian nuclear reactor accidents
    2. Nuclear materials safety ($49)
                                                     • Goal I.A.1 - maintain low frequency of events which
                                                     could lead to a severe accident
    3. Nuclear waste safety
                                                     • Goal I.B - zero deaths due to radiation or
    4. Common defense and security and               radioactivity releases from civilian nuclear reactors
                                                     • Goal I.B.1 - zero significant radiation exposures due
       international involvement
                                                     to civilian nuclear reactors
    5. Protecting the environment
                                               Strategic goal: nuclear materials safety ($49)
    6. Management and support
                                               Performance goals:
                                                     • Goal II.A - zero radiation-related deaths due to
                                                     civilian use of source, byproduct, and special nuclear
                                                     materials
                                                     • Goal II.A.1 - no increase in the number of significant
                                                     radiation exposures due to loss or use of source,
                                                     byproduct, and special nuclear materials
                                                     • Goal II.A.1.a - no increase in the number of losses
                                                     of licensed material as reported to Congress annually
                                                     • Goal II.A.1.b - no accidental criticality involving
                                                     licensed material
                                                     • Goal II.A.2 - no increase in the number of
                                                     misadministration events which cause significant
                                                     radiation exposures


Note: Dollars in millions.
Source: GAO analysis based on Nuclear Regulatory Commission’s fiscal year 1999 performance plan
and Budget of the United States Government Fiscal Year 1999—Appendix.


As indicated by the agencies profiled in the figures above, approaches that
were individually significant in allocating funding were often used in
combination. In fact, 6 of the 14 plans that allocated program activity
funding to performance goals used two or more of the approaches
identified above. For example, of the three agencies that changed their
program activity structures, two either presented simple relationships
between program activities and performance goals or fully integrated their
performance plans and budget justifications.

The 21 plans in our review that did not allocate funding to discrete sets of
performance goals were also generally characterized by common features.
These plans (1) did not reflect any significant change in the agency’s
account or program activity structures, (2) generally were separable from
the justification of estimates, and (3) presented either no explicit



Page 27                                        GAO/AIMD/GGD-99-67 Performance Budgeting
                          B-282035




                          relationship or a many-to-many relationship between performance goals
                          and program activities. These features were a hallmark of plans that did
                          not inform users of the performance consequences of budget decisions.



First-Year Experiences    Our review of selected fiscal year 1999 performance plans presents a mixed
                          picture. Certainly, some agencies were able to develop informative
Demonstrate Progress      approaches to connect budgetary resources to results. These approaches
and Continued             are addressing some of the challenges that have plagued performance
                          budgeting efforts prior to the Results Act. They can also be seen as the first
Challenges                step toward achieving a key objective of the act—a clearer understanding
                          of what is being achieved for what is being spent. Paralleling these changes
                          is growth in the Congress’ interest in performance information in its
                          resource allocation and other oversight processes. Nevertheless, our
                          review, as well as the delay of the performance budgeting pilots required by
                          the act, indicates continuing challenges for achieving a clearer relationship
                          between budgetary resources and results.


A Year of Progress in     The fiscal year 1999 budget process marked an important beginning in
Connecting Resources to   more clearly showing the performance consequences of budget decisions.
                          As indicated in the previous section, executive branch agencies developed
Results
                          a variety of approaches to link their performance plans and budget
                          requests. But equally important, the Congress also showed an awareness
                          of Results Act implementation efforts and a clear interest in obtaining
                          credible performance information during its appropriations and oversight
                          processes.

                          Executive agencies have demonstrated that linking complex budget and
                          planning structures demands adaptive approaches. The scope of the
                          federal government’s missions, the variety of its organizational models, and
                          the breadth of its processes—all subject to a multifaceted congressional
                          oversight environment—suggest that many approaches will be developed
                          to more clearly allocate requested funding levels to performance goals.
                          Our review found that agencies reflecting the heterogeneity of the federal
                          government—from direct service agencies (e.g., IRS) to agencies
                          principally involved in grant or loan making (e.g., USAID) to regulatory
                          agencies (e.g., NRC)—began to link budgetary resources and results.

                          In the fiscal year 1999 performance plans, the agencies we reviewed
                          developed several approaches to overcome a common problem of previous
                          performance budgeting initiatives—planning structures and presentations


                          Page 28                                GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




that were unconnected to budget structures and presentations. These
approaches include:

• changing budget structures to more closely align with performance
  plans. Three agencies proposed new program activities within existing
  budget accounts to generally reflect the strategic goals of their
  performance plan. These proposals sometimes facilitated a relatively
  simple relationship between program activities and performance goals
  that helped make connections clear.
• integrating performance information with budget justifications. In some
  cases, this took the form of fully integrating the performance plan with
  the agency’s budget justification, as in the IRS, the Customs Service, and
  the Federal Bureau of Investigation. Where plans were not fully
  integrated with budget justifications, some agencies used the
  justification to provide more detailed information on goals contained in
  their performance plan. For example USAID’s fiscal year 1999 budget
  justification contained “strategic support objectives” and “special
  objectives.” These objectives appear to further describe and support the
  performance goals expressed in USAID’s separate performance plan.
• crosswalking performance plans with budget structures. For example,
  ACF devised a crosswalk to identify the contribution of its over 60
  program activities to its 10 strategic objectives (see figure 4 for an
  excerpt from this crosswalk). The crosswalk identifies funding for
  accounts or program activities and—using consolidation and
  aggregation—relates an account or program activity to a strategic
  objective, and consequently, to the objective’s set of discrete
  performance goals. Account or program activity funding levels are
  summed to provide a proposed funding level for each strategic
  objective.

As executive agencies developed these approaches and presented their
fiscal year 1999 budget submissions, the Congress also indicated an
increasing interest in credible performance information to inform the
resource allocation process.25 We reviewed fiscal year 1999 appropriations
hearings and reports for the agencies in our review that allocated program


25The  Congressional Research Service recently documented the growth of congressional interest in
performance information in its memorandum, “Performance Measure Provisions in the 105 th Congress:
Analysis of a Selected Compilation,” dated December 17, 1998. This analysis concluded “while the
number of public laws with performance measure provisions nearly doubled from the 104th to the 105th
Congresses, the number of committee reports containing performance measure provisions. . .nearly
tripled.”




Page 29                                          GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




activity funding to performance goals and observed that members of the
Congress often made specific reference to the performance information
contained in the agency’s justification and/or performance plan. Some
notable examples include the following.

• ACF officials were questioned as to whether a 4 percent increase in
  children exiting foster care through reunification justified the
  appropriation being sought for these activities. 26
• U.S. Customs Service officials were asked how the appropriations
  committee should evaluate performance and resource requirements for
  Customs’ marine mission, given an apparent lack of measures for its
  marine enforcement program.27
• NRC officials were asked what performance measures would be used to
  justify U.S. participation and funding in international nuclear safety
  programs and how requested budget increases were related to NRC’s
  mission.28
• Food and Nutrition Service officials presented data on the number of
  meals being served in the school lunch and breakfast programs and
  were asked how much additional budgetary resources would be needed
  to serve all eligible children.29
• The Conference Report on HUD’s Fiscal Year 1999 Appropriations
  directed the agency to revise its performance plan to incorporate
  measurable goals and outcomes for providing housing vouchers and
  certificates to assist families in transitioning from welfare to work.30
• A House Subcommittee on Appropriations was unwilling to recommend
  funding for a request for community-based technology centers in part


26
  Departments of Labor, Health and Human Services, Education and Related Agencies for Fiscal Year
1999, Hearings before a Subcommittee of the Committee on Appropriations, House of Representatives,
105th Congress, 2d Sess., Pt. 2, p. 776.

27Treasury, Postal Service, and General Government Appropriations for Fiscal Year 1999, Hearings
before a Subcommittee of the Committee on Appropriations, House of Representatives, 105th Congress,
2d Sess., Pt. 1, p. 490.

28
  Energy and Water Development Appropriations for Fiscal Year1999, Hearings before a Subcommittee
of the Committee on Appropriations, House of Representatives, 105th Congress, 2d Sess., Pt. 5, p. 1842.

29
 Agriculture, Rural Development, Food and Drug Administration, and Related Agencies
Appropriations for Fiscal Year 1999, Hearings before a Subcommittee of the Committee on
Appropriations, House of Representatives, 105th Congress, 2d Sess., Pt. 6, p. 399.
30
 Making Appropriations for the Department of Veterans Affairs and Housing and Urban
Development. . .for the Fiscal Year Ending September 30, 1999, Conference Report, Committee on
Appropriations, House of Representatives, H. Rpt. No. 769, 105 th Cong. 2d Sess. p. 237 (1998).




Page 30                                            GAO/AIMD/GGD-99-67 Performance Budgeting
                           B-282035




                             because specific performance measures for this new program were not
                             presented.31
                           • The Treasury and General Government Appropriations Act of 1999
                             stated that the Office of National Drug Control Policy could not obligate
                             funds provided to continue its national media campaign until it
                             submitted the evaluation and results of the campaign.32


Challenges Facing Future   Deliberations on agencies’ appropriations also indicate that making
Performance Budgeting      effective linkages between budget program activities and performance
                           goals is one of many challenges that need to be addressed for performance
Efforts
                           information to be used in the budget process. Members of the Congress
                           also questioned agencies about why goals were not more results-oriented
                           and what steps were being taken to coordinate activities with other
                           agencies. For example, Office of Personnel Management officials were
                           asked if the agency’s performance plan could identify measures to help
                           determine the agency’s progress toward a result of recruiting and retaining
                           the federal workforce required for the 21st century. Members of the
                           Congress were also concerned about agencies’ use of program evaluation
                           and other techniques to ensure the validity and reliability of performance
                           data. ACF officials were asked about their approach for evaluating the
                           academic success of Head Start preschool participants after those students
                           leave the program.

                           These concerns demonstrate that agencies also need to adopt a broader
                           agenda for improving performance plans that includes focusing on results,
                           defining clear strategies, and improving their capacity to gather and use
                           performance data. Translating the use of agency resources into concrete
                           and measurable results will be a continual challenge that will require both
                           time and effort on the part of the agency. The uneven pace of progress
                           across government is not surprising; agencies are in the early years of
                           undertaking the changes that performance-based management entails.
                           Although some agencies, as indicated in our review, began to show the
                           performance consequences of budget decisions, improvements can be
                           made in the clarity and completeness of linkages between program
                           activities and performance goals. Agencies must also balance the scope


                           31
                             Departments of Labor, Health and Human Services, and Education and Related Agencies
                           Appropriation Bill, 1999, Committee on Appropriations, House of Representatives, H. Rpt. No. 635,
                           105th Cong. 2nd Sess. p. 145 (1998).

                           32
                                H.R. Conf. Rpt. No. 105-825 at 515 (1998).




                           Page 31                                           GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




and precision of funding estimates for performance goals with the
usefulness of such estimates for resource allocation decisions. In addition,
we believe weaknesses in the performance measurement systems
described in agencies’ fiscal year 1999 performance plans need to be
addressed.33

Linkages between plans and budgets must be supported by results-oriented
and credible performance data to be useful. Our assessment34 of agencies’
fiscal year 1999 performance plans found most goals were focused on
outputs, not outcomes. This presents a dilemma for future performance
budgeting efforts. While outcome information is clearly useful for
measuring performance, it may be more difficult to allocate funding to
outcomes that are far removed from the inputs that drive costs. Allocating
funding to outcomes presumes that inputs, outputs, and outcomes can be
clearly defined and definitively linked. For some agencies, these linkages
are unclear or unknown.35 For example, agencies that work with state or
local governments to achieve performance may have difficulty specifying
how each of multiple agencies’ funding contributes to an outcome.

In addition to understanding how actions affect outcomes, allocating
funding to outcomes also requires an ability to understand how costs are
related to outcomes. Agencies are noting the importance of cost
accounting and other management systems for success in allocating
funding to performance.36 For example, the Department of Housing and
Urban Development’s (HUD) fiscal year 1999 performance plan
acknowledged that the agency “has no mechanism for tracking resources
as they are applied to performance measures.”37 The plan noted that HUD


33
 For additional discussion, see Major Management Challenges and Program Risks: A Governmentwide
Perspective (GAO/OCG-99-1, January 1999), GAO/GGD/AIMD-98-228, September 8, 1998, and
GAO/GGD/AIMD-99-69, February 26, 1999.

34
     See GGD/AIMD-98-228, September 8, 1998.
35
  Program evaluation is critical to understanding and isolating an agency’s impact on outcomes. For a
discussion of performance measurement challenges, see Managing for Results: Measuring Program
Results That Are Under Limited Federal Control (GAO/GGD-99-16, December 11, 1998), Program
Evaluation: Agencies Challenged by New Demand for Information on Program Results
(GAO/GGD-98-53, April 24, 1998), and Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997).

36Our recent Performance and Accountability series cited developing cost accounting systems as a key
challenge in supporting the Results Act. See GAO/OCG-99-1, January 1999.

37
     HUD fiscal year 1999 performance plan, p. 6.




Page 32                                             GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




intends to develop a system that “will allow the Department to identify,
justify, and match resource requirements for effective and efficient
program administration and management.”38

Agencies are expected to develop such systems as they implement
managerial cost accounting standards developed by the Federal
Accounting Standards Advisory Board (FASAB).39 These standards
require that agencies develop and implement cost accounting systems that
can be used to relate the full costs of various programs and activities to
performance outputs. Although these standards were originally to become
effective for fiscal year 1997, the Chief Financial Officers (CFO)
Council—an interagency council of the CFOs of major agencies—
requested the effective date be delayed for 2 years due to shortfalls in cost
accounting systems. As FASAB recommended, the effective date was
extended by 1 year, to fiscal year 1998, with a clear expectation that there
would be no further delays. However, developing the necessary approach
to gather and analyze needed program and activity-level cost information
will be a substantial undertaking. While there is a broad recognition of the
importance of doing so, for the most part agencies have just begun this
effort.

As discussed earlier in this report, agencies’ difficulties in developing
performance planning and measurement and cost accounting systems were
cited by OMB in its 1997 decision to delay the performance budgeting pilots
required by the Results Act. To further discussion of those pilots, OMB
recently suggested possible formats and time frames for the pilots in a
September 1998 paper sent to federal agencies. In that discussion paper,
OMB noted that pilot projects would not be designated unless they could
“fairly test the [Results Act’s] concept of performance budgeting,” which it
described as “the application of multi-variate or optimization analysis to
budgeting.” The paper described three analytical alternatives that could be
tested involving performance tradeoffs (1) in the same program with
changes in program funding, (2) in the same program with no change in


38HUD   fiscal year 1999 performance plan, pp. 6-7.

39
  In October 1990, the nine member FASAB was established by the Secretary of Treasury, the Director
of the OMB, and the Comptroller General of the United States to consider and recommend accounting
standards to address the financial and budgetary information needs of the Congress, executive agen-
cies, and other users of federal financial information. Once FASAB recommends accounting standards,
the Secretary of the Treasury, the Director of OMB and the Comptroller General decide whether to
adopt the recommended standards. If they are adopted, the standards are published as Statements of
Federal Financial Accounting Standards by OMB and GAO.




Page 33                                               GAO/AIMD/GGD-99-67 Performance Budgeting
              B-282035




              total program funding, or (3) in several programs with shifts in intra-agency
              funding between these programs.

              At present, OMB has no definitive plans for proceeding with the
              performance budgeting pilots. OMB solicited agencies’ comments on the
              discussion paper and on their capability to produce the alternative budgets
              suggested in the committee report accompanying the Results Act.
              According to OMB, no agency has as yet volunteered to participate in the
              pilots. In its discussion paper, OMB stated that “the absence of designated
              pilots or having fewer designations than required would be an indication of
              agency readiness to do performance budgeting, and would be discussed in
              the OMB report to Congress.”

              These developments reflect some of the broader tensions involved in
              linking planning and budgeting structures. On one hand, performance plans
              need to be broad and wide-ranging if they are to articulate the missions and
              outcomes that agencies seek to influence. Often these plans will include
              goals that the agency can only influence indirectly because of
              responsibilities assigned to other actors, such as state and local
              governments. On the other hand, budget structures have evolved to help
              the Congress control and monitor agency activities and spending and, as
              such, are geared more to fostering accountability for inputs and outputs
              within the control of agencies.

              Performance budgeting poses the daunting task to agencies of discovering
              ways to address these competing values that are mutually reinforcing, not
              mutually exclusive. Strategies for bringing planning and budgeting
              structures together must balance both sets of values. For example, some
              agencies might let their planning structures be the starting point for making
              connections and seek to crosswalk broad overarching goals to their many
              program activities. This approach is consistent with a results orientation
              but can obscure the impact of specific funding decisions. Other agencies
              might decide to use the budget structure as their starting point and
              integrate performance information into formats familiar and useful for
              congressional oversight. Although this approach may be helpful in
              focusing on the performance consequences of budget decisions, such
              strategies risk confining performance information to structures that may be
              too limited to fully address the broad mission and outcomes of the agency.



Conclusions   The fiscal year 1999 annual performance plans—the first called for under
              the Results Act—showed potential for providing valuable information that



              Page 34                               GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




can be used to better program performance. However, the linkage between
requested funding and performance goals is just one of many elements that
need improvement for these plans to be useful for improving program
performance. Top management within agencies must provide the
consistent leadership necessary to direct the needed management changes
and to ensure momentum is maintained. Ultimately, performance-based
management should become an integral part of an agency’s culture. The
transition process must include proven “change management” approaches
to be successful and sustained. In addition, congressional use of results-
oriented program performance and cost information in its decision-making
about federal policies and programs will also spur agencies’ efforts to
implement the statutory framework by sending the unmistakable message
to the agencies that the Congress remains serious about performance-
based management and accountability.

As we have noted in a previous report, one of the Results Act’s most
conspicuous and useful features is its reliance on experimentation.40 This
is certainly true regarding performance budgeting. The act calls for one
form of performance budgeting by requiring that performance goals from
an agency’s annual performance plan cover the program activities of the
agency’s budget request while giving agencies flexibility in how this linkage
is made. This requirement, when coupled with OMB’s guidance that plans
reflect the funding levels being applied to achieve performance goals,
constitutes the first governmentwide expectation to directly associate
budgetary resources with expected results. In addition, the act also
requires pilot projects to test a specific form of performance budgeting that
presents varying levels of funding for varying levels of performance. The
committee report accompanying the act noted that “this pilot approach is
best because, while performance budgeting promises to link program
performance information with specific budget requests, it is unclear how
best to present that information and what the results will be.”41

The fiscal year 1999 performance planning and budgeting cycle produced a
useful experimentation in connecting planning and budgeting structures
that accommodated unique federal missions and structures. Some, but not
all of the agencies we reviewed, began to develop useful linkages.
Moreover, challenges in performance planning and measurement and


40GAO/AIMD-97-46,       March 27, 1997.

41
     S. Rpt. No. 103-58, p. 19.




Page 35                                   GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




deficiencies in cost accounting systems continue to confront federal
agencies. OMB has already cited these problems as the reasons why
performance budgeting pilot projects were not being initiated. The
progress that has been made and the challenges that persist underscore the
importance of developing a specific agenda to ensure continued progress in
better showing the performance consequences of budgetary decisions.42

The original goal for the act’s performance budgeting pilot projects was
twofold: to allow OMB and agencies to develop experience and capabilities
towards realizing the potential of performance budgeting, and to provide
OMB with a basis for reporting to the Congress on next steps and needed
changes. Although OMB stated in 1997 that agencies’ financial
management systems were not capable of the specific form of performance
budgeting called for in the act, our review demonstrates that some agencies
were able to develop approaches to make perhaps a more basic, but still
useful, connection between proposed spending and performance. These
fiscal year 1999 efforts to link performance goals and program activity
funding essentially constitute a first step toward achieving the intent of the
performance budgeting pilots. They also provide a baseline from which
OMB could assess future progress and determine what changes, if any, may
be needed to the act and federal budget processes.

In addition to its longstanding responsibilities regarding the formulation,
review, and presentation of the President’s annual budget requests, OMB is
the lead agency for overseeing a framework of recently enacted financial,
information resources, and performance planning and measurement
reforms designed to improve the effectiveness and responsiveness of
federal agencies.43 As such, OMB is well-situated to assess (1) the
practicality of performance budgeting pilots as currently defined in the law,
(2) agency approaches and continuing challenges to linking budgetary
resources and performance goals, and (3) options to encourage progress in
subsequent planning and budgeting cycles.




42InManaging for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance
Plans (GAO/GGD/AIMD-98-228, September 8, 1998), we recommended that OMB develop an agenda to
address five key opportunities for improving performance plans, including showing the performance
consequences of budgetary decisions.

43
  The Results Act: Observations on the Office of Management and Budget’s July 1997 Draft Strategic
Plan (GAO/AIMD/GGD-97-169R, August 21, 1997).




Page 36                                           GAO/AIMD/GGD-99-67 Performance Budgeting
                    B-282035




Recommendation to   In light of the indefinite delay of the performance budgeting pilots required
                    by the Results Act and the experiences of agencies during the fiscal year
OMB                 1999 performance planning and budgeting cycle, we recommend that the
                    Director of OMB assess the approaches agencies use to link performance
                    goals and program activities in the fiscal year 2000 performance plans.
                    OMB’s analysis, building on our review of fiscal year 1999 performance
                    plans, should develop a better understanding of promising approaches and
                    remaining challenges with respect to the concept of performance
                    budgeting within the federal government. OMB’s analysis should address,
                    for example:

                    • the extent of agencies’ progress in associating funding with specific or
                      sets of performance goals,
                    • how linkages between budgetary resources and results can be made
                      more useful to the Congress and to OMB,
                    • what types of pilot projects might be practical and beneficial, and
                    • when and how those pilot projects would take place.

                    On the basis of this analysis, we recommend that OMB work with agencies
                    and the Congress to develop a constructive and practical agenda to further
                    clarify the relationship between budgetary resources and results, beginning
                    with specific guidance for the preparation of agencies’ fiscal year 2001
                    plans. We further recommend that this analysis and the resulting agenda
                    become the foundation for OMB’s report to the Congress in March 2001, as
                    currently required by the Results Act, on the feasibility and advisability of
                    including a performance budget as part of the President’s budget and on
                    any other needed changes to the requirements of the act.



Agency Comments     On February 10, 1999, we met with the Deputy Director for Management
                    and other OMB officials to discuss this report and our recommendations;
                    on February 19, 1999, we provided a draft of this report to OMB for
                    comment. At both our meeting and subsequently, OMB provided technical
                    comments orally, which we have incorporated as appropriate. On March
                    26, 1999, OMB informed us they would have no written comments on this
                    draft.


                    As agreed with your office, unless you publicly announce its contents
                    earlier, we plan no further distribution of this report until 30 days from the
                    date of this letter. At that time, we will send copies to Senator Joseph



                    Page 37                                GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Lieberman, the Ranking Minority Member of your committee; other
appropriate congressional committees; and The Honorable Jacob Lew,
Director, Office of Management and Budget. We will also make copies
available to others on request.

Major contributors to this report are listed in appendix III. Please contact
me on (202) 512-9573 if you or your staff have any questions.

Sincerely yours,




Paul L. Posner
Director, Budget Issues




Page 38                               GAO/AIMD/GGD-99-67 Performance Budgeting
B-282035




Page 39    GAO/AIMD/GGD-99-67 Performance Budgeting
Contents


Letter                                                                                             1


Appendix I                                                                                        42
Objectives, Scope, and
Methodology

Appendix II                                                                                       55
Summary of Our
Analysis

Appendix III                                                                                      57
Major Contributors to
This Report

Tables                   Table 1: Complexity of Budget Structure Was Not Significant in
                           Allocating Funding to Performance Goals                                18
                         Table 2: Planning Structure Was Not a Significant Factor for
                           Agencies That Allocated Program Activity Funding to
                           Performance Goals                                                      23
                         Table I.1: Plans Reviewed                                                43
                         Table I.2: Characteristics and Associated Classification Frameworks      45


Figures                  Figure 1: Using the Results Act’s Flexibility to Align the Annual
                           Performance Plan With the Budget Account and Program Activity
                           Structure                                                               6
                         Figure 2: BLM’s Fiscal Year 1999 Performance Plan Presents
                           Many Goal Layers                                                       10
                         Figure 3: USAID’s Fiscal Year 1999 Performance Plan Links
                           Strategic Goals Directly to Performance Goals                          11
                         Figure 4: The Administration for Children and Families Uses
                           Consolidation to Link Program Activities to Strategic Objectives       13
                         Figure 5: The Health Resources and Services Administration Uses
                           Disaggregation to Link Program Activities to Sets of
                           Performance Goals                                                      14
                         Figure 6: Agencies Allocating Program Activity Funding to
                           Performance Goals in Fiscal Year 1999 Annual Performance Plans         17



                         Page 40                             GAO/AIMD/GGD-99-67 Performance Budgeting
Contents




Figure 7: Concentration of Requested Spending Was Not Significant
  in Allocating Funding to Performance Goals                            19
Figure 8: Commonalities Within Agencies’ Program Activity
  Structures Were Not Significant in Allocating Funding to
  Performance Goals                                                     20
Figure 9: Alignment of Budget and Planning Structures Was
  Significant for Allocating Funding to Performance Goals               21
Figure 10: EPA Proposed Aligning Budget and Planning Structures         22
Figure 11: Integrating Budget and Planning Documents Was
  Significant for Allocating Funding to Performance Goals               24
Figure 12: The Internal Revenue Service Integrates Its Budget
  Justification and Performance Plan                                    25
Figure 13: Simple Relationships Between Program Activities and
  Performance Goals Were Significant for Allocating Funding to
  Performance Goals                                                     26
Figure 14: The Nuclear Regulatory Commission Aligns Budget
  and Planning Structures to Create a Simple Relationship Between
  Program Activities and Performance Goals                              27
Figure I.1: Illustration of Characteristic 9                            48
Figure I.2: Illustration of Characteristic 10                           50




Abbreviations

ACF        Administration for Children and Families
BIA        Bureau of Indian Affairs
BLM        Bureau of Land Management
CFO        Chief Financial Officer
EPA        Environmental Protection Agency
HRSA       Health Resources and Services Administration
HUD        Department of Housing and Urban Development
IRS        Internal Revenue Service
NRC        Nuclear Regulatory Commission
OMB        Office of Management and Budget
USAID      U.S. Agency for International Development
VA         Department of Veterans Affairs




Page 41                            GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I

Objectives, Scope, and Methodology                                                                                    AppIexndi




              Our September 1998 report on agencies’ first performance plans
              establishes an agenda for improving several elements of agency plans,
              including showing the performance consequences of budget decisions.
              Following the issuance of that report, the Chairman of the Senate

              Committee on Governmental Affairs asked us to examine in more detail
              how the plans linked expected performance with budget requests. To do
              this, our objectives were to

              • describe agencies’ approaches to linking performance goals and
                budgetary resources,
              • examine characteristics that might be associated with different
                approaches to linkage, and
              • identify implications for future efforts to clarify the relationship
                between budgetary resources and results.



Scope         To address our objectives, we selected 35 fiscal year 1999 performance
              plans for review from departments and agencies1 covered by the CFO Act.
              We generally focused on bureau-level plans for each department but limited
              our review to the three largest bureaus with discretionary spending over
              $1 billion and/or the two largest bureaus.2 Table I.1 lists the agencies
              whose plans we reviewed.




              1In this report, we refer to a performance plan, whether of a department, agency, or bureau, as an
              “agency plan.”
              2
                Discretionary spending levels were used as an indication of the bureau's relevancy to appropriators
              since discretionary funding is affected by appropriations actions.




              Page 42                                            GAO/AIMD/GGD-99-67 Performance Budgeting
                  Appendix I
                  Objectives, Scope, and Methodology




                  Table I.1: Plans Reviewed
                    1. Administration for Children and Families (Department of Health and Human
                           Services)
                    2. Bureau of Indian Affairs (Department of Interior)
                    3. Bureau of Land Management (Department of Interior)
                    4. Department of Commerce
                    5. Customs Service (Department of the Treasury)
                    6. Department of Defense
                    7. Department of Education
                    8. Employment and Training Administration (Department of Labor)
                    9. Department of Energy
                   10. Environmental Protection Agency
                   11. Federal Aviation Administration (Department of Transportation)
                   12. Federal Bureau of Investigation (Department of Justice)
                   13. Federal Emergency Management Agency
                   14. Federal Highway Administration (Department of Transportation)
                   15. Federal Prison System (Department of Justice)
                   16. Food and Nutrition Service (Department of Agriculture)
                   17. Forest Service (Department of Agriculture)
                   18. General Services Administration
                   19. Health Resources and Services Administration (Department of Health and Human
                           Services)
                   20. Department of Housing and Urban Development
                   21. Immigration and Naturalization Service (Department of Justice)
                   22. Internal Revenue Service (Department of the Treasury)
                   23. National Aeronautics and Space Administration
                   24. National Institutes of Health (Department of Health and Human Services)
                   25. National Park Service (Department of Interior)
                   26. National Science Foundation
                   27. Nuclear Regulatory Commission
                   28. Occupational Safety and Health Administration (Department of Labor)
                   29. Office of Personnel Management
                   30. Rural Housing Service (Department of Agriculture)
                   31. Small Business Administration
                   32. Social Security Administration
                   33. Department of State
                   34. U.S. Agency for International Development
                   35. Department of Veterans Affairs




Data Collection   To describe agencies’ approaches to linking performance goals and
                  budgetary resources, we identified 12 characteristics that could be used to
                  describe agencies’ planning and budgeting structures and the linkages
                  between them. For each characteristic, we developed a classification
                  framework for differentiating between plans based on that characteristic.
                  These classification frameworks involved either straightforward counts of
                  plan components (e.g., number of strategic goals) or judgments based on
                  the content and structure of the plan. One staff member reviewed each plan
                  and classified the plan on each characteristic. To ensure consistency in


                  Page 43                                 GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




judgments, another staff member also independently reviewed the plans
and the assessment on each characteristic. Differences in judgments were
addressed by having staff members jointly reevaluate the coding of the
characteristic to resolve the difference. We then compiled our assessments
on plan characteristics into a database that was used to profile agencies’
first-year approaches to linking budgetary resources with results.

These characteristics generally fell into two groups: characteristics
describing agencies’ budget and planning structures (numbers 1 through 7
in table I.2), and characteristics describing agencies’ approaches to linking
these structures (numbers 8 through 12 in table I.2). Table I.2 presents
additional detail on the characteristics used in this review.




Page 44                               GAO/AIMD/GGD-99-67 Performance Budgeting
                                               Appendix I
                                               Objectives, Scope, and Methodology




Table I.2: Characteristics and Associated Classification Frameworks
Characteristic                            Description
 A. Characteristics describing budget and planning structures
(1) Number of accounts                The number of budget accounts in the Appendix of the Budget of the United States
                                      Government, Fiscal Year 1999 from which agencies proposed to make obligations in fiscal year
                                      1999.
(2) Concentration of spending             a) Single account—Agencies were classified as having a single account if they proposed to
                                          make 75 percent or more of their proposed fiscal year 1999 obligations from one budget
                                          account.

                                          b) Multiple accounts—Agencies were classified as having multiple accounts if the amount of
                                          gross obligations proposed for each account was less than 75 percent of the agency’s total
                                          proposed gross obligations.
(3) Number of program activities          The number of program activities shown in the Appendix of the Budget of the United States
                                          Government, Fiscal Year 1999 from which the agency proposed to make gross obligations in
                                          fiscal year 1999. When the Appendix does not list any program activities under a particular
                                          account, the account was deemed to have one program activity, reflecting the entire budget
                                          account.
(4) Common program activity structure     A common program activity structure means that most of the agency’s accounts contain
                                          program activities with the same titles.
(5) Budget structure used in              a) Changed structure—Agencies in this category proposed to substantially change their
performance plan                          account and/or program activity structures in the Appendix of the Budget of the United States
                                          Government, Fiscal Year 1999 from those used in the previous year’s Appendix.

                                          b) Used fiscal year 1998 structure—Agencies in this category generally used the same account
                                          and program activity structures as they did in the Appendix of the Budget of the United States
                                          Government, Fiscal Year 1998.
(6) Number of strategic goals             The number of goals that are related directly to the agency’s mission without any intervening
                                          plan elements. According to the Results Act, agencies’ strategic plans are to contain general
                                          goals and objectives that elaborate on the agency mission statement and provide a set of
                                          programmatic objectives for the major functions and operations of the agencies. We defined
                                          the first layer of goals under the agency’s mission statement as “strategic goals” regardless of
                                          how they were labeled in the plan.
(7) Number of strategic objectivesa  The number of strategic objectives, that is, goals that the plan related directly to the agency’s
                                     “strategic goals” as defined in (6) above. As in (6), we defined this layer of goals as “strategic
                                     objectives” regardless of how they were labeled in the plan.
B. Characteristics describing how agencies linked program activities with performance goals
(8) Integration of performance plan with a) Full Integration—An agency embedded its performance plans in its justification of estimates
the agency’s budget request              such that the justification could not logically or readily be detached from the performance plan.

                                          b) Separable—The agency’s plan was either (1) a separable component of the agency’s
                                          justification of estimates (e.g., an appendix) such that justification readers could either turn to
                                          or skip over the performance plan, but the plan would appear in the justification’s table of
                                          contents; or (2) an entirely separate document that may or may not have been transmitted at
                                          the same time as the justification. A user would need to read the plan, as opposed to the
                                          justification, in order to understand how the agency addressed Results Act requirements.




                                               Page 45                                         GAO/AIMD/GGD-99-67 Performance Budgeting
                                            Appendix I
                                            Objectives, Scope, and Methodology




(9) Program activities linked tob, c   Agencies were placed in one of five categories depending on the lowest performance planning
                                       structure to which the plan linked program activities. (See figure I.1 for an illustration.)

                                       a) Strategic goal—Plan related program activities to strategic goals as defined in (6) above.

                                       b) Strategic objective—Plan related program activities to goals that the plan related directly to
                                       the agency’s “strategic goals” as defined in (6) above. These goals may or may not have been
                                       labeled “strategic objectives” by the agency.

                                       c) Performance goal—Plan related program activities to performance goals and/or measures.
                                       As defined in the Results Act, a performance goal means a target level of performance
                                       expressed as a tangible, measurable objective against which actual achievement can be
                                       compared, including a goal expressed as a quantitative standard, value, or rate. For plans in
                                       this category, a program activity was associated with each individual performance goal/
                                       measure.

                                       d) Other—Plan related program activities to a structure other than its strategic goals, strategic
                                       objectives, or performance goals. In many cases, this structure was a business line or some
                                       other unit for which a single goal statement was not expressed.

                                       e) None—Agencies were placed in this category if the performance plan did not relate program
                                       activities to any of the above structures (strategic goal, strategic objective, performance goal, or
                                       other) in their performance plans.
(10) Plan associated dollars with      Agencies were placed in one of six categories depending on the lowest performance planning
                                       structure for which the plan presented dollar amounts. (See figure I.2 for an illustration.)

                                       a) Strategic goal—Plan presented dollar amounts for each strategic goal.

                                       b) Strategic objective—Plan presented dollar amounts for each strategic objective.

                                       c) Set of discrete performance goals—Plan presented dollar amounts for any set of
                                       performance goals presented in some combination other than as described in (a), (b), (d), or
                                       (e).

                                       d) Performance goal and/or measure— Plan presented dollar amounts for each performance
                                       goal and/or measure.

                                       e) Other—Plan presented dollar amounts for a unit of analysis other than strategic goals,
                                       strategic objectives, or performance goals.

                                       f) None—Plan did not present dollar amounts for any performance planning or other type of
                                       structure.




                                            Page 46                                         GAO/AIMD/GGD-99-67 Performance Budgeting
                                                Appendix I
                                                Objectives, Scope, and Methodology




(11) Relationship of program activities a) One program activity to one goal—Agencies were placed in this category if a program
to performance goals and/or measuresd activity was linked to one performance goal.

                                           b) One program activity to many goals—Agencies were placed in this category if a program
                                           activity was linked to more than one performance goal.

                                           c) Many program activities to one goal—Agencies were placed in this category if more than one
                                           program activity was linked to one performance goal.

                                           d) Many program activities to many goals—Agencies were placed in this category if more than
                                           one program activity was linked to more than one performance goal.

                                           e) Could not be determined—Agencies were placed in this category if the plan did not convey
                                           how program activities were related to performance goals.
(12) Funding allocated to a discrete set   Plans that allocated funding to a discrete set of goals
of goals and/or measurese
                                           (a) generally showed how program activities and their requested funding were allocated among
                                           performance goals/measures or sets of performance goals/measures (plans met this criteria
                                           even if only discretionary funding was allocated) and

                                           (b) used sets of goals/measures that were unique (i.e., a single performance goal/measure is
                                           related to only one strategic objective or strategic goal).
                                                aWhilewe quantified strategic objectives as defined, the plan may have contained other goal layers
                                                between strategic objectives and annual performance goals.
                                                b
                                                 We reviewed linkages between the program activities and performance goals presented in the plan.
                                                This characteristic did not assess whether all program activities were listed and covered in the
                                                performance plan.
                                                c
                                                  If accounts were linked to the planning structure, the underlying program activities were also
                                                presumed to be linked to this structure.
                                                dWhen   a plan contained performance goals distinct from performance measures, reviewers
                                                considered whether this assessment would change if the word “measure(s)” was substituted for the
                                                word “goal(s).” If so, measure(s) were used as the unit of analysis instead of goal(s). If agencies
                                                linked program activities to a structure other than performance goals (i.e., an intervening structure
                                                such as a strategic goal or objective), the plan reviewer determined this relationship by examining how
                                                program activities were related to the intervening structure and how the intervening structure was
                                                related to performance goals.
                                                e
                                                 The set of goals ranged in size and scope. For example, some of these plans presented allocations
                                                of funding to strategic goals or objectives, which represent discrete sets of performance goals.




                                                Page 47                                             GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




Figure I.1: Illustration of Characteristic 9
Example 9(a)

  Account 1

          Program                 Strategic goal 1
          activity 1
                                         Strategic objective 1
                                                 Performance goal 1
          Program
          activity 2                             Performance goal 2
                                         Strategic objective 2
          Program                                Performance goal 3
          activity 3
                                                 Performance goal 4
          Program                 Strategic goal 2
          activity 4                     Strategic objective 3
                                                 Performance goal 5
          Program                                Performance goal 6
          activity 5


Example 9(b)

  Account 1

          Program                 Strategic goal 1
          activity 1
                                        Strategic objective 1
                                                Performance goal 1
          Program
          activity 2                            Performance goal 2
                                        Strategic objective 2
          Program                               Performance goal 3
          activity 3
                                                Performance goal 4
          Program                 Strategic goal 2
          activity 4                     Strategic objective 3
                                                Performance goal 5
          Program                               Performance goal 6
          activity 5




Page 48                                        GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




Example 9(c)

  Account 1

          Program                Strategic goal 1
          activity 1
                                      Strategic objective 1
                                           Performance goal 1
          Program
          activity 2                       Performance goal 2
                                      Strategic objective 2
          Program
                                           Performance goal 3
          activity 3
                                           Performance goal 4
          Program                Strategic goal 2
          activity 4                  Strategic objective 3
                                           Performance goal 5
          Program
                                           Performance goal 6
          activity 5




Page 49                                  GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




Figure I.2: Illustration of Characteristic 10
Example 10(a)



                      Strategic goal 1
                           Strategic objective 1
                                 Performance goal 1
                                 Performance goal 2
                           Strategic objective 2
                                 Performance goal 3
                                 Performance goal 4
                      Strategic goal 2
                            Strategic objective 3
                                 Performance goal 5
                                 Performance goal 6


Example 10(b)



                     Strategic goal 1
                          Strategic objective 1
                                Performance goal 1
                                Performance goal 2
                          Strategic objective 2
                                Performance goal 3
                                Performance goal 4
                     Strategic goal 2
                           Strategic objective 3
                                Performance goal 5
                                Performance goal 6




Page 50                                     GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




Example 10(c)


                     Strategic goal 1
                          Strategic objective 1
                                Performance goal 1
                                Performance goal 2
                                Performance goal 3

                     Strategic goal 2
                           Strategic objective 2
                                Performance goal 4
                                Performance goal 5
                                Performance goal 6

Example 10(d)


                      Strategic goal 1
                           Strategic objective 1
                                 Performance goal 1
                                 Performance goal 2
                           Strategic objective 2
                                 Performance goal 3
                                 Performance goal 4
                      Strategic goal 2
                            Strategic objective 3
                                 Performance goal 5
                                 Performance goal 6




Page 51                              GAO/AIMD/GGD-99-67 Performance Budgeting
                Appendix I
                Objectives, Scope, and Methodology




Data Analysis   To determine which characteristics of agency planning and budgeting
                structures were associated with linkages that showed an allocation of
                budgetary resources to results, we prepared contingency tables depicting
                the relationship between characteristic 12 in table I.2, “Funding allocated
                to a discrete set of goals or measures,” and each of the other
                characteristics. To assess whether the relationships in the tables were
                statistically significant, we performed two statistical techniques. When one
                characteristic in the table contained numeric values (e.g., characteristic 6,
                which measured the number of strategic goals), we used logistic regression
                techniques. The logistic regression technique involved regressing the odds
                on funding being allocated to a discrete set of performance goals on each
                characteristic and determining—using likelihood ratio chi square tests—
                whether the characteristic was associated with significant differences in
                those odds.

                When a characteristic had nonnumeric values (e.g., characteristic 11, which
                had five discrete categories), we used standard contingency tables to
                analyze the data. The contingency table techniques involved calculating the
                percentages of agencies that allocated funding to a discrete set of goals
                across the categories of the other characteristic and computing likelihood-
                ratio chi square statistics to determine whether differences in those
                percentages were statistically significant.3 In addition to computing the
                likelihood ratio chi square statistic, we also computed Fisher’s exact test to
                assess whether a characteristic was significantly related to characteristic
                12. Fisher’s exact test was used to confirm the likelihood ratio chi square
                results because of the small number of observations in many of our tables.
                In most of our analyses, the likelihood ratio chi square and Fisher’s exact
                test yielded similar conclusions. Where they did not, the differences
                appeared negligible given our sample size.4

                When a statistically significant association was identified in a table where
                one or both characteristics being analyzed had more than two categories,
                we conducted a series of additional chi square tests before and after
                grouping various categories of those characteristics to discern whether our


                3Criterion for statistical significance was set at about the 0.05 or less probability level. If the two charac-
                teristics were independent of each other (i.e., unrelated), it would be unlikely that probability values at
                or less than 0.05 would occur. If such extreme probability values do occur, the null hypothesis that
                there is no association is rejected.
                4
                  We report those results as significant where one of the statistical tests yielded a probability level that
                was slightly above 0.05 and the other was at or below 0.05.




                Page 52                                                GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




description of the relationship of that characteristic with characteristic
12 could be simplified. Appendix II presents a summary of our analysis.

The intent of our statistical analyses was to quantitatively identify and
explore associations of various plan characteristics with plans that did or
did not allocate funding to a discrete set of goals and/or measures.
However, the following qualifications apply to our analysis.

• Although the results of our analyses apply to the plans we reviewed, our
  plan selection procedures preclude generalizing the results to agency
  plans not included in our population.
• For those characteristics we identified as having a significant
  association with characteristic 12, it is possible that this result occurred
  because of the close association of that characteristic with one or more
  of the other characteristics that were related to characteristic 12. The
  small number of plans reviewed precluded our use of statistical
  approaches that would enable us to assess the relationship of two or
  more characteristics simultaneously on characteristic 12. To provide
  some rudimentary insight into the extent to which characteristics
  significantly related to characteristic 12 were related to each other, we
  examined whether there were significant associations among those
  characteristics using the same procedures described above. Significant
  associations were found between (1) characteristics 5 and 8,5
  (2) characteristics 8 and 10,6 (3) characteristics 8 and 11,7 and
  (4) characteristics 10 and 11.8
• Aspects of agencies’ linkages not specifically mentioned in table I.2
  were not assessed. For example, we did not assess whether all agency
  program activities were listed and covered in the performance plan.
  This assessment was made in our September 1998 report.9
• Our analysis focused on linkages between performance goals and
  program activities in performance plans. We did not assess other


5The probabilities associated with the likelihood ratio chi square and Fisher’s exact test were 0.04 and
0.07, respectively.
6
  The probabilities associated with the likelihood ratio chi square and Fisher’s exact test were 0.01 and
0.03, respectively.
7
  The probabilities associated with the likelihood ratio chi square and Fisher’s exact test were 0.02 and
0.03, respectively.
8The probabilities associated with the likelihood ratio chi square and Fisher’s exact test were both less
than 0.00.
9
    See GGD/AIMD-98-228, September 8, 1998.




Page 53                                             GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix I
Objectives, Scope, and Methodology




  elements of the performance plan, such as the quality of any goals
  presented in the plan. We also did not independently verify the funding
  amounts that agencies allocated to performance goals. We did not
  systematically assess other documents, such as agency budget
  justifications.
• Our review focused on the 12 characteristics mentioned above.
  However, there may be other characteristics that might be associated
  with agencies’ linkages. For example, although our previous work has
  noted that account and program activity orientation may be important
  factors in making linkages, the subjective nature of this characteristic
  prevented its inclusion in our analysis.

Finally, to identify implications for future performance budgeting efforts,
we gathered information on congressional perspectives. In addition to
discussing the plans’ linkages between budgetary resources and results
with selected appropriations staff, we also reviewed the House Committee
on Appropriations’ hearing records on agencies’ fiscal year 1999
appropriations, giving special attention to how lawmakers reacted to the
performance information presented in performance plans and budget
justifications. We discussed the status of performance budgeting pilots
with OMB. We requested comments on a draft of this report from the
Director of OMB or his designee and incorporated OMB’s comments as
appropriate. We conducted our work in accordance with generally
accepted government auditing standards from August 1998 to February
1999.




Page 54                              GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix II

Summary of Our Analysis                                                                                                              ApIpexndi




                                                                          Did not allocate
                                                    Allocated program     program activity
                                                    activity funding to   funding to
Nonnumeric characteristics                          performance goals     performance goals   Total              Probabilities
                                                    Number     Percent    Number   Percent    Number     Likelihood       Fisher’s
                                                                                                         ratio            exact
Concentration of spending                                                                                0.766            1.000
   Spending in multiple accounts                    10         42         14       58         24
   Spending concentrated in one account             4          36         7        64         11
Program activity structure                                                                               0.768            1.000
   Not common                                       13         39         20       61         33
   Common                                           1          50         1        50         2
Budget structure used in the performance                                                                 0.015            0.056
plan
   Proposed change in structure                     3          100        0        0          3
   Used fiscal year 1998 structure                  11         34         21       66         32
Integration of performance plan                                                                          0.016            0.028
   Separate from budget justification               9          31         20       69         29
   Fully integrated with budget justification       5          83         1        17         6
Program activities linked to                                                                             0.199            0.375
   Strategic goal                                   3          50         3        50         6
   Strategic objective                              3          43         4        57         7
   Performance goal                                 7          50         7        50         14
   Other                                            1          33         2        67         3
   None                                             0          0          5        100        5
                                                                                                         a                a
Plan associated dollars with
   Strategic goal                                   5          71         2        29         7
   Strategic objective                              3          50         3        50         6
   Set of performance goals                         4          100        0        0          4
   Performance goal                                 0          0          1        100        1
   Other                                            1          50         1        50         2
   None                                             1          7          14       93         15
Relationship of program activities to performance goals                                                  0.000 b          0.001 b
   Simple relationship                              8          100        0        0          8
   Complex relationship                             6          27         16       73         22




                                                Page 55                                  GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix II
Summary of Our Analysis




aAlthough    the probabilities for this characteristic were 0.05 or less, our analysis revealed the source of
statistical significance was the difference between the “none” category and all of the other categories
shown for this characteristic combined.
b
 We could not determine the relationship between program activities and performance goals for 5
agencies. Therefore, we determined whether there were significant differences among the remaining
30 agencies.




Page 56                                               GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix III

Major Contributors to This Report
                                                                                                      AIpIexndi




Accounting and          Michael J. Curro, Assistant Director
                        Laura E. Castro, Evaluator-in-Charge
Information             Linda F. Baker, Senior Evaluator
Management Division,    Toni J. Wehman, Evaluator
Washington, D.C.

General Government      Thomas M. Beall, Senior Social Science Analyst
                        Douglas M. Sloane, Supervisory Social Science Analyst
Division, Washington,
D.C.




(935292)       eL
                rtet    Page 57                                GAO/AIMD/GGD-99-67 Performance Budgeting
Appendix III
Major Contributors to This Report




Page 58                             GAO/AIMD/GGD-99-67 Performance Budgeting
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested