oversight

Results Act: Observations on CFTC's Fiscal Year 2000 Annual Performance Plan

Published by the Government Accountability Office on 1999-03-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to the Honorable Thomas W.
                Ewing, Chairman, Subcommittee on Risk
                Management and Specialty Crops,
                Committee on Agriculture,
                House of Representatives
March 1999
                RESULTS ACT
                Observations on
                CFTC’s Fiscal Year
                2000 Annual
                Performance Plan




GAO/GGD-99-51
GAO                United States
                   General Accounting Office
                   Washington, D.C. 20548

                   General Government Division



                   B-281743

                   March 15, 1999

                   The Honorable Thomas W. Ewing
                   Chairman, Subcommittee on Risk Management
                    and Specialty Crops
                   Committee on Agriculture
                   House of Representatives

                   Dear Mr. Chairman:

                   As part of your continuing effort to ensure that the Government
                   Performance and Results Act of 1993 (the Results Act) achieves its full
                   promise, you asked us to review the fiscal year 2000 annual performance
                   plan of the Commodity Futures Trading Commission (CFTC) to determine
                   whether it complies with the requirements of the Results Act. In October
                                                                   1
                   1998, we testified on the results of our review, and you subsequently
                   asked that we provide you with a report on our results.

                   This report provides our observations on CFTC’s fiscal year 2000 annual
                   performance plan. Successful implementation of a performance-based
                   management system, as envisioned by the Act, represents a significant
                   challenge that requires sustained agency attention. Our observations are
                   intended to assist CFTC in its continuing efforts to improve its plan. As a
                   result, our report focuses on the areas in which CFTC could improve
                   future plans—not on areas in which CFTC has already made significant
                   progress.

                   CFTC actions show a good-faith effort and commitment both to comply
Results in Brief   with the Results Act and meet congressional expectations for information
                   about the agency’s performance goals, including how the agency will
                   accomplish the goals and measure their results. However, CFTC could
                   improve future performance plans in several areas to make the plans more
                   useful to congressional and other decisionmakers. First, CFTC could
                                                  2          3
                   improve its performance goals, measures, and targets to provide a clearer

                   1
                       Results Act: Observations on CFTC’s Annual Performance Plan (GAO/T-GGD-99-10, Oct. 8, 1998).
                   2
                    Performance goals, as defined in the Results Act, are levels of performance that are expressed as
                   tangible, measurable objectives against which actual achievement can be compared; they include goals
                   expressed as quantitative standards, values, or rates.
                   3
                    Performance measures are tabulations, calculations, recordings of activity or effort, or assessments of
                   results that are compared to an intended purpose with the objective of assessing progress toward
                   achieving performance goals. CFTC’s plan uses the term “performance indicator” instead of
                   “performance measure.” We use the term performance measure in this report.




                   Page 1                        GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
             B-281743




             picture of its intended performance. Second, CFTC could better connect
             its mission, goals, and activities to demonstrate more fully how it plans to
             chart annual progress toward achieving its strategic goals. Third, CFTC
             could more fully address crosscutting efforts that involve other federal
             agencies to better ensure that the overall effectiveness of federal efforts is
             maximized. Fourth, CFTC could discuss in greater detail the strategies and
             resources used to achieve its performance goals to better enable
             decisionmakers to assess their reasonableness. Fifth, CFTC should
             describe the specific means for verifying and validating the quality of
             performance information and ensuring the reliability of such information.
             Finally, CFTC could link past accomplishments to performance goals to
             enhance the plan’s clarity.

             Although CFTC could improve its plan in these areas, we have found that
             other federal agencies, including federal financial regulators, could
                                                             4
             improve their plans in many of the same areas. Moreover, it is important
             to recognize that the Results Act anticipates that the process of developing
             an effective planning process and performance plan could take several
             planning cycles.

             CFTC, an independent agency created by Congress in 1974, administers
Background                                                        5
             the Commodity Exchange Act (CEA), as amended. The principal purposes
             of the CEA are to protect the public interest in the proper functioning of
             the futures and option markets’ price discovery and risk-shifting functions.
             In administering the CEA, CFTC is responsible for fostering the economic
             utility of the markets by encouraging their efficiency, monitoring their
             integrity, and protecting market participants from abusive trade practices
             and fraud.

             In recent years, agencies have moved toward a performance-based
             approach to management. Congress enacted the Results Act in 1993 as part
             of a framework of reform legislation that included the Chief Financial
             Officers Act and information technology legislation, such as the Clinger-
             Cohen Act of 1996, to instill performance-based management in the federal
             government. The Results Act seeks to improve the management of federal
             programs, including their efficiency and effectiveness, by establishing a
             system under which agencies set goals for program performance and
             measure their results. The Act is intended to shift the focus of government
             decisionmaking and accountability away from a preoccupation with

             4
                 See the listing of related GAO products at the end of this report.
             5
                 7 U.S.C. §§ 1-25.




             Page 2                          GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
              B-281743




              activities—such as the number of market surveillance reports prepared—
              to a focus on the results of those activities, such as protecting the
              economic functioning of the commodity futures and option markets.

              Under the Results Act, strategic plans are the starting point for setting
              goals and measuring progress toward them. The Results Act requires
              virtually every executive agency to develop a strategic plan, covering a
              period of at least 5 years forward from the fiscal year in which the plan is
              submitted. In September 1997, CFTC formally submitted its fiscal years
              1997-2002 strategic plan to Congress and the Office of Management and
              Budget (OMB). This plan established three strategic goals: (1) protect the
              economic functions of the commodity futures and option markets; (2)
              protect market users and the public; and (3) foster open, competitive, and
              financially sound markets.

              The Results Act also requires a federal agency to prepare an annual
              performance plan covering the program activities set out in its budget. In
              establishing the requirement for a performance plan, the Results Act
              establishes the first statutory link between an agency’s budget request and
              its performance-planning efforts. The performance plan is to reinforce the
              connections between the long-term strategic goals outlined in the agency’s
              strategic plan and the daily activities of program managers and staff. CFTC
              submitted its initial performance plan for fiscal year 2000 to OMB in
                                6
              September 1998. It is the second performance plan that CFTC has
              prepared under the Act.

              Finally, the Results Act requires executive agencies to prepare annual
              reports on program performance for the previous fiscal year. The
              performance reports are to be issued by March 31 each year, with the first
              report (for fiscal year 1999) to be issued by March 31, 2000. In each report,
              the agency is to compare its performance against its goals, summarize
              findings of program evaluations completed during the year, and describe
              the actions needed to address any unmet goals.

              To develop our observations, we compared the CFTC fiscal year 2000
Scope and     annual performance plan to the Results Act requirements, CFTC fiscal year
Methodology   1999 annual performance plan, and CFTC fiscal years 1997-2002 strategic
              plan. We also discussed these plans with CFTC staff in Washington, D.C. In
              addition, our observations were based on our knowledge of the agency’s

              6
               According to OMB guidance, initial performance plans are due to OMB with the agencies’ budget
              requests. Final annual performance plans are to be sent to Congress, following the transmittal of the
              President’s budget.




              Page 3                      GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                       B-281743




                       operations and programs; past reviews of CFTC; and results of our work
                       on other agencies’ performance plans and the Results Act.

                       The criteria we used to assess whether CFTC’s annual performance plan
                       complied with the requirements of the Results Act were the Act itself;
                       OMB guidance on preparing strategic and performance plans (OMB
                       Circular A-11, Part 2); and our guidance on assessing agency performance
                             7
                       plans. Specifically, we used the criteria and questions contained in our
                       guidance to determine whether the performance plan met the
                       requirements of the Act, identify strengths and weaknesses in the plan, and
                       assess the plan’s usefulness for congressional oversight. We also assessed
                       the plan’s discussion of actual performance data for fiscal year 1998.

                       We did our work between September 1998 and February 1999 at CFTC in
                       Washington, D.C., in accordance with generally accepted government
                       auditing standards. We requested comments on a draft of this report from
                       CFTC. These comments are discussed near the end of this letter and
                       reprinted in appendix I.

                       The Results Act requires that annual performance plans (1) establish
CFTC’s Performance     performance goals that define the levels of performance to be achieved;
Plan Could Provide a   (2) express the performance goals in an objective, quantifiable, and
Clearer Picture of     measurable form; (3) establish performance measures to be used in
                       assessing progress toward achieving the goals; and (4) provide a basis for
Intended Performance   comparing actual program results with the established goals. CFTC’s
                       performance plan takes important steps toward addressing these
                       requirements by, among other things, providing performance goals,
                       measures, and targets that reflect CFTC’s mission and long-term strategic
                       goals. However, the plan could provide a clearer picture of intended
                       performance and be of greater use in assessing progress toward achieving
                       intended performance if performance goals, measures, and targets were
                       improved. Specifically, CFTC could improve its plan by (1) making its
                       performance goals and measures more results-oriented and providing
                                                                                                  8
                       goals for all measures; (2) making some performance goals self-measuring
                       and others more objective; (3) replacing, restating, or deleting certain
                       7
                       See Agencies’ Annual Performance Plans Under the Results Act: An Assessment Guide to Facilitate
                       Congressional Decisionmaking (GAO/GGD/AIMD-10.1.18, Feb. 1998) and The Results Act: An
                       Evaluator’s Guide to Assessing Agency Annual Performance Plans (GAO/GGD-10.1.20, Apr. 1998).
                       8
                        Self-measuring goals are expressed objectively and quantifiably and thus do not require the use of
                       additional measures. According to OMB guidance, the Act allows an agency to define a performance
                       goal in a way that is not self-measuring. If a goal is not self-measuring, one cannot determine whether
                       the goal was achieved by objectively comparing actual performance to the goal. Therefore, for a goal
                       that is not self-measuring, the plan must include one or more performance measures that set out
                       specific, measurable values or characteristics.




                       Page 4                      GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                           B-281743




                           performance measures; and (4) establishing baselines against which
                           annual targets could be compared.

Goals and Measures Could   Results-oriented, or outcome, goals and measures provide the clearest
                           picture of intended and actual performance. Such goals and measures
Be More Results-Oriented   focus on the intended result, effect, or consequence that is expected to
and Complete               occur from carrying out a program or activity. We recognize that
                           establishing outcome measures is particularly challenging for regulatory
                           agencies as they move from a focus on the activities that they undertake to
                           the results that they are trying to achieve. However, a key shortcoming of
                           CFTC’s performance plan is that it relies predominantly on output
                                                                                              9
                           measures that describe completed activities, not program results. In our
                           October 1997 testimony before this Subcommittee, we highlighted a
                                                                         10
                           similar problem with CFTC’s strategic plan.

                           CFTC’s plan has 31 performance goals and 228 performance measures.
                           About two-thirds of the goals do not have outcome measures to describe
                           intended results. Of the plan’s 228 measures, 184 are output measures that
                           capture program outputs such as the number of meetings attended and
                           number of reports and research projects completed. These measures are
                           weighted toward measuring the quantity of completed activities, rather
                           than the quality, cost, or timeliness of performance outcomes. In addition,
                           15 of the performance measures are input measures that capture workload
                           inputs such as the number of reports that CFTC received from futures
                           firms and exchanges. Although such information is helpful for defining the
                           amount of work that must be performed, it has limited utility in conveying
                           the expected outcome to result from such work. As previously mentioned,
                           the Results Act is intended to shift the focus of government
                           decisionmaking and accountability away from a preoccupation with
                           completed activities to a focus on the results of such activities.

                           The focus of CFTC’s performance plan on output measures appears to
                           flow from CFTC’s strategy of deriving performance goals from program
                           activities. The agency’s reliance on program activities to develop
                           performance goals may be counterproductive because it appears to focus
                           attention on completed activities and not on the results of such activities.
                           For example, one of the plan’s program activities is to monitor the Internet
                           9
                            OMB guidance emphasizes that outcome goals should be included in a performance plan whenever
                           possible. A performance plan should include outcome goals when the related goals are to be achieved
                           in the year covered by the plan. The guidance notes that most plans will supplement outcome goals
                           with output measures. Output measures can be the predominant goals and measures in an annual
                           performance plan under certain circumstances.
                           10
                                Results Act: Observations on CFTC’s Strategic Plan (GAO/T-GGD-98-17, Oct. 22, 1997).




                           Page 5                         GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
B-281743




and other media for fraudulent activities and other possible violations of
the CEA. The performance goal for this activity is to aggressively identify,
investigate, and take action against individuals engaged in fraudulent
Internet and media activities. The measure for the goal is the number of
referrals to enforcement authorities generated from Internet and media
monitoring—an output of the activity, not an outcome.

According to CFTC officials, the plan’s performance goals are basically
restatements of their associated performance activities, which support the
plan’s outcome objectives. The officials said that a CFTC task force
examined the plan’s 50 program activities last year and reduced the
number of activities to 44 by modifying and consolidating several of them.
They told us that, in the upcoming year, the task force will examine
CFTC’s performance goals and targets with the objective of making them
more results-oriented.

CFTC could use the narrative sections of its performance plan that discuss
the agency’s fiscal year 1998 accomplishments to identify more results-
oriented performance goals. For example, the Enforcement Program
accomplishment section discusses the effectiveness of its quick-strike
ability—the ability to file injunctive actions quickly after detecting fraud—
to obtain, among other things, timely injunctive relief and enhance the
possibility that customer funds will be recovered. This accomplishment
section describes cases, filed within days or weeks of CFTC’s discovering
an illegal activity, that stopped fraud at an early stage and preserved
customer funds. CFTC’s performance measures could be made more
results-oriented by replacing measures, such as reports on activities
related to bringing injunctive actions and sanctioning violators, with more
outcome-related measures, such as the percentage of quick-strike cases
filed within a certain number of days of starting an investigation that
resulted in sanctions and the percentage of funds recovered.

CFTC could also learn from the plans of other federal financial regulators
that are attempting the transition to results-oriented goals. For example,
the National Credit Union Administration is developing new outcome
performance goals. One of its outcome goals is to ensure that federally
insured credit unions are adequately capitalized. A performance goal is to
reduce the percentage of federally insured credit unions that are
undercapitalized by 10 percent, from 372 to 335.

Finally, CFTC could also improve its performance plan and make it more
complete by providing performance goals for all of its program activities
that have performance measures. CFTC’s plan has 15 program activities



Page 6               GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                            B-281743




                            that have no corresponding performance goals but have performance
                            measures. For example, no performance goal exists for the activity of
                            reviewing the adequacy of self-regulatory organization disciplinary actions.
                            Without performance goals, it is not clear what performance is expected.
                            As discussed below, opportunities exist for CFTC to develop self-
                            measuring performance goals for some of these activities and their
                            measures.

Some Goals Could Be Made    Although not required by the Results Act, CFTC could redefine some
                            performance goals so that they are self-measuring, thereby reducing the
Self-Measuring and Others   number of measures and the complexity of the plan. Currently, 29 of
Could Be Made More          CFTC’s 31 performance goals are stated as abstract goals—that is, as goals
Objective                   requiring that specific performance measures be defined to assess
                            progress toward their achievement.

                            Performance goals that can be redefined so that they are self-measuring
                            generally have just one measure or two or more measures that can be
                            combined. CFTC’s plan has nine activities or goals that have only one
                            performance measure, and opportunities exist to redefine some of them in
                            self-measuring terms. For example, the performance goal for the activity of
                            reviewing the adequacy of self-regulatory organization disciplinary actions
                            could be made self-measuring by stating it the following way: On an annual
                            basis, review a defined percentage of self-regulatory organization
                            disciplinary actions to ensure compliance with CFTC standards. This
                            approach, which has been taken by other federal financial regulators, such
                            as the National Credit Union Administration and the Federal Deposit
                            Insurance Corporation, reduces the amount of subjectivity in interpreting
                            performance goals and more clearly defines performance expectations.
                            For example, the Federal Deposit Insurance Corporation has the following
                            performance goal: Market 80 percent of a failing financial institution’s
                            assets based on book value at the time of resolution or within 90 days.

                            CFTC could also improve its performance plan by making its performance
                            goals more objective. According to OMB guidance, to the extent possible,
                            goals should not require subjective considerations. CFTC’s plan has 14
                            performance goals that require subjective consideration in determining
                            whether they have been met. For example, one such goal is defined as
                            follows: Bring important cases (including matters involving ongoing
                            conduct and complex transactions) aggressively. The performance goal
                            does not define what an important case is or what it means to aggressively
                            bring a case. Although the goal has 11 performance measures—including
                            the numbers of cases filed, counts, and settlements—none clearly define
                            the terms “important” or “aggressively.”



                            Page 7              GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                         B-281743




Certain Performance      CFTC’s performance plan has 31 performance goals with 228 performance
                         measures to address them. CFTC could replace, restate, or delete
Measures Could Be        performance measures for certain performance goals to enhance its ability
Replaced, Restated, or   to assess the progress made in achieving performance goals.
Deleted
                         First, CFTC could replace certain performance measures to better capture
                         key aspects of the performance the measures are trying to assess. CFTC’s
                         plan has six performance goals that are partly defined in terms of
                         timeliness but do not have measures that capture timeliness. One such
                         goal is to review every designation application and rule change request
                         within 10 to 45 days. The goal’s performance measures are the number of
                         designation applications processed and the number of contract market
                         rule changes processed. Because neither measure captures the amount of
                         time it takes to process applications or changes, one option for improving
                         the plan is to replace these measures with one measure that captures the
                         percentage of applications and changes processed in 45 days or less.

                         Second, CFTC could restate or delete certain performance measures,
                         because they do not appear to be clearly related to their performance goals
                         and/or appear to be unduly affected by external factors. For example, the
                         number of active futures markets is used to measure, in part, two goals: (1)
                         Identify traders who can influence futures prices and (2) determine
                         whether traders are influencing futures prices. Because the number of
                         futures markets is largely determined by exchanges and other external
                         factors, this number has little direct bearing on the two goals and, thus,
                         could be deleted as a measure. Another example of such a measure is the
                         number of requests for assistance from foreign authorities, which is used,
                         in part, to measure the following goal: Maximize the enforcement
                         program’s effectiveness by cooperating with domestic and foreign
                         authorities in gathering information. This is an input measure that captures
                         action taken by foreign regulators, not CFTC.

                         Third, CFTC could restate or delete certain performance measures
                         because the measures might have limitations that preclude them from
                         accurately capturing intended performance or might promote unintended
                         consequences. For example, one of the plan’s performance goals is to
                         conduct important investigations and refer potential violations to other
                         authorities, as appropriate. The performance measures for this goal
                         include the number of documents obtained through subpoena or
                         inspection, number of witnesses from whom testimony was taken, and
                         number of witnesses interviewed. These output-oriented measures could
                         provide an incentive for staff to conduct more interviews, take more
                         testimonies, and obtain more documents than necessary, which could add



                         Page 8              GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                            B-281743




                            cost and time to investigations without necessarily contributing
                            commensurately to their success.

Baselines Could Be          Although CFTC’s performance plan includes annual targets for
                            performance goals, it could make such information more useful by
Established Against Which   providing baselines, or a context, for assessing the reasonableness and
to Compare Annual Targets   appropriateness of such targets. As we found in our review of other
                            performance plans, agencies that go beyond the requirements of the
                            Results Act and include baseline or trend data for their goals provide a
                            more informative basis for assessing expected performance.

                            CFTC’s performance plan provides many quantitative annual targets for
                            each fiscal year from 1998 through 2000. Many targets are set at the same
                            performance level as or at a lower performance level than in prior fiscal
                            years without an explanation. For example, the performance measures for
                            assessing sanctions under CFTC’s first strategic goal are the same in each
                            year covered by the plan for the number of cease-and-desist orders,
                            registration sanctions, and trading prohibitions. However, the bases for
                            setting the specific targets and the contributions of these targets to the
                                                                         11
                            outcome objective are not readily apparent. Without this contextual
                            information, it is not clear whether CFTC’s output-oriented performance
                            targets are reasonable and appropriate.

                            Consistent with the Results Act, CFTC’s performance plan attempts to
Mission, Goals, and         show the relationship between the agency’s annual performance goals and
Activities Could Be         its fiscal years 1997 through 2002 mission and strategic goals. To do so, the
Better Connected            plan uses tables that connect each strategic goal to its accompanying set of
                            performance goals, measures, and annual targets. The plan also ranks
                            CFTC’s outcome objectives by dollars budgeted, which is a starting point
                            for providing useful information about its priorities. However, the plan
                            could better connect mission, goals, and activities by more fully
                            demonstrating how CFTC will chart annual progress toward achieving its
                            long-term strategic goals.

                            The performance plan’s presentation of many separate program areas;
                            program activities; and performance goals, measures, and targets makes it
                            difficult to link CFTC’s mission and strategic goals to performance goals
                            across the entire agency. The plan’s 31 performance goals and 228
                            performance measures support 3 strategic goals and 9 strategic objectives,
                            covering 5 program areas. Although the plan’s ranking of outcome

                            11
                             The outcome objective is to foster futures and option markets that accurately reflect the forces of
                            supply and demand for the underlying commodity and are free of disruptive activity.




                            Page 9                      GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                       B-281743




                       objectives offers a useful perspective on CFTC’s priorities, given the high
                       level of complexity, it is difficult to (1) identify the agency’s key priorities
                       among the many goals and measures, (2) differentiate efforts to meet these
                       priorities, and (3) understand what will be achieved if all of the
                       performance goals are met.

                       Similarly, as we found with other agency performance plans, CFTC’s plan
                       associates one performance goal with multiple program activities and
                       strategic goals. For example, the performance goal—assess sanctions that
                       are remedial and deter violators—is associated with three different
                       program activities and three different strategic goals. Additionally, the
                       measures and targets for this performance goal differ with each program
                       activity and strategic goal. Associating one performance goal with multiple
                       program activities and strategic goals makes it difficult not only to
                       determine whether all activities are substantially covered but also to
                       understand how specific program activities are intended to contribute to
                       CFTC’s strategic goals.

                       The Results Act seeks to ensure that crosscutting goals of federal
Crosscutting Efforts   programs are consistent; strategies are mutually reinforcing; and, as
That Involve Other     appropriate, progress is assessed through the use of common performance
Agencies Could Be      measures. By accomplishing these goals, agencies could avoid wasting
                       scarce resources and could better ensure that the overall effectiveness of
More Fully Addressed   federal efforts is maximized. OMB guidance tasks performance plans with
                       identifying those performance goals that are being mutually undertaken
                       with other federal agencies in support of programs or activities of a
                                            12
                       crosscutting nature. It encourages agencies to go beyond coordination
                       and develop common performance goals and measures for related
                       programs. CFTC’s plan recognizes the need to address crosscutting efforts.
                       However, the agency could address such efforts in more substantive terms
                       if it worked with the cognizant federal agencies to develop performance
                       goals and measures that better reflect the nature and extent of their
                       common efforts.

                       CFTC’s performance plan briefly discusses CFTC’s need to work with
                       other U.S. financial regulators through, among other means, the
                                                                        13
                       President’s Working Group on Financial Markets. However, the related

                       12
                        OMB Circular A-11 guidance states that, at a minimum, the performance plan should indicate those
                       programs or activities that are being undertaken with other agencies to achieve a common purpose or
                       objective. An agency should also review the performance plans of other agencies participating with it
                       in a crosscutting program or activity to ensure that goals and measures are consistent and harmonious.
                       13
                          The President’s Working Group on Financial Markets was created following the October 1987 stock
                       market crash to address issues concerning the competitiveness, integrity, and efficiency of the




                       Page 10                     GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
B-281743




performance goals and measures do not substantively address the type of
crosscutting performance that the Working Group was created to address.
For example, CFTC’s performance goal covering the Working Group is for
CFTC to contribute to the performance of the group. The measure for this
goal is the number of meetings attended, and the fiscal year 2000 target is
two meetings. Neither the goal nor its measure captures the substantive
work of the Working Group’s efforts.

As supported by OMB guidance, CFTC could strengthen its performance
plan by participating with the members of the Working Group and other
federal regulators involved in crosscutting programs to develop common
performance goals and measures. For example, the continued growth and
development of the over-the-counter (off-exchange) derivatives market has
raised several potential regulatory concerns that affect CFTC and other
members of the Working Group. In addition, the potential need to develop
a financial markets contingency plan to address the “Year 2000” computer-
                14
dating problem could involve coordination among CFTC and other
federal financial regulators.

Additionally, CFTC’s performance plan could be expanded to include
performance goals and measures related to other crosscutting efforts, such
as those identified in its fiscal year 2000 budget justification. These efforts
include CFTC’s sharing of information with other financial regulators;
working with the U.S. Department of Agriculture on a risk-management
education program; contributing to a Department of the Treasury initiative
that encourages global financial stability; and engaging in cooperative
enforcement efforts with the Department of Justice, Federal Bureau of
Investigation, Federal Reserve Board, Federal Trade Commission,
Securities and Exchange Commission, and U.S. Postal Inspection Service.




financial markets. The Secretary of the Treasury chairs the group, and other members include the
chairs of CFTC, the Federal Reserve System, and the Securities and Exchange Commission.
14
 The Year 2000 computer-dating problem relates to the need for computer systems to be changed to
accommodate dates beyond the year 1999.




Page 11                    GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                           B-281743




                           Consistent with the Results Act, CFTC’s performance plan provides
Strategies and             important information on how strategies and resources will be used to
Resources Used to                            15
                           achieve its goals. However, the plan’s discussion could be expanded to
Achieve Goals Could        better enable congressional and other decisionmakers to judge the
                           reasonableness and appropriateness of CFTC’s strategies and anticipated
Be Discussed in            resource deployment. Specifically, the plan could be strengthened by (1)
Greater Detail             describing more fully the strategies needed to achieve performance goals,
                           including those for addressing both internal management challenges and
                           external factors, and (2) discussing in more detail the resources required
                           to achieve each performance goal.

The Strategies Needed to   Consistent with the Results Act and OMB guidance, CFTC’s performance
                           plan attempts to address the strategies that CFTC will use to achieve its
Achieve Goals Could Be     performance goals. Aligning its discussion with CFTC’s strategic goals, the
Described More Fully       plan briefly describes the major activities, or operational processes, of
                           each program in relation to its performance goals and measures. In a few
                           cases, the plan also discusses skills or technologies that programs will use
                           in relation to performance goals and measures. The plan could be made
                           more useful by providing additional information on the skills and, if
                           appropriate, technologies used in connection with operational processes
                           to achieve program goals.

                           CFTC’s fiscal years 1997-2002 strategic plan identifies several internal
                           management challenges, but CFTC’s performance plan does not include a
                           strategy—consisting of specific goals and measures—to address them.
                           These challenges include diminishing resources, recruiting and retaining
                           qualified professionals, remaining abreast of current technology, and
                           remaining educated and informed as innovation changes the industry.
                           According to OMB guidance, performance goals for management problems
                           should be included in the annual plan, particularly goals for those
                           problems whose resolution is mission-critical and which could impede
                           achievement of program goals. To better respond to the intent of the
                           Results Act, CFTC could add agencywide performance goals to the plan to
                           address these challenges or incorporate these challenges within the plan’s
                           existing structure of performance goals and measures. For example, CFTC
                           could add a performance goal that covers recruiting and retaining a highly
                           qualified workforce to handle complex issues facing the agency.

                           CFTC’s performance plan also discusses some of the external factors that
                           can affect its performance goals, but it does not identify the strategies that

                           15
                            Typically, strategies cover an agency’s operational processes, skills, and technology, while resources
                           cover human, capital, funding, and other resources.




                           Page 12                     GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                            B-281743




                            the agency plans to use to address them. In our testimony on CFTC’s fiscal
                            year 1997-2002 strategic plan, we highlighted this area, noting that CFTC
                            could strengthen its plan by describing how external factors are linked to
                            particular goals and how a particular goal could be affected by external
                            factors. Although the Results Act does not require performance plans to
                            discuss the specific impact of external factors on achieving performance
                            goals, such information can provide a better context for assessing CFTC’s
                            expected performance. Moreover, OMB guidance notes that agencies may
                            choose to include information on external factors in their annual plan
                            when the factors may bear directly on goal achievement for the fiscal year
                            covered by the plan.

Resources Required to       According to OMB guidance, a performance plan should discuss the
                            resources that will be applied to achieve each of the agency’s performance
Achieve Performance Goals   goals. Using tables and graphics, CFTC’s plan shows the amount of budget
Could Be Discussed in       funding and the number of full-time-equivalent employees that will be
Greater Detail              needed to achieve each strategic goal and related outcome objectives.
                            CFTC’s plan could be improved by addressing OMB guidance that also
                            calls for plans to describe the resources required to achieve each
                            performance goal.

                            At the October 1998 hearing, the CFTC Chairperson testified that the
                            agency is working to improve its capacity to account for budgetary
                            resources by activity, outcome objective, and performance goal. She said
                            that the agency’s financial reporting system was being modified to provide
                            budget data on a program and performance goal basis. She also said that
                            CFTC plans to link its financial reporting system to the activities of its
                            performance plan and, if successful, use it to formulate its fiscal year 2001
                            budget.




                            Page 13              GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                           B-281743




                           Although required by the Results Act and OMB guidance, CFTC’s fiscal
The Means for              year 2000 performance plan does not describe the procedures that the
Verifying and                                       16            17
                           agency will use to verify and validate that performance information is
Validating                 sufficiently complete, accurate, and consistent. These procedures can
                           include internal controls over data collection, maintenance, and entry.
Performance                External assessments, such as audits, evaluations, and peer reviews, are
Information and            other important procedures for verifying and validating performance
Ensuring Its Reliability   information. In addition, the plan does not, as required by the Act, discuss
Should Be Described        the extent to which the performance information and the means for
                           collecting, maintaining, and analyzing it are reliable. CFTC’s performance
                           plan should be expanded to address these requirements.

                           To assess progress toward achieving its goals, CFTC will need to collect
                           information on the over 200 performance measures in its plan. For some of
                           these measures, the amount of information to be collected is voluminous
                           and covers activities across CFTC headquarters and regional offices.
                           Errors can occur in collecting, maintaining, processing, and reporting such
                           information—potentially introducing bias and resulting in inaccurate
                           estimates of program performance. As a result, CFTC should have
                           procedures for ensuring that its performance information is free of
                           significant levels of error and that bias is not introduced. Such procedures
                           can include internal controls over data collection, maintenance, and entry
                           as well as audits, evaluations, and peer reviews.

                           At the October 1998 hearing, the CFTC Chairperson testified that the
                           agency is adapting its Quarterly Objectives review—a public document
                           that briefly describes each program’s priorities and accomplishments—to
                           match the annual performance plan’s goals, measures, and targets.
                           According to the Chairperson, the goal is to use the performance data in its
                           Quarterly Objectives review to evaluate whether CFTC is meeting its
                           performance plan targets.




                           16
                            Verify means to check or test performance data to reduce the risk of using data that contain
                           significant errors.
                           17
                            Validate means to ensure that the data are free of significant levels of error and that bias is not
                           introduced.




                           Page 14                      GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
                       B-281743




                       Although not required by the Results Act or OMB guidance, CFTC’s
Past Accomplishments   performance plan presents its fiscal year 1998 accomplishments. CFTC
Could Be Linked to     could improve the plan’s clarity by linking past accomplishments to
Performance Goals to   performance goals. CFTC’s plan provides a narrative section under each
                       strategic goal that discusses the agency’s fiscal year 1998 accomplishments
Enhance the Plan’s     by program. For example, in the first strategic goal section, the plan
Clarity                discusses, among other things, the major accomplishments of the Market
                       Surveillance, Analysis, and Research Program. These accomplishments
                       include conducting daily surveillance of 157 futures markets, collecting
                       and analyzing about 7,980 large trader reports, and preparing about 2,538
                       weekly surveillance reports. In some cases, CFTC links the discussion of
                       its past accomplishments to specific outcome objectives. However, CFTC
                       does not link the accomplishments to the performance goals described in
                       its fiscal year 1999 annual performance plan. Without this linkage, it is
                       difficult to determine the progress that CFTC made in achieving its
                       performance goals and, in turn, strategic goals.

                       Our observations on CFTC’s fiscal year 2000 annual performance plan are
Conclusions            intended to assist the agency in improving future plans so that they can be
                       more valuable tools for congressional and other decisionmakers. Although
                       we identified opportunities for CFTC to improve its plan—as we have done
                       for many other federal agencies—the Results Act anticipated that the
                       process of developing an effective planning process and performance plan
                       could take several planning cycles. Additionally, the agency’s ongoing
                       efforts reflect a commitment to fully complying with the Act.

                       In the upcoming year, CFTC plans to examine its performance goals to
                       make them more results-oriented, use performance data to evaluate
                       whether performance plan targets are being met, and improve its
                       accounting system to track budgetary resources by performance goals. To
                       further enhance its plan, CFTC could focus additional attention on (1)
                       improving its performance goals, measures, and targets to provide a
                       clearer picture of intended performance; (2) demonstrating more fully how
                       progress toward achieving strategic goals will be charted; (3) addressing
                       crosscutting efforts in more substantive terms; (4) linking strategies and
                       resources more clearly; and (5) providing confidence that performance
                       data will be sufficiently reliable. CFTC could also enhance the plan’s
                       clarity by linking past accomplishments to performance goals.

                       Written comments from CFTC on a draft of this report are contained in
Agency Comments        appendix I. CFTC said it appreciated the suggestions and
                       recommendations made in our report and by others. It also said it is




                       Page 15             GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
B-281743




committed to improving its future plans to make them as useful as
possible.

We are sending copies of this letter to Representative Gary A. Condit, the
Ranking Minority Member of your Subcommittee; and the Honorable
Brooksley Born, Chairperson of CFTC. We will also make copies available
to others on request.

This report was prepared under the direction of Cecile Trop, Assistant
Director. Other major contributors included Sharon Caudle and Richard
Tsuhara. Please contact me at (202) 512-8678 or Ms. Trop at (312) 220-7600
if you or your staff have any questions.

Sincerely yours,




Richard J. Hillman
Associate Director, Financial Institutions
 and Markets Issues




Page 16             GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
Page 17   GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
Appendix I

Comments From the Commodity Futures
Trading Commission




             Page 18   GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
Appendix I
Comments From the Commodity Futures Trading Commission




Page 19               GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
Related GAO Products



             The Results Act: Observations on the Federal Reserve’s 1998-99 Biennial
             Performance Plan (GAO/GGD-99-9R, Nov. 9, 1998).

             The Results Act: Observations on NCUA’s Annual Performance Plan
             (GAO/GGD-98-192R, Sept. 15, 1998).

             The Results Act: Observations on FDIC’s Annual Performance Plan
             (GAO/GGD-98-190R, Sept. 15, 1998).

             The Results Act: Observations on OCC’s Annual Performance Plan
             (GAO/GGD-98-189R, Sept. 15, 1998).

             Managing for Results: An Agenda to Improve the Usefulness of Agencies’
             Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

             Results Act: Observations on Treasury’s Fiscal Year 1999 Annual
             Performance Plan (GAO/GGD-98-149, June 30, 1998).

             Managing for Results: Experiences of Selected Credit Programs
             (GAO/GGD-98-41, Feb. 19, 1998).

             Managing for Results: Agencies’ Annual Performance Plans Can Help
             Address Strategic Planning Challenges GAO/GGD-98-44, Jan. 30, 1998).

             The Results Act: Observations on Draft Strategic Plans of Five Financial
             Regulatory Agencies (GAO/T-GGD-97-164, July 29, 1997).

             Managing for Results: Regulatory Agencies Identified Significant Barriers
             to Focusing on Results (GAO/GGD-97-83, June 24, 1997).

             The Government Performance and Results Act: 1997 Governmentwide
             Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

             Managing for Results: Analytic Challenges in Measuring Performance
             (GAO/HEHS/GGD-97-138, May 30, 1997).




             Page 20             GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan
Ordering Information

The first copy of each GAO report and testimony is free. Additional
copies are $2 each. Orders should be sent to the following address,
accompanied by a check or money order made out to the
Superintendent of Documents, when necessary. VISA and
MasterCard credit cards are accepted, also. Orders for 100 or more
copies to be mailed to a single address are discounted 25 percent.

Order by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
     th                  th
700 4 St. NW (corner of 4 and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using fax
number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony.
To receive facsimile copies of the daily list or any list from the past
30 days, please call (202) 512-6000 using a touch-tone phone. A
recorded menu will provide information on how to obtain these
lists.

For information on how to access GAO reports on the INTERNET,
send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested




(233595)