United States General Accounting Office GAO Report to the Honorable Thomas W. Ewing, Chairman, Subcommittee on Risk Management and Specialty Crops, Committee on Agriculture, House of Representatives March 1999 RESULTS ACT Observations on CFTC’s Fiscal Year 2000 Annual Performance Plan GAO/GGD-99-51 GAO United States General Accounting Office Washington, D.C. 20548 General Government Division B-281743 March 15, 1999 The Honorable Thomas W. Ewing Chairman, Subcommittee on Risk Management and Specialty Crops Committee on Agriculture House of Representatives Dear Mr. Chairman: As part of your continuing effort to ensure that the Government Performance and Results Act of 1993 (the Results Act) achieves its full promise, you asked us to review the fiscal year 2000 annual performance plan of the Commodity Futures Trading Commission (CFTC) to determine whether it complies with the requirements of the Results Act. In October 1 1998, we testified on the results of our review, and you subsequently asked that we provide you with a report on our results. This report provides our observations on CFTC’s fiscal year 2000 annual performance plan. Successful implementation of a performance-based management system, as envisioned by the Act, represents a significant challenge that requires sustained agency attention. Our observations are intended to assist CFTC in its continuing efforts to improve its plan. As a result, our report focuses on the areas in which CFTC could improve future plans—not on areas in which CFTC has already made significant progress. CFTC actions show a good-faith effort and commitment both to comply Results in Brief with the Results Act and meet congressional expectations for information about the agency’s performance goals, including how the agency will accomplish the goals and measure their results. However, CFTC could improve future performance plans in several areas to make the plans more useful to congressional and other decisionmakers. First, CFTC could 2 3 improve its performance goals, measures, and targets to provide a clearer 1 Results Act: Observations on CFTC’s Annual Performance Plan (GAO/T-GGD-99-10, Oct. 8, 1998). 2 Performance goals, as defined in the Results Act, are levels of performance that are expressed as tangible, measurable objectives against which actual achievement can be compared; they include goals expressed as quantitative standards, values, or rates. 3 Performance measures are tabulations, calculations, recordings of activity or effort, or assessments of results that are compared to an intended purpose with the objective of assessing progress toward achieving performance goals. CFTC’s plan uses the term “performance indicator” instead of “performance measure.” We use the term performance measure in this report. Page 1 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 picture of its intended performance. Second, CFTC could better connect its mission, goals, and activities to demonstrate more fully how it plans to chart annual progress toward achieving its strategic goals. Third, CFTC could more fully address crosscutting efforts that involve other federal agencies to better ensure that the overall effectiveness of federal efforts is maximized. Fourth, CFTC could discuss in greater detail the strategies and resources used to achieve its performance goals to better enable decisionmakers to assess their reasonableness. Fifth, CFTC should describe the specific means for verifying and validating the quality of performance information and ensuring the reliability of such information. Finally, CFTC could link past accomplishments to performance goals to enhance the plan’s clarity. Although CFTC could improve its plan in these areas, we have found that other federal agencies, including federal financial regulators, could 4 improve their plans in many of the same areas. Moreover, it is important to recognize that the Results Act anticipates that the process of developing an effective planning process and performance plan could take several planning cycles. CFTC, an independent agency created by Congress in 1974, administers Background 5 the Commodity Exchange Act (CEA), as amended. The principal purposes of the CEA are to protect the public interest in the proper functioning of the futures and option markets’ price discovery and risk-shifting functions. In administering the CEA, CFTC is responsible for fostering the economic utility of the markets by encouraging their efficiency, monitoring their integrity, and protecting market participants from abusive trade practices and fraud. In recent years, agencies have moved toward a performance-based approach to management. Congress enacted the Results Act in 1993 as part of a framework of reform legislation that included the Chief Financial Officers Act and information technology legislation, such as the Clinger- Cohen Act of 1996, to instill performance-based management in the federal government. The Results Act seeks to improve the management of federal programs, including their efficiency and effectiveness, by establishing a system under which agencies set goals for program performance and measure their results. The Act is intended to shift the focus of government decisionmaking and accountability away from a preoccupation with 4 See the listing of related GAO products at the end of this report. 5 7 U.S.C. §§ 1-25. Page 2 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 activities—such as the number of market surveillance reports prepared— to a focus on the results of those activities, such as protecting the economic functioning of the commodity futures and option markets. Under the Results Act, strategic plans are the starting point for setting goals and measuring progress toward them. The Results Act requires virtually every executive agency to develop a strategic plan, covering a period of at least 5 years forward from the fiscal year in which the plan is submitted. In September 1997, CFTC formally submitted its fiscal years 1997-2002 strategic plan to Congress and the Office of Management and Budget (OMB). This plan established three strategic goals: (1) protect the economic functions of the commodity futures and option markets; (2) protect market users and the public; and (3) foster open, competitive, and financially sound markets. The Results Act also requires a federal agency to prepare an annual performance plan covering the program activities set out in its budget. In establishing the requirement for a performance plan, the Results Act establishes the first statutory link between an agency’s budget request and its performance-planning efforts. The performance plan is to reinforce the connections between the long-term strategic goals outlined in the agency’s strategic plan and the daily activities of program managers and staff. CFTC submitted its initial performance plan for fiscal year 2000 to OMB in 6 September 1998. It is the second performance plan that CFTC has prepared under the Act. Finally, the Results Act requires executive agencies to prepare annual reports on program performance for the previous fiscal year. The performance reports are to be issued by March 31 each year, with the first report (for fiscal year 1999) to be issued by March 31, 2000. In each report, the agency is to compare its performance against its goals, summarize findings of program evaluations completed during the year, and describe the actions needed to address any unmet goals. To develop our observations, we compared the CFTC fiscal year 2000 Scope and annual performance plan to the Results Act requirements, CFTC fiscal year Methodology 1999 annual performance plan, and CFTC fiscal years 1997-2002 strategic plan. We also discussed these plans with CFTC staff in Washington, D.C. In addition, our observations were based on our knowledge of the agency’s 6 According to OMB guidance, initial performance plans are due to OMB with the agencies’ budget requests. Final annual performance plans are to be sent to Congress, following the transmittal of the President’s budget. Page 3 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 operations and programs; past reviews of CFTC; and results of our work on other agencies’ performance plans and the Results Act. The criteria we used to assess whether CFTC’s annual performance plan complied with the requirements of the Results Act were the Act itself; OMB guidance on preparing strategic and performance plans (OMB Circular A-11, Part 2); and our guidance on assessing agency performance 7 plans. Specifically, we used the criteria and questions contained in our guidance to determine whether the performance plan met the requirements of the Act, identify strengths and weaknesses in the plan, and assess the plan’s usefulness for congressional oversight. We also assessed the plan’s discussion of actual performance data for fiscal year 1998. We did our work between September 1998 and February 1999 at CFTC in Washington, D.C., in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from CFTC. These comments are discussed near the end of this letter and reprinted in appendix I. The Results Act requires that annual performance plans (1) establish CFTC’s Performance performance goals that define the levels of performance to be achieved; Plan Could Provide a (2) express the performance goals in an objective, quantifiable, and Clearer Picture of measurable form; (3) establish performance measures to be used in assessing progress toward achieving the goals; and (4) provide a basis for Intended Performance comparing actual program results with the established goals. CFTC’s performance plan takes important steps toward addressing these requirements by, among other things, providing performance goals, measures, and targets that reflect CFTC’s mission and long-term strategic goals. However, the plan could provide a clearer picture of intended performance and be of greater use in assessing progress toward achieving intended performance if performance goals, measures, and targets were improved. Specifically, CFTC could improve its plan by (1) making its performance goals and measures more results-oriented and providing 8 goals for all measures; (2) making some performance goals self-measuring and others more objective; (3) replacing, restating, or deleting certain 7 See Agencies’ Annual Performance Plans Under the Results Act: An Assessment Guide to Facilitate Congressional Decisionmaking (GAO/GGD/AIMD-10.1.18, Feb. 1998) and The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance Plans (GAO/GGD-10.1.20, Apr. 1998). 8 Self-measuring goals are expressed objectively and quantifiably and thus do not require the use of additional measures. According to OMB guidance, the Act allows an agency to define a performance goal in a way that is not self-measuring. If a goal is not self-measuring, one cannot determine whether the goal was achieved by objectively comparing actual performance to the goal. Therefore, for a goal that is not self-measuring, the plan must include one or more performance measures that set out specific, measurable values or characteristics. Page 4 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 performance measures; and (4) establishing baselines against which annual targets could be compared. Goals and Measures Could Results-oriented, or outcome, goals and measures provide the clearest picture of intended and actual performance. Such goals and measures Be More Results-Oriented focus on the intended result, effect, or consequence that is expected to and Complete occur from carrying out a program or activity. We recognize that establishing outcome measures is particularly challenging for regulatory agencies as they move from a focus on the activities that they undertake to the results that they are trying to achieve. However, a key shortcoming of CFTC’s performance plan is that it relies predominantly on output 9 measures that describe completed activities, not program results. In our October 1997 testimony before this Subcommittee, we highlighted a 10 similar problem with CFTC’s strategic plan. CFTC’s plan has 31 performance goals and 228 performance measures. About two-thirds of the goals do not have outcome measures to describe intended results. Of the plan’s 228 measures, 184 are output measures that capture program outputs such as the number of meetings attended and number of reports and research projects completed. These measures are weighted toward measuring the quantity of completed activities, rather than the quality, cost, or timeliness of performance outcomes. In addition, 15 of the performance measures are input measures that capture workload inputs such as the number of reports that CFTC received from futures firms and exchanges. Although such information is helpful for defining the amount of work that must be performed, it has limited utility in conveying the expected outcome to result from such work. As previously mentioned, the Results Act is intended to shift the focus of government decisionmaking and accountability away from a preoccupation with completed activities to a focus on the results of such activities. The focus of CFTC’s performance plan on output measures appears to flow from CFTC’s strategy of deriving performance goals from program activities. The agency’s reliance on program activities to develop performance goals may be counterproductive because it appears to focus attention on completed activities and not on the results of such activities. For example, one of the plan’s program activities is to monitor the Internet 9 OMB guidance emphasizes that outcome goals should be included in a performance plan whenever possible. A performance plan should include outcome goals when the related goals are to be achieved in the year covered by the plan. The guidance notes that most plans will supplement outcome goals with output measures. Output measures can be the predominant goals and measures in an annual performance plan under certain circumstances. 10 Results Act: Observations on CFTC’s Strategic Plan (GAO/T-GGD-98-17, Oct. 22, 1997). Page 5 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 and other media for fraudulent activities and other possible violations of the CEA. The performance goal for this activity is to aggressively identify, investigate, and take action against individuals engaged in fraudulent Internet and media activities. The measure for the goal is the number of referrals to enforcement authorities generated from Internet and media monitoring—an output of the activity, not an outcome. According to CFTC officials, the plan’s performance goals are basically restatements of their associated performance activities, which support the plan’s outcome objectives. The officials said that a CFTC task force examined the plan’s 50 program activities last year and reduced the number of activities to 44 by modifying and consolidating several of them. They told us that, in the upcoming year, the task force will examine CFTC’s performance goals and targets with the objective of making them more results-oriented. CFTC could use the narrative sections of its performance plan that discuss the agency’s fiscal year 1998 accomplishments to identify more results- oriented performance goals. For example, the Enforcement Program accomplishment section discusses the effectiveness of its quick-strike ability—the ability to file injunctive actions quickly after detecting fraud— to obtain, among other things, timely injunctive relief and enhance the possibility that customer funds will be recovered. This accomplishment section describes cases, filed within days or weeks of CFTC’s discovering an illegal activity, that stopped fraud at an early stage and preserved customer funds. CFTC’s performance measures could be made more results-oriented by replacing measures, such as reports on activities related to bringing injunctive actions and sanctioning violators, with more outcome-related measures, such as the percentage of quick-strike cases filed within a certain number of days of starting an investigation that resulted in sanctions and the percentage of funds recovered. CFTC could also learn from the plans of other federal financial regulators that are attempting the transition to results-oriented goals. For example, the National Credit Union Administration is developing new outcome performance goals. One of its outcome goals is to ensure that federally insured credit unions are adequately capitalized. A performance goal is to reduce the percentage of federally insured credit unions that are undercapitalized by 10 percent, from 372 to 335. Finally, CFTC could also improve its performance plan and make it more complete by providing performance goals for all of its program activities that have performance measures. CFTC’s plan has 15 program activities Page 6 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 that have no corresponding performance goals but have performance measures. For example, no performance goal exists for the activity of reviewing the adequacy of self-regulatory organization disciplinary actions. Without performance goals, it is not clear what performance is expected. As discussed below, opportunities exist for CFTC to develop self- measuring performance goals for some of these activities and their measures. Some Goals Could Be Made Although not required by the Results Act, CFTC could redefine some performance goals so that they are self-measuring, thereby reducing the Self-Measuring and Others number of measures and the complexity of the plan. Currently, 29 of Could Be Made More CFTC’s 31 performance goals are stated as abstract goals—that is, as goals Objective requiring that specific performance measures be defined to assess progress toward their achievement. Performance goals that can be redefined so that they are self-measuring generally have just one measure or two or more measures that can be combined. CFTC’s plan has nine activities or goals that have only one performance measure, and opportunities exist to redefine some of them in self-measuring terms. For example, the performance goal for the activity of reviewing the adequacy of self-regulatory organization disciplinary actions could be made self-measuring by stating it the following way: On an annual basis, review a defined percentage of self-regulatory organization disciplinary actions to ensure compliance with CFTC standards. This approach, which has been taken by other federal financial regulators, such as the National Credit Union Administration and the Federal Deposit Insurance Corporation, reduces the amount of subjectivity in interpreting performance goals and more clearly defines performance expectations. For example, the Federal Deposit Insurance Corporation has the following performance goal: Market 80 percent of a failing financial institution’s assets based on book value at the time of resolution or within 90 days. CFTC could also improve its performance plan by making its performance goals more objective. According to OMB guidance, to the extent possible, goals should not require subjective considerations. CFTC’s plan has 14 performance goals that require subjective consideration in determining whether they have been met. For example, one such goal is defined as follows: Bring important cases (including matters involving ongoing conduct and complex transactions) aggressively. The performance goal does not define what an important case is or what it means to aggressively bring a case. Although the goal has 11 performance measures—including the numbers of cases filed, counts, and settlements—none clearly define the terms “important” or “aggressively.” Page 7 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 Certain Performance CFTC’s performance plan has 31 performance goals with 228 performance measures to address them. CFTC could replace, restate, or delete Measures Could Be performance measures for certain performance goals to enhance its ability Replaced, Restated, or to assess the progress made in achieving performance goals. Deleted First, CFTC could replace certain performance measures to better capture key aspects of the performance the measures are trying to assess. CFTC’s plan has six performance goals that are partly defined in terms of timeliness but do not have measures that capture timeliness. One such goal is to review every designation application and rule change request within 10 to 45 days. The goal’s performance measures are the number of designation applications processed and the number of contract market rule changes processed. Because neither measure captures the amount of time it takes to process applications or changes, one option for improving the plan is to replace these measures with one measure that captures the percentage of applications and changes processed in 45 days or less. Second, CFTC could restate or delete certain performance measures, because they do not appear to be clearly related to their performance goals and/or appear to be unduly affected by external factors. For example, the number of active futures markets is used to measure, in part, two goals: (1) Identify traders who can influence futures prices and (2) determine whether traders are influencing futures prices. Because the number of futures markets is largely determined by exchanges and other external factors, this number has little direct bearing on the two goals and, thus, could be deleted as a measure. Another example of such a measure is the number of requests for assistance from foreign authorities, which is used, in part, to measure the following goal: Maximize the enforcement program’s effectiveness by cooperating with domestic and foreign authorities in gathering information. This is an input measure that captures action taken by foreign regulators, not CFTC. Third, CFTC could restate or delete certain performance measures because the measures might have limitations that preclude them from accurately capturing intended performance or might promote unintended consequences. For example, one of the plan’s performance goals is to conduct important investigations and refer potential violations to other authorities, as appropriate. The performance measures for this goal include the number of documents obtained through subpoena or inspection, number of witnesses from whom testimony was taken, and number of witnesses interviewed. These output-oriented measures could provide an incentive for staff to conduct more interviews, take more testimonies, and obtain more documents than necessary, which could add Page 8 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 cost and time to investigations without necessarily contributing commensurately to their success. Baselines Could Be Although CFTC’s performance plan includes annual targets for performance goals, it could make such information more useful by Established Against Which providing baselines, or a context, for assessing the reasonableness and to Compare Annual Targets appropriateness of such targets. As we found in our review of other performance plans, agencies that go beyond the requirements of the Results Act and include baseline or trend data for their goals provide a more informative basis for assessing expected performance. CFTC’s performance plan provides many quantitative annual targets for each fiscal year from 1998 through 2000. Many targets are set at the same performance level as or at a lower performance level than in prior fiscal years without an explanation. For example, the performance measures for assessing sanctions under CFTC’s first strategic goal are the same in each year covered by the plan for the number of cease-and-desist orders, registration sanctions, and trading prohibitions. However, the bases for setting the specific targets and the contributions of these targets to the 11 outcome objective are not readily apparent. Without this contextual information, it is not clear whether CFTC’s output-oriented performance targets are reasonable and appropriate. Consistent with the Results Act, CFTC’s performance plan attempts to Mission, Goals, and show the relationship between the agency’s annual performance goals and Activities Could Be its fiscal years 1997 through 2002 mission and strategic goals. To do so, the Better Connected plan uses tables that connect each strategic goal to its accompanying set of performance goals, measures, and annual targets. The plan also ranks CFTC’s outcome objectives by dollars budgeted, which is a starting point for providing useful information about its priorities. However, the plan could better connect mission, goals, and activities by more fully demonstrating how CFTC will chart annual progress toward achieving its long-term strategic goals. The performance plan’s presentation of many separate program areas; program activities; and performance goals, measures, and targets makes it difficult to link CFTC’s mission and strategic goals to performance goals across the entire agency. The plan’s 31 performance goals and 228 performance measures support 3 strategic goals and 9 strategic objectives, covering 5 program areas. Although the plan’s ranking of outcome 11 The outcome objective is to foster futures and option markets that accurately reflect the forces of supply and demand for the underlying commodity and are free of disruptive activity. Page 9 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 objectives offers a useful perspective on CFTC’s priorities, given the high level of complexity, it is difficult to (1) identify the agency’s key priorities among the many goals and measures, (2) differentiate efforts to meet these priorities, and (3) understand what will be achieved if all of the performance goals are met. Similarly, as we found with other agency performance plans, CFTC’s plan associates one performance goal with multiple program activities and strategic goals. For example, the performance goal—assess sanctions that are remedial and deter violators—is associated with three different program activities and three different strategic goals. Additionally, the measures and targets for this performance goal differ with each program activity and strategic goal. Associating one performance goal with multiple program activities and strategic goals makes it difficult not only to determine whether all activities are substantially covered but also to understand how specific program activities are intended to contribute to CFTC’s strategic goals. The Results Act seeks to ensure that crosscutting goals of federal Crosscutting Efforts programs are consistent; strategies are mutually reinforcing; and, as That Involve Other appropriate, progress is assessed through the use of common performance Agencies Could Be measures. By accomplishing these goals, agencies could avoid wasting scarce resources and could better ensure that the overall effectiveness of More Fully Addressed federal efforts is maximized. OMB guidance tasks performance plans with identifying those performance goals that are being mutually undertaken with other federal agencies in support of programs or activities of a 12 crosscutting nature. It encourages agencies to go beyond coordination and develop common performance goals and measures for related programs. CFTC’s plan recognizes the need to address crosscutting efforts. However, the agency could address such efforts in more substantive terms if it worked with the cognizant federal agencies to develop performance goals and measures that better reflect the nature and extent of their common efforts. CFTC’s performance plan briefly discusses CFTC’s need to work with other U.S. financial regulators through, among other means, the 13 President’s Working Group on Financial Markets. However, the related 12 OMB Circular A-11 guidance states that, at a minimum, the performance plan should indicate those programs or activities that are being undertaken with other agencies to achieve a common purpose or objective. An agency should also review the performance plans of other agencies participating with it in a crosscutting program or activity to ensure that goals and measures are consistent and harmonious. 13 The President’s Working Group on Financial Markets was created following the October 1987 stock market crash to address issues concerning the competitiveness, integrity, and efficiency of the Page 10 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 performance goals and measures do not substantively address the type of crosscutting performance that the Working Group was created to address. For example, CFTC’s performance goal covering the Working Group is for CFTC to contribute to the performance of the group. The measure for this goal is the number of meetings attended, and the fiscal year 2000 target is two meetings. Neither the goal nor its measure captures the substantive work of the Working Group’s efforts. As supported by OMB guidance, CFTC could strengthen its performance plan by participating with the members of the Working Group and other federal regulators involved in crosscutting programs to develop common performance goals and measures. For example, the continued growth and development of the over-the-counter (off-exchange) derivatives market has raised several potential regulatory concerns that affect CFTC and other members of the Working Group. In addition, the potential need to develop a financial markets contingency plan to address the “Year 2000” computer- 14 dating problem could involve coordination among CFTC and other federal financial regulators. Additionally, CFTC’s performance plan could be expanded to include performance goals and measures related to other crosscutting efforts, such as those identified in its fiscal year 2000 budget justification. These efforts include CFTC’s sharing of information with other financial regulators; working with the U.S. Department of Agriculture on a risk-management education program; contributing to a Department of the Treasury initiative that encourages global financial stability; and engaging in cooperative enforcement efforts with the Department of Justice, Federal Bureau of Investigation, Federal Reserve Board, Federal Trade Commission, Securities and Exchange Commission, and U.S. Postal Inspection Service. financial markets. The Secretary of the Treasury chairs the group, and other members include the chairs of CFTC, the Federal Reserve System, and the Securities and Exchange Commission. 14 The Year 2000 computer-dating problem relates to the need for computer systems to be changed to accommodate dates beyond the year 1999. Page 11 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 Consistent with the Results Act, CFTC’s performance plan provides Strategies and important information on how strategies and resources will be used to Resources Used to 15 achieve its goals. However, the plan’s discussion could be expanded to Achieve Goals Could better enable congressional and other decisionmakers to judge the reasonableness and appropriateness of CFTC’s strategies and anticipated Be Discussed in resource deployment. Specifically, the plan could be strengthened by (1) Greater Detail describing more fully the strategies needed to achieve performance goals, including those for addressing both internal management challenges and external factors, and (2) discussing in more detail the resources required to achieve each performance goal. The Strategies Needed to Consistent with the Results Act and OMB guidance, CFTC’s performance plan attempts to address the strategies that CFTC will use to achieve its Achieve Goals Could Be performance goals. Aligning its discussion with CFTC’s strategic goals, the Described More Fully plan briefly describes the major activities, or operational processes, of each program in relation to its performance goals and measures. In a few cases, the plan also discusses skills or technologies that programs will use in relation to performance goals and measures. The plan could be made more useful by providing additional information on the skills and, if appropriate, technologies used in connection with operational processes to achieve program goals. CFTC’s fiscal years 1997-2002 strategic plan identifies several internal management challenges, but CFTC’s performance plan does not include a strategy—consisting of specific goals and measures—to address them. These challenges include diminishing resources, recruiting and retaining qualified professionals, remaining abreast of current technology, and remaining educated and informed as innovation changes the industry. According to OMB guidance, performance goals for management problems should be included in the annual plan, particularly goals for those problems whose resolution is mission-critical and which could impede achievement of program goals. To better respond to the intent of the Results Act, CFTC could add agencywide performance goals to the plan to address these challenges or incorporate these challenges within the plan’s existing structure of performance goals and measures. For example, CFTC could add a performance goal that covers recruiting and retaining a highly qualified workforce to handle complex issues facing the agency. CFTC’s performance plan also discusses some of the external factors that can affect its performance goals, but it does not identify the strategies that 15 Typically, strategies cover an agency’s operational processes, skills, and technology, while resources cover human, capital, funding, and other resources. Page 12 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 the agency plans to use to address them. In our testimony on CFTC’s fiscal year 1997-2002 strategic plan, we highlighted this area, noting that CFTC could strengthen its plan by describing how external factors are linked to particular goals and how a particular goal could be affected by external factors. Although the Results Act does not require performance plans to discuss the specific impact of external factors on achieving performance goals, such information can provide a better context for assessing CFTC’s expected performance. Moreover, OMB guidance notes that agencies may choose to include information on external factors in their annual plan when the factors may bear directly on goal achievement for the fiscal year covered by the plan. Resources Required to According to OMB guidance, a performance plan should discuss the resources that will be applied to achieve each of the agency’s performance Achieve Performance Goals goals. Using tables and graphics, CFTC’s plan shows the amount of budget Could Be Discussed in funding and the number of full-time-equivalent employees that will be Greater Detail needed to achieve each strategic goal and related outcome objectives. CFTC’s plan could be improved by addressing OMB guidance that also calls for plans to describe the resources required to achieve each performance goal. At the October 1998 hearing, the CFTC Chairperson testified that the agency is working to improve its capacity to account for budgetary resources by activity, outcome objective, and performance goal. She said that the agency’s financial reporting system was being modified to provide budget data on a program and performance goal basis. She also said that CFTC plans to link its financial reporting system to the activities of its performance plan and, if successful, use it to formulate its fiscal year 2001 budget. Page 13 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 Although required by the Results Act and OMB guidance, CFTC’s fiscal The Means for year 2000 performance plan does not describe the procedures that the Verifying and 16 17 agency will use to verify and validate that performance information is Validating sufficiently complete, accurate, and consistent. These procedures can include internal controls over data collection, maintenance, and entry. Performance External assessments, such as audits, evaluations, and peer reviews, are Information and other important procedures for verifying and validating performance Ensuring Its Reliability information. In addition, the plan does not, as required by the Act, discuss Should Be Described the extent to which the performance information and the means for collecting, maintaining, and analyzing it are reliable. CFTC’s performance plan should be expanded to address these requirements. To assess progress toward achieving its goals, CFTC will need to collect information on the over 200 performance measures in its plan. For some of these measures, the amount of information to be collected is voluminous and covers activities across CFTC headquarters and regional offices. Errors can occur in collecting, maintaining, processing, and reporting such information—potentially introducing bias and resulting in inaccurate estimates of program performance. As a result, CFTC should have procedures for ensuring that its performance information is free of significant levels of error and that bias is not introduced. Such procedures can include internal controls over data collection, maintenance, and entry as well as audits, evaluations, and peer reviews. At the October 1998 hearing, the CFTC Chairperson testified that the agency is adapting its Quarterly Objectives review—a public document that briefly describes each program’s priorities and accomplishments—to match the annual performance plan’s goals, measures, and targets. According to the Chairperson, the goal is to use the performance data in its Quarterly Objectives review to evaluate whether CFTC is meeting its performance plan targets. 16 Verify means to check or test performance data to reduce the risk of using data that contain significant errors. 17 Validate means to ensure that the data are free of significant levels of error and that bias is not introduced. Page 14 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 Although not required by the Results Act or OMB guidance, CFTC’s Past Accomplishments performance plan presents its fiscal year 1998 accomplishments. CFTC Could Be Linked to could improve the plan’s clarity by linking past accomplishments to Performance Goals to performance goals. CFTC’s plan provides a narrative section under each strategic goal that discusses the agency’s fiscal year 1998 accomplishments Enhance the Plan’s by program. For example, in the first strategic goal section, the plan Clarity discusses, among other things, the major accomplishments of the Market Surveillance, Analysis, and Research Program. These accomplishments include conducting daily surveillance of 157 futures markets, collecting and analyzing about 7,980 large trader reports, and preparing about 2,538 weekly surveillance reports. In some cases, CFTC links the discussion of its past accomplishments to specific outcome objectives. However, CFTC does not link the accomplishments to the performance goals described in its fiscal year 1999 annual performance plan. Without this linkage, it is difficult to determine the progress that CFTC made in achieving its performance goals and, in turn, strategic goals. Our observations on CFTC’s fiscal year 2000 annual performance plan are Conclusions intended to assist the agency in improving future plans so that they can be more valuable tools for congressional and other decisionmakers. Although we identified opportunities for CFTC to improve its plan—as we have done for many other federal agencies—the Results Act anticipated that the process of developing an effective planning process and performance plan could take several planning cycles. Additionally, the agency’s ongoing efforts reflect a commitment to fully complying with the Act. In the upcoming year, CFTC plans to examine its performance goals to make them more results-oriented, use performance data to evaluate whether performance plan targets are being met, and improve its accounting system to track budgetary resources by performance goals. To further enhance its plan, CFTC could focus additional attention on (1) improving its performance goals, measures, and targets to provide a clearer picture of intended performance; (2) demonstrating more fully how progress toward achieving strategic goals will be charted; (3) addressing crosscutting efforts in more substantive terms; (4) linking strategies and resources more clearly; and (5) providing confidence that performance data will be sufficiently reliable. CFTC could also enhance the plan’s clarity by linking past accomplishments to performance goals. Written comments from CFTC on a draft of this report are contained in Agency Comments appendix I. CFTC said it appreciated the suggestions and recommendations made in our report and by others. It also said it is Page 15 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan B-281743 committed to improving its future plans to make them as useful as possible. We are sending copies of this letter to Representative Gary A. Condit, the Ranking Minority Member of your Subcommittee; and the Honorable Brooksley Born, Chairperson of CFTC. We will also make copies available to others on request. This report was prepared under the direction of Cecile Trop, Assistant Director. Other major contributors included Sharon Caudle and Richard Tsuhara. Please contact me at (202) 512-8678 or Ms. Trop at (312) 220-7600 if you or your staff have any questions. Sincerely yours, Richard J. Hillman Associate Director, Financial Institutions and Markets Issues Page 16 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan Page 17 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan Appendix I Comments From the Commodity Futures Trading Commission Page 18 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan Appendix I Comments From the Commodity Futures Trading Commission Page 19 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan Related GAO Products The Results Act: Observations on the Federal Reserve’s 1998-99 Biennial Performance Plan (GAO/GGD-99-9R, Nov. 9, 1998). The Results Act: Observations on NCUA’s Annual Performance Plan (GAO/GGD-98-192R, Sept. 15, 1998). The Results Act: Observations on FDIC’s Annual Performance Plan (GAO/GGD-98-190R, Sept. 15, 1998). The Results Act: Observations on OCC’s Annual Performance Plan (GAO/GGD-98-189R, Sept. 15, 1998). Managing for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998). Results Act: Observations on Treasury’s Fiscal Year 1999 Annual Performance Plan (GAO/GGD-98-149, June 30, 1998). Managing for Results: Experiences of Selected Credit Programs (GAO/GGD-98-41, Feb. 19, 1998). Managing for Results: Agencies’ Annual Performance Plans Can Help Address Strategic Planning Challenges GAO/GGD-98-44, Jan. 30, 1998). The Results Act: Observations on Draft Strategic Plans of Five Financial Regulatory Agencies (GAO/T-GGD-97-164, July 29, 1997). Managing for Results: Regulatory Agencies Identified Significant Barriers to Focusing on Results (GAO/GGD-97-83, June 24, 1997). The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997). Page 20 GAO/GGD-99-51 CFTC’s Fiscal Year 2000 Annual Performance Plan Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Order by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 th th 700 4 St. NW (corner of 4 and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touch-tone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send e-mail message with “info” in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested (233595)
Results Act: Observations on CFTC's Fiscal Year 2000 Annual Performance Plan
Published by the Government Accountability Office on 1999-03-15.
Below is a raw (and likely hideous) rendition of the original report. (PDF)