United States General Accounting Office National Security and Washington, D.C. 20548 International Affairs Division B-282909 July 20, 1999 The Honorable Dick Armey Majority Leader House of Representatives The Honorable Dan Burton Chairman, Committee on Government Reform House of Representatives The Honorable Fred Thompson Chairman, Committee on Governmental Affairs United States Senate Subject: Observations on the U.S. Agency for International Development’s Fiscal Year 2000 Performance Plan As you requested, we have reviewed and evaluated the fiscal year 2000 performance plans for the 24 Chief Financial Officers (CFO) Act agencies that were submitted to Congress as required by the Government Performance and Results Act of 1993 (Results Act). Enclosure I to this letter provides our observations on the fiscal year 2000 performance plan for the U.S. Agency for International Development (USAID). Enclosure II lists management challenges we and USAID’s Inspector General identified that face the agency and the applicable goals and measures in the fiscal year 2000 annual performance plan. Our objectives were to (1) assess the usefulness of the agency’s plan for decisionmaking and (2) identify the degree of improvement the agency’s fiscal year 2000 performance plan represents over the fiscal year 1999 plan. Our observations were generally based on the requirements of the Results Act, guidance to agencies from the Office of Management and Budget (OMB) for developing the plan (OMB Circular A-11, Part 2), our previous reports and knowledge of USAID’s operations and programs, and our observations on USAID’s fiscal year 1999 performance plan. Our summary report on the CFO Act agencies’ fiscal year 2000 plans 1 contains a complete discussion of our objectives, scope, and methodology. 1 Managing for Results: Opportunities for Continued Improvements in Agencies’ Performance Plans (GAO/GGD/AIMD-99-215, July 20, 1999). Page 1 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan B-282909 As agreed, unless you announce the contents of this letter earlier, we plan no further distribution until 30 days from the date of the letter. The major contributors to this report are listed in enclosure III. Please call me on (202) 512-4128 if you or your staff have any questions. Sincerely yours, Benjamin F. Nelson Director, International Relations and Trade Issues Enclosures - 3 Page 2 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 USAID’s fiscal year 2000 annual performance plan provides a general overview of intended performance across the agency and a general discussion of the strategies and resources the agency will use to achieve its goals. However, USAID needs to develop a clearer linkage between broad development goals and specific USAID country program goals and results. For example, although the plan cites increased reliance on private markets as one of USAID’s long-term performance goals, the plan provides no information as to how USAID strategies or programs support this goal. USAID also needs to continue its efforts to improve the quality of data used to measure performance. Figure 1 highlights the plan’s major strengths and key weaknesses. Figure 1: Major Strengths and Key Weaknesses of USAID’S Fiscal Year 2000 Annual Performance Plan Major Strengths • Contains results-oriented goals and quantifiable performance measures, • Generally discusses strategies and resources for achieving intended performance, and • Discusses data limitations and external factors affecting results. Key Weaknesses • Does not develop clear linkage between agency and individual country goals, • Does not identify the full range of other agency and other donor programs that may contribute to achieving overall goals, and • Continues to rely on weak financial and program results data. USAID’S fiscal year 2000 performance plan represents a moderate improvement over the fiscal year 1999 plan in that it shows some progress in addressing the weaknesses that we identified in our assessment of the fiscal year 1999 plan. In reviewing the fiscal year 1999 plan, we observed that (1) most of the goals and measures were broadly defined to reflect the overall goals of the international donor community, making it difficult to assess the results of USAID’s specific activities; (2) the plan did not provide detail on USAID’s specific strategies and programs, the external factors involved, or the specific resources to be provided; and (3) the plan did not discuss the reliability of performance information that it will use to demonstrate the linkage of its programs to results. Among improvements in the fiscal year 2000 plan, we note that (1) USAID has provided greater detail in discussing agency goals and performance indicators by geographic regions, although not comprehensively by country; (2) USAID has in some areas tried to provide data linking performance indicators to countries having USAID programs, rather than just providing overall regional data; and (3) USAID has provided a more thorough discussion of data limitations and external factors affecting the results of its programs. Page 3 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 USAID’s Performance Plan Provides a General Overview of Intended Performance Across the Agency USAID’s plan consists of 6 strategic goals and 1 management goal supported by a total of 25 long-term performance goals. For each of these 25 performance goals, USAID has presented one or more quantitative performance indicators and a table showing, for four geographic 1 groupings, the baseline measurement, the most recent available data, the fiscal year 2000 goal, and the strategic plan goal for fiscal year 2007. For example, under the strategic goal, “The World’s Environment Protected for Long-term Sustainability,” USAID has six performance goals, one of which is to reduce the deforestation rate in tropical forests and improve the management of natural resources and tree systems. Progress under this goal is to be measured using an indicator identifying the average annual change in total forest area, both in land area and percent change – with specific goals cited for each geographic region for fiscal years 2000 and 2007. As we reported last year, many of these goals reflect the broadly defined goals of the international donor community but will be difficult to relate closely to USAID programs. For example, under USAID’s strategic goal of encouraging broad-based economic growth and agricultural development, USAID identifies increased reliance on private markets as a long- term performance goal. Progress under this goal is to be assessed based on performance indicators relating to foreign direct investment, trade volume, and the Economic Freedom Index compiled by the Heritage Foundation. USAID provides no information discussing USAID strategies or programs under this goal, nor does it mention the possible roles that other U.S. agencies – such as the Overseas Private Investment Corporation and the U.S. Export-Import Bank – might play in contributing to this goal. USAID recognizes that its goals continue to be broadly stated development goals and that its programs may not always be key factors in accomplishing development goals; but USAID states that it is seeking to find ways to relate its specific efforts to progress in reaching these goals. This year’s plan presents an improved discussion of USAID’s goals and related performance indicators by geographic regions. However, USAID’s experiences and programs differ widely, even within the same geographic area, and overall regional performance goals and indicators do not capture the actual goals and results intended for individual aid recipient countries or USAID operational units. Under its discussion of each of these 25 performance goals, USAID’s plan has provided selected descriptions by region of USAID’s programs, but these are not comprehensive descriptions of USAID programs by country, with specific performance goals, nor do they mention funding levels. 1 Sub-Saharan Africa; Asia, Near East, and North Africa; Europe and the former Soviet Union; and Latin America and the Caribbean. Page 4 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 In addition, even when USAID does discuss specific programs, it does not routinely use the performance indicators it has identified for the long-term goal. For example, with respect to USAID’s performance goal of improving urban population access to safe drinking water and sanitation services, the plan mentions that USAID has significant programs in Egypt, but it does not use the performance indicator it presents in its accompanying table – percent of urban population with access to safe water and to sanitation services. Instead, for Egypt, USAID notes only that 5.49 million people will be served by USAID-funded sewage and wastewater treatment facilities and that over 1 billion liters of water per day will be treated. This year’s plan recognizes the many ways in which USAID’s strategic goals interrelate and reinforce each other. The plan also describes how its goals support the overall U.S. International Affairs Strategic Plan (prepared by the State Department) covering 19 foreign policy goals, although it does not present specific information at a country or regional level. The plan also includes a strategic goal related to improving the management of USAID programs. USAID acknowledges the challenges it faces in improving its management information systems, and it notes that it will be implementing specific improvements over the next several years. Overall, the fiscal year 2000 performance plan indicates moderate progress in addressing the weaknesses that we identified in our assessment of the fiscal year 1999 plan as it relates to providing a clear picture of intended performance across USAID. In reviewing the fiscal year 1999 plan, we observed that (1) most of the results measures set out in the plan were not directly linked to USAID’s own specific activities and (2) the plan did not articulate a strategy for coordination with nonforeign affairs agencies. Among improvements in the fiscal year 2000 plan are (1) an enhanced discussion of USAID’s strategic goals and performance indicators by geographic regions and (2) a clearer presentation of how USAID’s strategic goals interrelate and support the U.S. International Affairs Strategic Plan. However, the plan continues to need a closer linkage established between its overall development goals and the expected results of specific USAID programs. USAID’s Performance Plan Provides a General Discussion of the Strategy and Resources the Agency Will Use to Achieve Performance Goals USAID’s fiscal year 2000 performance plan demonstrates an enhanced effort to identify how USAID’s 25 performance goals support its 7 strategic and management goals and how each of these 7 goals is supported by USAID’s fiscal year 2000 budget request. It includes a table indicating, for example, that $69.8 million would be used in the newly independent states of the former Soviet Union to support the strategic goal of “Building Democracy and Page 5 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 Governance.” The plan does not, however, include a breakdown into amounts targeted for the 25 performance goals that USAID identifies as supporting its 7 strategic goals. The plan also presents more information on how USAID’s resources support its goals and the goals of the U.S. International Affairs Strategic Plan. For example, the USAID plan shows that USAID’s humanitarian programs totaling $184 million for fiscal year will combine with $450 million in Public Law 480 Title II funds in support of the U.S. International Affairs Strategic Plan’s goal of providing humanitarian responses in short-term crises. As previously noted, USAID’s strategic development goals reflect the goals of the international donor community and are mutually reinforcing. However, the plan does not specifically discuss the activities of other international donors or of other U.S. government agencies, such as the U.S. Overseas Private Investment Corporation, the U.S. Export-Import Bank, or the U.S. Department of Agriculture. The plan discusses how USAID has revised certain performance goals and indicators in order to serve its strategic goals, but it does not generally report on the changes USAID has made over the past year to its program approaches in specific regions or countries. Some discussion of the results of USAID’s recent program evaluations would contribute to a more specific description of changes in program approaches, as well as provide some information on how specific strategies or programs can lead to intended results. A significant improvement in this year’s plan is its recognition of the many external factors that can affect the outcomes of USAID programs. In particular, USAID cites the impact of the Asian financial crisis and its international consequences as an important external variable. It notes, however, that many of the poorest countries are not closely connected with the global economy and may not be significantly affected by the financial crisis. As noted previously, USAID explicitly recognizes that, for some goals, it plays a relatively small role and that progress achieved cannot be directly linked to USAID efforts. The agency states that it will continue to try to develop measures that can plausibly associate progress with the impact of USAID programs, but it also notes that it will assess the costs, both in resources and in agency focus, of efforts to improve data and measurement indicators. With respect to agency resources needed to achieve the plan’s performance goals, the plan notes the limitations of USAID’s budget and hiring capacity and states that the agency will seek to improve its capabilities through a strategy of training and selective hiring, as attrition allows. Also, with respect to USAID’s goal of improving the management and delivery of its development assistance resources, this year’s plan acknowledges the continuing problems of Page 6 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 the agency’s management information systems. It notes that a new, commercial, off-the-shelf core accounting system will be implemented by fiscal year 2000. Overall, the fiscal year 2000 performance plan indicates some progress in addressing the weaknesses that we identified in our assessment of the fiscal year 1999 performance plan as it relates to providing a complete discussion of strategies and resources USAID will use to achieve its performance goals. In reviewing the fiscal year 1999 plan, we noted that (1) USAID did not discuss all of its goals with a consistent level of detail that would convey the nature and magnitude of its efforts and (2) USAID only partially discussed key external factors or other donors’ related efforts to achieve the same goal. Among improvements in the fiscal year 2000 plan are (1) an enhanced effort to describe its 25 performance goals and identify how they support its 7 strategic goals and (2) a clearer acknowledgement of the numerous external factors that can affect the outcomes of USAID goals. USAID’s Performance Plan Provides Limited Confidence That the Agency’s Performance Information Will Be Credible USAID’s performance plan provides limited confidence that the agency has the capacity to obtain credible, results-oriented, program performance information. USAID acknowledges that continued data improvements are needed in order to achieve a uniform, high-quality database permitting cost-benefit analyses of USAID’s overall development assistance program. The performance plan lists several limitations that USAID faces in seeking to improve data quality. First, since the agency is dependent on international organizations and thousands of partner institutions for data, USAID does not have full control over how data are collected, reported, or verified. Second, programs aimed at creating new capacity are difficult to measure, particularly at their initial stages. Finally, many information sources have different reporting periods, and the data are often not available for a year or more afterwards. In addition, sometimes these data are significantly adjusted after publication. The agency indicates that it is seeking to find ways to improve data quality for some of its performance indicators. For example, for its goal of reducing by 10 percent the number of deaths due to infectious diseases of major public health importance by 2007, USAID reports that no data are available on a country-specific basis and that it will be working with the World Health Organization to improve the data by 2002. In other instances, USAID indicates that it will seek to assure collection of relevant data by conducting periodic surveys in USAID-assisted countries. We recently reported that USAID’s internal accounting and information systems do not have the capacity to generate reliable data to support its performance plan and to produce credible Page 7 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 2 performance reports. USAID’s financial management system does not meet the federal financial management systems requirements, and material weaknesses in internal controls impair the integrity of its financial information. The agency has indicated that it is committed to developing a financial management system that will meet federal standards, but the USAID Inspector General has reported that the agency has made only limited progress in correcting its system deficiencies. USAID has acknowledged its problems with data quality and has made some improvements. If information is collected by other organizations, the agency requires staff to periodically confirm its usefulness for USAID’s purposes. USAID has included a discussion in the performance plan of its system of intra-agency checks and balances in reviewing and interpreting data. The agency has also begun operational changes that it believes will eventually lead to implementation of managerial cost accounting, allowing the full cost of outputs at the intermediate results level to be calculated. The fiscal year 2000 plan recognizes the weaknesses that we identified in our assessment of the fiscal year 1999 performance plan as it relates to providing full confidence that the agency’s performance information will be credible and makes specific commitments or shows actual attempts to address those weaknesses. However, real progress is not yet evident, and the fiscal year 2000 plan in this regard represents little, if any, improvement over the 1999 plan. In reviewing the fiscal year 1999 plan, we observed that the plan (1) addressed the issue of verification and validation of performance data only very generally and did not discuss the issue of data reliability, (2) did not specifically state how the agency planned to achieve its primary financial improvement goal of “making financial and program results information more accurate and readily available for decisionmakers,” and (3) acknowledged significant data limitations but did not adequately address the problem of verifying the credibility of the external performance information it needs. As previously discussed, these issues are still evident. In addition, USAID’s Inspector General found many instances of unsupported and inaccurate data in a recent review of the agency’s internal performance reports. Among improvements in the fiscal year 2000 plan are a “methodology notes” section used to explain how performance indicators were calculated. Although this detail does not ensure reliable data, it does allow the reader to determine data limitations. 2 Performance and Accountability Series: Major Management Challenges and Program Risks: Agency for International Development (GAO/OCG-99-16, January 1999). Page 8 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure I Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000 Other Observations on USAID’s Implementation of Performance-Based Management USAID notes that it will be revising its strategic plan during 1999 and, accordingly, some of its performance goals will be changed. We note that if these revisions in strategic and performance goals are significant, some clarification may be needed on how USAID will be expected to report on revised performance goals. Agency Comments On May 4, 1999, we obtained comments from USAID officials, including the Deputy Assistant Administrator, Bureau for Policy and Program Coordination, on a draft of our analysis of USAID’s fiscal year 2000 annual performance plan. These officials generally agreed with our analysis. In addition, with respect to our comments regarding the need to link agency goals with individual country goals, they noted that USAID is currently developing methods of improving the linkage among the Annual Performance Plan, the Annual Performance Report, and the country coverage provided in USAID’s Congressional Presentation. They also noted that they are exploring ways to improve the quality of data used to assess performance. Page 9 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure II Management Challenges The key management challenge confronting USAID is the need to implement reliable management information systems. USAID needs to institute both a comprehensive information management system and a financial management information system. Table II.1: Management Challenges in USAID’s Fiscal Year 2000 Performance Plan Management challenge Applicable references in the fiscal year 2000 annual performance plan USAID has not implemented a comprehensive The plan identifies the goal of improving USAID’s information management system. management of in formation resources through the acquisition of commercial, off-the-shelf solutions for addressing property and human resource management requirements. The plan notes that these are to be deployed by fiscal year 2000, but it provides no details on the resources being targeted at this goal. USAID’s financial management information is The plan states that the agency has decided that the unreliable. least costly, lowest risk approach to fulfill the financial management system requirements is to purchase a commercial, off-the-shelf core accounting system. By fiscal year 2000, USAID expects to implement the new system in Washington, D.C., followed by implementation in the field in 2001. USAID could face disruptions due to the Year 2000 The goal is to have completed the work to assure technology update problem that all mission-critical systems and their supporting infrastructure are Y2K compliant by the beginning of FY 2000. The plan states that, if necessary, the agency will undertake emergency remediation and be prepared to implement continuity of operations plans for any essential functions that might be interrupted. In addition, the agency will complete remediation of approximately 14 corporate applications systems previously identified as not mission critical. The plan also states that the Y2K program will receive the highest priority for allocation of information management resources. However, it is important to note that, according to the agency’s own reporting, it is far behind the government’s required schedule. No measures or indicators are discussed in the plan. Information security None. The plan does not address this. However, it states that the confidentiality, integrity, and availability of data will be improved with enhanced information security management and reporting. Inspector General’s areas of concern Applicable references in USAID’s fiscal year 2000 annual performance plan The quality of USAID’s performance data is None. The plan identifies selected efforts to improve questionable. data quality. Page 10 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Enclosure III GAO Contacts and Staff Acknowledgments GAO Contact Benjamin F. Nelson, (202) 512-4128 Acknowledgments In addition to the contact named above, Jess T. Ford, Virginia C. Hughes, Elizabeth Johnson, Sue Piyapongroj, and Linda P. Garrison made key contributions to this product. (711418) Page 11 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Page 12 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Page 13 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan Page 14 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Observations on the U.S. Agency for International Development's Fiscal Year 2000 Performance Plan
Published by the Government Accountability Office on 1999-07-20.
Below is a raw (and likely hideous) rendition of the original report. (PDF)