oversight

Observations on the U.S. Agency for International Development's Fiscal Year 2000 Performance Plan

Published by the Government Accountability Office on 1999-07-20.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States General Accounting Office                                                                  National Security and
Washington, D.C. 20548                                                                            International Affairs Division




                 B-282909

                 July 20, 1999

                 The Honorable Dick Armey
                 Majority Leader
                 House of Representatives

                 The Honorable Dan Burton
                 Chairman, Committee on Government Reform
                 House of Representatives

                 The Honorable Fred Thompson
                 Chairman, Committee on Governmental Affairs
                 United States Senate

                 Subject: Observations on the U.S. Agency for International Development’s Fiscal Year 2000
                 Performance Plan

                 As you requested, we have reviewed and evaluated the fiscal year 2000 performance plans for
                 the 24 Chief Financial Officers (CFO) Act agencies that were submitted to Congress as
                 required by the Government Performance and Results Act of 1993 (Results Act). Enclosure I
                 to this letter provides our observations on the fiscal year 2000 performance plan for the U.S.
                 Agency for International Development (USAID). Enclosure II lists management challenges we
                 and USAID’s Inspector General identified that face the agency and the applicable goals and
                 measures in the fiscal year 2000 annual performance plan.

                 Our objectives were to (1) assess the usefulness of the agency’s plan for decisionmaking and
                 (2) identify the degree of improvement the agency’s fiscal year 2000 performance plan
                 represents over the fiscal year 1999 plan. Our observations were generally based on the
                 requirements of the Results Act, guidance to agencies from the Office of Management and
                 Budget (OMB) for developing the plan (OMB Circular A-11, Part 2), our previous reports and
                 knowledge of USAID’s operations and programs, and our observations on USAID’s fiscal year
                 1999 performance plan. Our summary report on the CFO Act agencies’ fiscal year 2000 plans
                                                                                             1
                 contains a complete discussion of our objectives, scope, and methodology.


                 1
                  Managing for Results: Opportunities for Continued Improvements in Agencies’ Performance Plans (GAO/GGD/AIMD-99-215,
                 July 20, 1999).




                 Page 1                                              GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
B-282909


As agreed, unless you announce the contents of this letter earlier, we plan no further
distribution until 30 days from the date of the letter. The major contributors to this report are
listed in enclosure III. Please call me on (202) 512-4128 if you or your staff have any
questions.

Sincerely yours,




Benjamin F. Nelson

Director, International Relations and Trade Issues

Enclosures - 3




Page 2                                    GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I

Observations on the U.S. Agency for
International Development's
Performance Plan for Fiscal Year
2000
USAID’s fiscal year 2000 annual performance plan provides a general overview of intended
performance across the agency and a general discussion of the strategies and resources the
agency will use to achieve its goals. However, USAID needs to develop a clearer linkage
between broad development goals and specific USAID country program goals and results. For
example, although the plan cites increased reliance on private markets as one of USAID’s
long-term performance goals, the plan provides no information as to how USAID strategies or
programs support this goal. USAID also needs to continue its efforts to improve the quality of
data used to measure performance. Figure 1 highlights the plan’s major strengths and key
weaknesses.

Figure 1: Major Strengths and Key Weaknesses of USAID’S Fiscal Year 2000 Annual
Performance Plan

Major Strengths
• Contains results-oriented goals and quantifiable performance measures,
• Generally discusses strategies and resources for achieving intended performance, and
• Discusses data limitations and external factors affecting results.

Key Weaknesses
• Does not develop clear linkage between agency and individual country goals,
• Does not identify the full range of other agency and other donor programs that may
contribute to achieving overall goals, and
• Continues to rely on weak financial and program results data.

USAID’S fiscal year 2000 performance plan represents a moderate improvement over the
fiscal year 1999 plan in that it shows some progress in addressing the weaknesses that we
identified in our assessment of the fiscal year 1999 plan. In reviewing the fiscal year 1999
plan, we observed that (1) most of the goals and measures were broadly defined to reflect the
overall goals of the international donor community, making it difficult to assess the results of
USAID’s specific activities; (2) the plan did not provide detail on USAID’s specific strategies
and programs, the external factors involved, or the specific resources to be provided; and (3)
the plan did not discuss the reliability of performance information that it will use to
demonstrate the linkage of its programs to results. Among improvements in the fiscal year
2000 plan, we note that (1) USAID has provided greater detail in discussing agency goals and
performance indicators by geographic regions, although not comprehensively by country; (2)
USAID has in some areas tried to provide data linking performance indicators to countries
having USAID programs, rather than just providing overall regional data; and (3) USAID has
provided a more thorough discussion of data limitations and external factors affecting the
results of its programs.




Page 3                                    GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




USAID’s Performance Plan Provides a General Overview of
Intended Performance Across the Agency
USAID’s plan consists of 6 strategic goals and 1 management goal supported by a total of 25
long-term performance goals. For each of these 25 performance goals, USAID has presented
one or more quantitative performance indicators and a table showing, for four geographic
            1
groupings, the baseline measurement, the most recent available data, the fiscal year 2000
goal, and the strategic plan goal for fiscal year 2007. For example, under the strategic goal,
“The World’s Environment Protected for Long-term Sustainability,” USAID has six
performance goals, one of which is to reduce the deforestation rate in tropical forests and
improve the management of natural resources and tree systems. Progress under this goal is to
be measured using an indicator identifying the average annual change in total forest area,
both in land area and percent change – with specific goals cited for each geographic region
for fiscal years 2000 and 2007.

As we reported last year, many of these goals reflect the broadly defined goals of the
international donor community but will be difficult to relate closely to USAID programs. For
example, under USAID’s strategic goal of encouraging broad-based economic growth and
agricultural development, USAID identifies increased reliance on private markets as a long-
term performance goal. Progress under this goal is to be assessed based on performance
indicators relating to foreign direct investment, trade volume, and the Economic Freedom
Index compiled by the Heritage Foundation. USAID provides no information discussing
USAID strategies or programs under this goal, nor does it mention the possible roles that
other U.S. agencies – such as the Overseas Private Investment Corporation and the U.S.
Export-Import Bank – might play in contributing to this goal. USAID recognizes that its goals
continue to be broadly stated development goals and that its programs may not always be key
factors in accomplishing development goals; but USAID states that it is seeking to find ways
to relate its specific efforts to progress in reaching these goals.

This year’s plan presents an improved discussion of USAID’s goals and related performance
indicators by geographic regions. However, USAID’s experiences and programs differ widely,
even within the same geographic area, and overall regional performance goals and indicators
do not capture the actual goals and results intended for individual aid recipient countries or
USAID operational units. Under its discussion of each of these 25 performance goals,
USAID’s plan has provided selected descriptions by region of USAID’s programs, but these
are not comprehensive descriptions of USAID programs by country, with specific
performance goals, nor do they mention funding levels.


1
 Sub-Saharan Africa; Asia, Near East, and North Africa; Europe and the former Soviet Union; and Latin America and the
Caribbean.




Page 4                                                 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




In addition, even when USAID does discuss specific programs, it does not routinely use the
performance indicators it has identified for the long-term goal. For example, with respect to
USAID’s performance goal of improving urban population access to safe drinking water and
sanitation services, the plan mentions that USAID has significant programs in Egypt, but it
does not use the performance indicator it presents in its accompanying table – percent of
urban population with access to safe water and to sanitation services. Instead, for Egypt,
USAID notes only that 5.49 million people will be served by USAID-funded sewage and
wastewater treatment facilities and that over 1 billion liters of water per day will be treated.

This year’s plan recognizes the many ways in which USAID’s strategic goals interrelate and
reinforce each other. The plan also describes how its goals support the overall U.S.
International Affairs Strategic Plan (prepared by the State Department) covering 19 foreign
policy goals, although it does not present specific information at a country or regional level.

The plan also includes a strategic goal related to improving the management of USAID
programs. USAID acknowledges the challenges it faces in improving its management
information systems, and it notes that it will be implementing specific improvements over the
next several years.

Overall, the fiscal year 2000 performance plan indicates moderate progress in addressing the
weaknesses that we identified in our assessment of the fiscal year 1999 plan as it relates to
providing a clear picture of intended performance across USAID. In reviewing the fiscal year
1999 plan, we observed that (1) most of the results measures set out in the plan were not
directly linked to USAID’s own specific activities and (2) the plan did not articulate a strategy
for coordination with nonforeign affairs agencies. Among improvements in the fiscal year
2000 plan are (1) an enhanced discussion of USAID’s strategic goals and performance
indicators by geographic regions and (2) a clearer presentation of how USAID’s strategic
goals interrelate and support the U.S. International Affairs Strategic Plan. However, the plan
continues to need a closer linkage established between its overall development goals and the
expected results of specific USAID programs.

USAID’s Performance Plan Provides a General Discussion of the
Strategy and Resources the Agency Will Use to Achieve
Performance Goals
USAID’s fiscal year 2000 performance plan demonstrates an enhanced effort to identify how
USAID’s 25 performance goals support its 7 strategic and management goals and how each of
these 7 goals is supported by USAID’s fiscal year 2000 budget request. It includes a table
indicating, for example, that $69.8 million would be used in the newly independent states of
the former Soviet Union to support the strategic goal of “Building Democracy and




Page 5                                           GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




Governance.” The plan does not, however, include a breakdown into amounts targeted for the
25 performance goals that USAID identifies as supporting its 7 strategic goals.

The plan also presents more information on how USAID’s resources support its goals and the
goals of the U.S. International Affairs Strategic Plan. For example, the USAID plan shows that
USAID’s humanitarian programs totaling $184 million for fiscal year will combine with $450
million in Public Law 480 Title II funds in support of the U.S. International Affairs Strategic
Plan’s goal of providing humanitarian responses in short-term crises.

As previously noted, USAID’s strategic development goals reflect the goals of the
international donor community and are mutually reinforcing. However, the plan does not
specifically discuss the activities of other international donors or of other U.S. government
agencies, such as the U.S. Overseas Private Investment Corporation, the U.S. Export-Import
Bank, or the U.S. Department of Agriculture.

The plan discusses how USAID has revised certain performance goals and indicators in order
to serve its strategic goals, but it does not generally report on the changes USAID has made
over the past year to its program approaches in specific regions or countries.

Some discussion of the results of USAID’s recent program evaluations would contribute to a
more specific description of changes in program approaches, as well as provide some
information on how specific strategies or programs can lead to intended results.

A significant improvement in this year’s plan is its recognition of the many external factors
that can affect the outcomes of USAID programs. In particular, USAID cites the impact of the
Asian financial crisis and its international consequences as an important external variable. It
notes, however, that many of the poorest countries are not closely connected with the global
economy and may not be significantly affected by the financial crisis. As noted previously,
USAID explicitly recognizes that, for some goals, it plays a relatively small role and that
progress achieved cannot be directly linked to USAID efforts. The agency states that it will
continue to try to develop measures that can plausibly associate progress with the impact of
USAID programs, but it also notes that it will assess the costs, both in resources and in
agency focus, of efforts to improve data and measurement indicators.

With respect to agency resources needed to achieve the plan’s performance goals, the plan
notes the limitations of USAID’s budget and hiring capacity and states that the agency will
seek to improve its capabilities through a strategy of training and selective hiring, as attrition
allows.

Also, with respect to USAID’s goal of improving the management and delivery of its
development assistance resources, this year’s plan acknowledges the continuing problems of




Page 6                                           GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




the agency’s management information systems. It notes that a new, commercial, off-the-shelf
core accounting system will be implemented by fiscal year 2000.

Overall, the fiscal year 2000 performance plan indicates some progress in addressing the
weaknesses that we identified in our assessment of the fiscal year 1999 performance plan as
it relates to providing a complete discussion of strategies and resources USAID will use to
achieve its performance goals. In reviewing the fiscal year 1999 plan, we noted that (1) USAID
did not discuss all of its goals with a consistent level of detail that would convey the nature
and magnitude of its efforts and (2) USAID only partially discussed key external factors or
other donors’ related efforts to achieve the same goal. Among improvements in the fiscal year
2000 plan are (1) an enhanced effort to describe its 25 performance goals and identify how
they support its 7 strategic goals and (2) a clearer acknowledgement of the numerous
external factors that can affect the outcomes of USAID goals.

USAID’s Performance Plan Provides Limited Confidence That
the Agency’s Performance Information Will Be Credible
USAID’s performance plan provides limited confidence that the agency has the capacity to
obtain credible, results-oriented, program performance information. USAID acknowledges
that continued data improvements are needed in order to achieve a uniform, high-quality
database permitting cost-benefit analyses of USAID’s overall development assistance
program. The performance plan lists several limitations that USAID faces in seeking to
improve data quality. First, since the agency is dependent on international organizations and
thousands of partner institutions for data, USAID does not have full control over how data
are collected, reported, or verified. Second, programs aimed at creating new capacity are
difficult to measure, particularly at their initial stages. Finally, many information sources have
different reporting periods, and the data are often not available for a year or more afterwards.
In addition, sometimes these data are significantly adjusted after publication.

The agency indicates that it is seeking to find ways to improve data quality for some of its
performance indicators. For example, for its goal of reducing by 10 percent the number of
deaths due to infectious diseases of major public health importance by 2007, USAID reports
that no data are available on a country-specific basis and that it will be working with the
World Health Organization to improve the data by 2002. In other instances, USAID indicates
that it will seek to assure collection of relevant data by conducting periodic surveys in
USAID-assisted countries.

We recently reported that USAID’s internal accounting and information systems do not have
the capacity to generate reliable data to support its performance plan and to produce credible




Page 7                                           GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




                           2
performance reports. USAID’s financial management system does not meet the federal
financial management systems requirements, and material weaknesses in internal controls
impair the integrity of its financial information. The agency has indicated that it is committed
to developing a financial management system that will meet federal standards, but the USAID
Inspector General has reported that the agency has made only limited progress in correcting
its system deficiencies.

USAID has acknowledged its problems with data quality and has made some improvements.
If information is collected by other organizations, the agency requires staff to periodically
confirm its usefulness for USAID’s purposes. USAID has included a discussion in the
performance plan of its system of intra-agency checks and balances in reviewing and
interpreting data. The agency has also begun operational changes that it believes will
eventually lead to implementation of managerial cost accounting, allowing the full cost of
outputs at the intermediate results level to be calculated.

The fiscal year 2000 plan recognizes the weaknesses that we identified in our assessment of
the fiscal year 1999 performance plan as it relates to providing full confidence that the
agency’s performance information will be credible and makes specific commitments or
shows actual attempts to address those weaknesses. However, real progress is not yet
evident, and the fiscal year 2000 plan in this regard represents little, if any, improvement over
the 1999 plan. In reviewing the fiscal year 1999 plan, we observed that the plan (1) addressed
the issue of verification and validation of performance data only very generally and did not
discuss the issue of data reliability, (2) did not specifically state how the agency planned to
achieve its primary financial improvement goal of “making financial and program results
information more accurate and readily available for decisionmakers,” and (3) acknowledged
significant data limitations but did not adequately address the problem of verifying the
credibility of the external performance information it needs. As previously discussed, these
issues are still evident. In addition, USAID’s Inspector General found many instances of
unsupported and inaccurate data in a recent review of the agency’s internal performance
reports. Among improvements in the fiscal year 2000 plan are a “methodology notes” section
used to explain how performance indicators were calculated. Although this detail does not
ensure reliable data, it does allow the reader to determine data limitations.




2
 Performance and Accountability Series: Major Management Challenges and Program Risks: Agency for International
Development (GAO/OCG-99-16, January 1999).




Page 8                                               GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure I
Observations on the U.S. Agency for International Development's Performance Plan for Fiscal Year 2000




Other Observations on USAID’s Implementation of
Performance-Based Management
USAID notes that it will be revising its strategic plan during 1999 and, accordingly, some of its
performance goals will be changed. We note that if these revisions in strategic and
performance goals are significant, some clarification may be needed on how USAID will be
expected to report on revised performance goals.

Agency Comments
On May 4, 1999, we obtained comments from USAID officials, including the Deputy Assistant
Administrator, Bureau for Policy and Program Coordination, on a draft of our analysis of
USAID’s fiscal year 2000 annual performance plan. These officials generally agreed with our
analysis. In addition, with respect to our comments regarding the need to link agency goals
with individual country goals, they noted that USAID is currently developing methods of
improving the linkage among the Annual Performance Plan, the Annual Performance Report,
and the country coverage provided in USAID’s Congressional Presentation. They also noted
that they are exploring ways to improve the quality of data used to assess performance.




Page 9                                           GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure II

Management Challenges


The key management challenge confronting USAID is the need to implement reliable
management information systems. USAID needs to institute both a comprehensive
information management system and a financial management information system.

Table II.1: Management Challenges in USAID’s Fiscal Year 2000 Performance Plan
Management challenge                                     Applicable references in the fiscal year 2000
                                                         annual performance plan
USAID has not implemented a comprehensive                The plan identifies the goal of improving USAID’s
information management system.                           management of in formation resources through the
                                                         acquisition of commercial, off-the-shelf solutions for
                                                         addressing property and human resource
                                                         management requirements. The plan notes that
                                                         these are to be deployed by fiscal year 2000, but it
                                                         provides no details on the resources being targeted
                                                         at this goal.
USAID’s financial management information is              The plan states that the agency has decided that the
unreliable.                                              least costly, lowest risk approach to fulfill the
                                                         financial management system requirements is to
                                                         purchase a commercial, off-the-shelf core
                                                         accounting system. By fiscal year 2000, USAID
                                                         expects to implement the new system in
                                                         Washington, D.C., followed by implementation in the
                                                         field in 2001.
USAID could face disruptions due to the Year 2000        The goal is to have completed the work to assure
technology update problem                                that all mission-critical systems and their supporting
                                                         infrastructure are Y2K compliant by the beginning of
                                                         FY 2000. The plan states that, if necessary, the
                                                         agency will undertake emergency remediation and
                                                         be prepared to implement continuity of operations
                                                         plans for any essential functions that might be
                                                         interrupted. In addition, the agency will complete
                                                         remediation of approximately 14 corporate
                                                         applications systems previously identified as not
                                                         mission critical. The plan also states that the Y2K
                                                         program will receive the highest priority for allocation
                                                         of information management resources. However, it
                                                         is important to note that, according to the agency’s
                                                         own reporting, it is far behind the government’s
                                                         required schedule. No measures or indicators are
                                                         discussed in the plan.
Information security                                     None. The plan does not address this. However, it
                                                         states that the confidentiality, integrity, and
                                                         availability of data will be improved with enhanced
                                                         information security management and reporting.
Inspector General’s areas of concern                     Applicable references in USAID’s fiscal year
                                                         2000 annual performance plan
The quality of USAID’s performance data is               None. The plan identifies selected efforts to improve
questionable.                                            data quality.




Page 10                                        GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Enclosure III

GAO Contacts and Staff
Acknowledgments

GAO Contact
Benjamin F. Nelson,    (202) 512-4128

Acknowledgments
In addition to the contact named above, Jess T. Ford, Virginia C. Hughes, Elizabeth Johnson,
Sue Piyapongroj, and Linda P. Garrison made key contributions to this product.




(711418)




Page 11                                 GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Page 12   GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Page 13   GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan
Page 14   GAO/NSIAD-99-188R USAID’s Fiscal Year 2000 Performance Plan