oversight

Foreign Assistance: USAID's Reengineering at Overseas Missions

Published by the Government Accountability Office on 1997-09-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                   United States General Accounting Office

GAO                Report to the Chairman, Committee on
                   International Relations, House of
                   Representatives


September 1997
                   FOREIGN
                   ASSISTANCE
                   USAID’s Reengineering
                   at Overseas Missions




GAO/NSIAD-97-194
             United States
GAO          General Accounting Office
             Washington, D.C. 20548

             National Security and
             International Affairs Division

             B-277413

             September 12, 1997

             The Honorable Benjamin A. Gilman
             Chairman, Committee on International Relations
             House of Representatives

             Dear Mr. Chairman:

             In response to your request, we have reviewed the effect of the U.S.
             Agency for International Development’s (USAID) reforms on its overseas
             missions’ operations and delivery of assistance. Specifically, we focused
             on (1) how USAID missions have reengineered their operations, (2) how
             reengineering has affected the content of USAID’s assistance program,
             (3) how USAID monitors and evaluates the results of its projects,1 (4) how
             USAID allocates funds for its projects, and (5) how USAID’s New
             Management System supports mission operations. In doing this review, we
             visited missions in El Salvador, Honduras, Bangladesh, and the
             Philippines.

             This is the second of a two-part review of USAID’s reform efforts. In
             September 1996, we reported that USAID had adopted a development
             strategy with priority areas of concentration, reorganized its headquarters
             units, closed some overseas missions, cut its total workforce, developed
             an information resource management system for its business and service
             functions, and had begun to reengineer its procurement and overseas
             mission operations.2


             USAID  has provided foreign aid since 1961 in an effort to improve countries’
Background   economies, health, environment, and democratic processes. In times of
             crisis, the agency has also provided humanitarian assistance to those in
             need. USAID currently has 72 overseas missions and offices that manage
             projects associated with this foreign assistance. These projects are
             generally implemented by private voluntary organizations,
             nongovernmental organizations, international agencies, universities, and
             contractors.




             1
              For consistency and simplicity, we use the term “project” when referring to individual USAID
             development efforts. As part of its reengineering efforts, USAID is now referring to “projects” as
             “activities” that support its strategic objectives. USAID reports to Congress by project and activity.
             2
              See Foreign Assistance: Status of USAID’s Reforms (GAO/NSIAD-96-241BR, Sept. 24, 1996).



             Page 1                                                        GAO/NSIAD-97-194 Foreign Assistance
B-277413




Since the end of the Cold War, Congress has reduced USAID’s
appropriations, citing other funding priorities as well as basic questions
about the effectiveness of foreign aid. USAID’s fiscal year 1996 obligations
of $5.7 billion were 13 percent less than those in the previous year (see
app. I for information on USAID obligations for fiscal years 1995 through
1997). To accommodate budget reductions, USAID cut its total workforce
from 11,150 in September 1993 to 7,609 in June 1997 and will have closed
24 missions by the end of fiscal year 1997. Under the executive branch’s
decision to consolidate the Department of State, the Arms Control and
Disarmament Agency, and the U.S. Information Agency, the USAID
Administrator will report directly to the Secretary of State rather than, as
previously, to the President.

Congress specifically directs how USAID spends much of its funding,
earmarking substantial amounts of USAID’s direct appropriation for
development assistance for specific purposes, such as child survival.
Congress also earmarks funds for economic support and food assistance,
which are administered by USAID. Congressional earmarks and executive
branch directives (e.g., for the population program) accounted for
59.8 percent of development assistance funds in fiscal year 1995,
66.5 percent in fiscal year 1996, and 69.5 percent for fiscal year 1997 (see
app. II for more details on directed and undirected assistance).

In 1993, in response to criticisms about USAID’s inefficient processes and
inability to demonstrate a significant impact on developing countries and
some calls for the agency’s abolition, USAID’s Administrator volunteered
the entire agency as a “reinvention laboratory.” USAID established five
strategic goals to meet its agency mission of pursuing sustainable
development in developing countries: (1) achieving broad-based economic
growth, (2) building democracy, (3) stabilizing world population and
protecting human health, (4) protecting the environment, and
(5) providing humanitarian assistance.

According to USAID, reengineering the agency’s overseas operations has
revolved around (1) increasing its customer focus, (2) managing for
results, (3) enhancing staff participation and teamwork, (4) empowering
and increasing the accountability of staff, and (5) valuing diversity. The
reengineering included reorganizing mission organizational structures,
eliminating unnecessary administrative requirements, and including the
participation of development partners and recipients in program planning.




Page 2                                       GAO/NSIAD-97-194 Foreign Assistance
                   B-277413




                   In 1994, USAID selected 10 missions to test these reengineering principles,
                   concepts, and approaches prior to deployment worldwide. Based on
                   observations of the experiments and feedback from the participating
                   missions, USAID initiated full implementation of its reengineering approach
                   on October 1, 1995. In fiscal year 1996, USAID deployed the New
                   Management System, which was intended to provide an integrated
                   database of financial and program information to facilitate the
                   management of resources and monitoring of results. (See app. III for a
                   chronology of USAID reforms.)

                   USAID’sAdministrator has noted that the agency’s reforms are consistent
                   with the Government Performance and Results Act of 1993 (GPRA) and
                   have positioned USAID well to meet the act’s requirements for strategic
                   planning and performance monitoring.3


                   In reengineering the way assistance is delivered, USAID missions have only
Results in Brief   recently made the investment and operational changes necessary to bring
                   about long-term change. As our previous work has shown, reengineering
                   efforts of the magnitude of USAID’s can require years of steady work to
                   yield intended benefits. Thus, while the agency has built momentum for
                   change, the efforts have not yet led to a fundamental change in the
                   substance of its program, and USAID cannot yet demonstrate that it has
                   made significant progress in achieving its strategic goals. Additional
                   important factors such as foreign policy objectives and congressional
                   earmarks will continue to significantly influence the allocation of USAID’s
                   resources and affect its progress in changing its program. To date, USAID’s
                   reengineering efforts to improve its delivery of assistance have shown
                   some benefits in the areas of planning, implementing, and monitoring
                   projects. Notwithstanding this progress, USAID still has major obstacles to
                   overcome in deploying its New Management System and establishing valid
                   and reliable performance measures.

                   Overseas missions have changed the way they manage assistance in
                   support of USAID’s strategic goals. USAID has increased the authority and
                   flexibility of missions to plan, design, and implement their activities, and
                   mission directors have delegated much of their authority to staff teams.
                   These teams have fostered collaboration among mission staff,
                   development partners, and recipients of assistance in designing and
                   implementing development activities. However, because missions had no

                   3
                    We recently issued a report analyzing USAID’s draft strategic plan and its compliance with GPRA. See
                   The Results Act: Observations on USAID’s November 1996 Draft Strategic Plan (GAO/NSIAD-97-197R,
                   July 11, 1997).



                   Page 3                                                     GAO/NSIAD-97-194 Foreign Assistance
B-277413




baseline data on operations management prior to reengineering their
processes, neither they nor we can identify measurable increases in
efficiency or effectiveness of the delivery of assistance. Nonetheless,
mission officials were optimistic about the potential impact of their
changes in operations.

USAID missions have, to varying degrees, begun to establish more
results-oriented indicators and report the results of their projects annually.
However, the missions still have difficulty linking their activities to the
broad indicators of development—such as a country’s rate of economic
growth. Since many other factors outside USAID’s control have a significant
impact on these indicators, USAID cannot easily measure how effectively its
programs are achieving the agency’s strategic goals. One way to provide a
more complete picture of program performance could be to supplement
performance measurement data with impact evaluation studies.

Although the relative performance of mission programs is clearly a factor
in USAID’s resource allocation decisions, these decisions are still largely
driven by other considerations, such as contributions to foreign policy and
agency priorities, country need and commitment, and funding priorities of
the executive branch and Congress. The decisions to provide funds for
Haitian democratization efforts and to support the Guatemalan peace
process, for example, were made independently of performance
considerations.

USAID’s New Management System, one of the agency’s key tools in
reforming its operations, is not working as intended. This computer
system, which is expected to cost at least $100 million by the end of fiscal
year 1998, was designed to enable the agency to manage its resources and
monitor results more effectively by consolidating accounting, budgeting,
personnel, procurement, and program operations into a single, integrated
network that can be accessed worldwide. Despite warnings that the
system had not been tested and did not meet basic federal requirements,
USAID activated the system in July 1996 in Washington and deployed it in
October 1996 in its missions. USAID suspended much of the system’s
operation in April 1997, after it failed to work properly. Correcting system
deficiencies will be critical to continued progress of the agency’s reform
effort.




Page 4                                       GAO/NSIAD-97-194 Foreign Assistance
                            B-277413




                            Overseas missions have changed their processes to streamline the way
Missions Are                they provide assistance in support of USAID’s strategic goals. These changes
Reengineering Their         include taking a team approach to management, accepting increased
Operations in Line          accountability through the authority and flexibility given them by USAID,
                            and increasing the participation of development partners in the design and
With USAID Reforms          implementation of projects. USAID has supported the missions’ reform
                            efforts with information and advice. Mission officials believe that these
                            changes will result in more effective assistance programs. Moreover, our
                            review indicated that the changes show promise that USAID is making
                            progress in better managing the process of providing assistance.

                            We could not fully assess the impact of the missions’ operational changes
                            for the following reasons: (1) missions are at varying stages of
                            implementing changes; (2) the impact of these changes may not be seen
                            for several years, especially those that affect the design of new projects;
                            and (3) the missions we visited had not established baseline information
                            on the efficiency or effectiveness of their management operations before
                            reengineering their processes and therefore could not demonstrate many
                            measurable improvements. In some missions, officials indicated that some
                            baseline data could be compiled from historical records but that doing so
                            would demand an inordinate amount of scarce mission resources.


Missions Have Established   Overseas missions we visited have reengineered their organizational
Team Structures             structure to focus on achieving strategic objectives. Most missions have
                            replaced their former office structure with a team approach to
                            management, and each team is responsible for one of the mission’s
                            strategic objectives. Team members at the missions we visited commented
                            that the team approach has improved the management of their activities.
                            USAID said that overseas missions have observed that teams are now more
                            likely to (1) remain focused, (2) rethink tactics quickly when activities do
                            not go as planned, and (3) terminate activities that only marginally
                            contribute to results.

                            Prior to reengineering, missions had several offices: technical offices, such
                            as the education, health, and agriculture offices, which had day-to-day
                            responsibility for implementing assistance projects, and support offices,
                            such as the contracts, controller, and legal adviser’s offices, which assisted
                            the technical offices. According to mission staff, in the past, each of these
                            offices had its own management hierarchy and separately reviewed
                            proposed actions such as expenditure requests or new project starts. This
                            review was often done sequentially, and as each office raised problems or



                            Page 5                                       GAO/NSIAD-97-194 Foreign Assistance
                            B-277413




                            issues, action was delayed until they could be resolved; the proposal was
                            then passed to the next office for review.

                            The four missions that we visited had reorganized their offices into teams
                            to manage assistance activities that support each mission’s strategic
                            objectives. Staff members that previously worked for different technical
                            and support offices now work together on a team toward a specific
                            common objective. In most missions, the teams replaced much of the
                            traditional office structure. For example, the mission in El Salvador
                            combined the staff of its economic, education, and productive resources
                            offices into a single economic growth strategic objective team, which also
                            includes members from the contracts and controller offices.

                            At the missions we visited, both technical and support staff said that
                            issues that had previously delayed progress under the old process can now
                            be resolved within the teams. For example, contract office staff are now
                            involved in the early stages of project design and implementation,
                            suggesting appropriate procurement instruments and helping draft scopes
                            of work for proposed contracts. According to contract officers, this
                            approach has reduced the number of sequential reviews and rewrites of
                            proposals and the tensions that have historically existed between staff and
                            technical offices.


Mission Management and      Under reengineering, USAID increased the authority and flexibility of
Staff Have More Authority   missions in managing their activities by eliminating many administrative
and Flexibility             requirements previously imposed on the missions. For example, USAID
                            eliminated the requirement for missions to prepare elaborate project
                            papers and obtain bureau approval for new projects. Once a regional
                            bureau approves a mission’s strategic objectives, the mission is authorized
                            to develop activities to meet those objectives without the bureau’s further
                            review and approval. USAID also eliminated the 10-year time limit on all
                            projects, allowing the missions the flexibility to extend projects when it is
                            advantageous to do so. For example, in Honduras, the mission extended
                            an ongoing policy project in order to continue to provide high-level policy
                            advice to the newly elected Honduran administration. Before
                            reengineering, the mission would have had to design and authorize a
                            completely new project or apply for a waiver from USAID to extend the
                            project.

                            Furthermore, USAID delegated to the missions a host of executive
                            authorities previously held by officials at USAID headquarters. For example,



                            Page 6                                       GAO/NSIAD-97-194 Foreign Assistance
                             B-277413




                             missions may now independently issue implementation letters and
                             negotiate and implement agreements with other U.S. government agencies.

                             In accordance with the agency’s emphasis on staff empowerment and the
                             assignment of responsibility and accountability under reengineering,
                             mission directors delegated new authorities to their missions’ strategic
                             objective teams. For example, the mission director in Honduras authorized
                             team leaders to approve individual expenditures of up to $100,000 in
                             program funds without her review.


Missions Are Implementing    In the past, USAID was more concerned with the required inputs and
a System to Report Results   expected outputs of a particular project than the results to be achieved.
of Activities                Projects had individual purposes that often related to an overall goal or
                             goals in only a very general way, and the goals themselves were in broad
                             categories rather than specific objectives against which progress could be
                             measured. Also, missions reported on their activities in a variety of
                             reports, including semiannual reviews and project implementation
                             reviews, that did not necessarily focus on development results but rather
                             focused on the status of project implementation.

                             USAID is increasingly holding missions accountable for results. In fiscal
                             year 1996, it began requiring the missions to report annually on the results
                             of their programs in a report called the Results Review and Resource
                             Request (R4). Missions must now report to high-level management on their
                             progress in meeting agreed-upon strategic objectives and agency goals and
                             quantify that progress through performance indicators developed by the
                             missions, often with assistance from USAID’s Center for Development
                             Information and Evaluation. Missions are to supplement the data in the
                             annual R4 document with analyses of performance data and other
                             evidence that the mission is making progress in meeting its strategic
                             objectives.


Participation of             USAID has given its development partners4 a more active role in developing
Development Partners Has     country strategies and designing and managing projects. Many of these
Increased                    partners were enthusiastic about this change. In some cases, the increased
                             interaction has accelerated the design and management process. Also,


                             4
                              USAID has defined its partners as host-country governments, U.S. private voluntary agencies,
                             indigenous or international nongovernmental agencies, universities, professional or business
                             associations, other U.S. government agencies, other bilateral or multilateral donors, and international
                             financial institutions.



                             Page 7                                                       GAO/NSIAD-97-194 Foreign Assistance
B-277413




with input from customers,5 the missions believe they have been better
able to respond to a country’s needs and develop a strategy for meeting
those needs.

Missions previously presented completed project designs to host countries
for acceptance. Today, however, the missions invite the host country,
other development partners, and customers to participate in planning and
designing activities. For example, according to several representatives
from nongovernmental organizations in El Salvador, the mission has
invited them to its planning and designing meetings, and their input was
included in the mission’s development strategy. In Bangladesh, the mission
brought together the customers and development partners to redesign a
health and population program that for the first time combines services in
both family planning and maternal child health care.

Missions also include development partners as members of strategic
objective teams. For example, Bangladesh government officials from the
Rural Electrification Board told us that they actively participated with
USAID team members, contractor staff, and representatives from local rural
electric cooperatives in designing a rural power for the poor project and
selecting the indicators by which results will be measured.

Many nongovernment organizations and donors welcomed the missions’
new collaborative approach to designing and implementing activities and
were very positive about the missions’ interaction with them. The benefit
of this increased interaction can be seen in Bangladesh, for example,
where the democracy strategic objective team, working with
nongovernmental organizations and staff offices, designed a new project
in 6 months compared to the 2 years that project designs had generally
required before.

Some donors have also observed increased cooperation with USAID
missions. In Bangladesh, an official of the British Office of Development
Assistance said that the mission was instrumental in influencing the host
government’s development of a national health and population strategy.
Increased cooperation with other donors was not always evident at the
missions we visited, however. For example, in El Salvador, mission
officials told us they have tried to coordinate with the Inter-American
Development Bank but have received little response. Officials also said
that some other donors had been reluctant to share information.



5
 USAID has defined its customers as the end users or beneficiaries of assistance.



Page 8                                                      GAO/NSIAD-97-194 Foreign Assistance
                          B-277413




                          Teams are increasingly relying on surveys of customers to determine their
                          needs and interests when designing activities and developing strategies.
                          For example, in Bangladesh, the mission has focused its democracy and
                          governance activities on local government rather than the national
                          government because customers indicated that local government was a
                          more effective agent of change. In El Salvador, strategic objective teams,
                          with the assistance of local contractors, held focus groups throughout the
                          country and used the results to develop customer service plans and as
                          input for the mission’s country strategy.


USAID Headquarters Has    USAIDhas established new systems and practices to support missions’
Supported Reengineering   implementation of its reengineering efforts by providing advice and
at Missions               information on reengineering. However, it has not provided training
                          needed to learn new skills in team dynamics and personnel matters.

                          USAID provides information, advice, and answers to staff questions on
                          reengineering through a monthly newsletter, best practices reports and
                          other publications, and an electronic help desk. The missions can more
                          easily access up-to-date agency guidance through an on-line computer
                          system that has replaced 33 manuals on policies, procedures, and program
                          operations guidance. Also, USAID’s Management Bureau sent a team to visit
                          several missions in 1997 to identify “significant management,
                          organizational, and personnel-related constraints being encountered by
                          USAID missions as they embrace reengineering principles” and to provide
                          assistance to resolve them.

                          At the time of our visits to the four missions, USAID was not providing
                          needed training in new job skills and team operations. For example, USAID
                          has promoted the use of performance-based contracting without providing
                          the requisite training to the missions. As a result, the mission contracts
                          officer in Honduras was developing his own training course. Also, USAID
                          support in personnel matters has lagged behind mission needs, according
                          to mission managers. Thus, the missions have, on their own, developed
                          foreign employees’ evaluations and position descriptions and
                          classifications but are still waiting for USAID’s reengineered position
                          descriptions for American staff.

                          USAID  acknowledged that it had not provided adequate guidance to the
                          field in training and personnel matters. However, USAID noted that there is
                          broad agreement that training requirements and Foreign Service position
                          classification should be the next issues that the agency examines.



                          Page 9                                      GAO/NSIAD-97-194 Foreign Assistance
                              B-277413




Portfolios Are Beginning to   Missions have begun to change their project portfolios in response to
Change but Are Subject to     agency reengineering. However, three factors have constrained the
Constraints                   restructuring of portfolios. First, reduced mission budgets have limited the
                              availability of funds for new projects. Second, the missions do not have
                              the authority, without headquarters approval, to retain deobligated funds
                              and shift them from one project to another under a restructured portfolio,
                              according to USAID officials. Third, during the first year of reengineering,
                              the missions concentrated more on reorganization activities and the
                              development of strategic frameworks, including defining performance
                              indicators and obtaining baseline data, which limited the time available for
                              portfolio restructuring and new project development.

                              The missions we visited were still restructuring their portfolios to focus on
                              projects that more directly support their strategic objectives. They were
                              doing this largely by designing new projects specifically to help achieve
                              each strategic objective and by shifting funds to activities that are more
                              effective at achieving the objective. To date, only a few new projects have
                              been started. In the Philippines, the mission is reshaping its portfolio by
                              selecting one project for each strategic objective to serve as a funding
                              mechanism for all activities under the objective. In Bangladesh, the
                              mission initiated one new project for each of the mission’s strategic
                              objectives, which were designed using reengineering principles. In 1996,
                              USAID approved a new country strategy that refocused activities in El
                              Salvador on the rural poor rather than war-to-peace transition activities
                              and macroeconomic reform at the national level, which were the past
                              focus. The mission in Honduras largely restructured its portfolio in 1994
                              and 1995, prior to the start of reengineering, and was awaiting approval of
                              its new country strategy before it initiated any new projects.

                              Reengineering efforts were undertaken during a period of severe budget
                              reductions. Between 1994 and 1997, expenditures at the four missions we
                              visited declined between 43 and 69 percent. A change in the number of
                              projects can be used as a rough approximation of the change in mission
                              activities and the restructuring of the portfolio. In general, the number of
                              active projects declined at these missions, mostly between 1994 and 1995,
                              before the formal start of reengineering agencywide. In three missions, a
                              limited number of projects had been initiated since the start of
                              reengineering.

                              Because most current mission projects were designed and funded before
                              the reengineering began and because USAID’s budget has been reduced, a
                              limited amount of funding has been available for new projects. According



                              Page 10                                      GAO/NSIAD-97-194 Foreign Assistance
                      B-277413




                      to mission officials, the inability to automatically retain funds that are
                      deobligated from a project limits a mission’s flexibility to use funds most
                      effectively. It may also serve as a disincentive for terminating projects.
                      Consequently, missions may allow projects to continue, even though they
                      could identify other projects that may contribute more to their strategic
                      objectives.

                      During the first year of reengineering, the missions were busy reorganizing
                      into strategic objective teams, learning how to function as teams, selecting
                      measurable indicators, educating their partners about their new approach,
                      and training staff in the use of the New Management System. When we
                      visited the four missions in February 1997, the mission in El Salvador had
                      just completed the development of a country strategy and was still refining
                      its indicators for the economic growth strategic objective. The mission in
                      Honduras was developing a new country strategy and had not
                      implemented any new projects since the start of reengineering.


                      Under reengineering, missions have made varying degrees of progress in
Performance           developing performance indicators through which they can measure the
Indicators Are        short-term results of individual projects or groups of projects, but these
Improving, but        indicators have not been in use long enough to have made a significant
                      impact on program management. Missions have had difficulty in
USAID’s Impact on     developing indicators that tally up the results of individual or groups of
Overall Development   projects to demonstrate achievement of strategic objectives and overall,
                      long-term development goals, such as stimulating broad-based economic
Is Still Unclear      growth. USAID officials acknowledge that in only a few cases have their
                      programs been directly linked to changes in country-level results. USAID
                      management indicated that the ability of a program to affect country-level
                      indicators will depend on the size of the country, the budget available to
                      support specific strategic objectives, and other factors such as the context
                      in which programs are implemented. USAID is now developing common
                      indicators through which it hopes to combine mission results into overall
                      agency accomplishments worldwide. We recently reported that program
                      evaluations can be valuable management tools for demonstrating the
                      programs’ effectiveness.6 Such evaluations are an option for the agency to
                      consider to help in its efforts to meet GPRA requirements.




                      6
                      See Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138,
                      May 30, 1997).



                      Page 11                                                GAO/NSIAD-97-194 Foreign Assistance
                             B-277413




Missions at Various Stages   USAID missions have been developing performance indicators on two
in Adopting Measures of      levels. First, missions are developing project-level indicators intended to
Intermediate Project         monitor the performance and intermediate results of individual projects or
                             groups of projects in their portfolio. Second, missions are developing
Results                      strategic objective indicators that are intended to gauge progress in
                             achieving the missions’ long-term or strategic objectives, such as
                             increasing national rural household incomes or improving national
                             systems of trade and investment.

                             The four missions we visited were at various stages in developing and
                             using performance measures for the projects intended to achieve one of
                             their strategic objectives—economic growth. In most cases, the
                             project-level indicators were too new for the missions to show impact on
                             program management. Although one mission had demonstrated a
                             thorough integration of its measures into its program management
                             processes, another was still defining the indicators it would use. Some of
                             the missions had fundamentally revised their project-level performance
                             measures in 1996 to make them more closely correspond to their strategic
                             objectives at the country level.

                             In Honduras, the mission has been using indicators to measure project
                             results since before reengineering. For example, to measure the results of
                             its agricultural assistance activities, the mission has monitored the volume
                             and value of exports for six crops: sweet onions, ginger, okra, snow peas,
                             asparagus, and plantains. The mission has several activities aimed at
                             promoting these exports, including providing technical assistance to
                             farmers for production and marketing. Indicators the mission uses to
                             measure intermediate results for other activities include the number of
                             land titles issued by the government and the number of vocational center
                             graduates employed. Before adopting these types of results indicators, the
                             mission focused primarily on measuring the physical outputs supported by
                             USAID assistance, such as the number of agricultural research projects
                             conducted or the number of schools built. The mission has been using
                             results indicators extensively for program management, including
                             incorporating them in annual work plans for contractors and other
                             development partners and in its annual performance review.

                             In El Salvador, the mission recently established performance indicators
                             that are intended to measure the short-term results of its assistance
                             projects. In 1996, the missions relied almost exclusively on
                             macroeconomic statistics to report on the results of its economic
                             growth-related projects. The mission revised its indicators in 1996 to



                             Page 12                                     GAO/NSIAD-97-194 Foreign Assistance
                        B-277413




                        better reflect the direct outcomes of its program activities. For example,
                        new indicators include the number of people in rural areas that are active
                        clients (that have an outstanding loan and/or a savings account) in
                        participating financial institutions and the number of people in rural areas
                        receiving assistance in management, agricultural technology, and
                        marketing. However, because these indicators are new, they have not been
                        used extensively by the mission for program management.

                        In the Philippines, the mission had developed an extensive set of
                        performance indicators, which it reported in 1996. However, mission
                        officials acknowledged that these indicators measured development
                        results that were beyond the scope of their activities in that country. Since
                        then, the mission has improved many indicators. For example, indicators
                        for projects in Mindanao have been revised to more accurately reflect the
                        desired results of the individual activities. The indicators include the total
                        value of USAID-facilitated or -assisted private investments and the number
                        of government policies or practices modified to facilitate rapid and
                        equitable economic growth. Although many of the mission’s indicators are
                        new, the mission has collected historical data against which to measure
                        future progress.

                        In Bangladesh, the mission was restructuring its strategic objectives and
                        revising its indicators. Many of the mission’s indicators for its economic
                        growth objective had focused on results at the national level and not
                        specifically on areas targeted in USAID’s activities. The mission is now more
                        closely aligning its indicators with its activities to reflect what can feasibly
                        be achieved, given its economic growth resources. Some of the mission’s
                        more meaningful indicators include the amount of fertilizer and improved
                        seed marketed nationally and the percentage of the population with access
                        to disaster relief supplies within 72 hours. Some historical data were
                        available for comparison with the current indicators, but the mission plans
                        to use a new data collection methodology for one key indicator being
                        developed, and baseline data are not yet available.


USAID Often Unable to   Projects are expected to contribute to the achievement of missions’
Demonstrate Impact of   strategic objectives and USAID’s overall, long-term goals. USAID is working to
Projects on Mission     develop better ways to measure the extent of its contribution to a
                        country’s development. According to USAID officials, internal and external
Objectives and Agency   factors, including political instability and the level of other donors’
Goals                   assistance, affect the missions’ country-level indicators. According to USAID
                        management, it is not possible to measure, in a quantifiable and precise



                        Page 13                                       GAO/NSIAD-97-194 Foreign Assistance
    B-277413




    way, the impact of USAID activities alone on country-level development
    indicators. Therefore, missions’ strategic objective indicators are intended
    to reflect the results not only of mission projects but also of the efforts of
    other development partners. The collective impact of activities at the four
    missions we visited was not significant enough to have a measurable
    impact on country-level indicators. USAID does not claim responsibility for
    the development results measured by its strategic objective indicators but
    rather claims a “plausible association” with the results.

    Missions usually measure progress toward their strategic objectives using
    country-level development indicators. In the countries we visited,
    indicators of progress toward economic growth objectives included the

•   number of people employed nationally in the agricultural, industrial, and
    service sectors in Honduras;
•   percentage of the total population in El Salvador with access to potable
    water;
•   national ratios of total exports and imports to gross domestic product in
    the Philippines; and
•   per capita gross domestic product in Bangladesh.

    At each of the four missions we visited, mission documents showed or
    USAID officials acknowledged that the activities did not significantly affect
    country-level indicators of economic growth. For example, for 1997, the
    Bangladesh mission reported that the per capita gross domestic product
    growth and agricultural and industrial employment had improved but that
    “given USAID’s relatively small investments in this strategic objective, we
    cannot claim major achievements at this national level.” In El Salvador, the
    mission reported that its projects were not sufficient to achieve its
    economic growth strategic objective. The mission’s economic growth
    portfolio is composed of a limited number of projects that focus primarily
    on microfinance, small business, basic education, small-scale agriculture,
    and policy reform. The mission in El Salvador reported that achieving the
    mission’s strategic objective depends on major contributions from other
    partners, especially international banks, which are expected to provide
    most of the funding required for activities relating to land, policy, and
    infrastructure.

    Internal and external factors can have a profound impact on
    macroeconomic conditions and thus on country-level indicators. Such
    factors include political instability, the commitment of political leaders to
    necessary reforms, the magnitude and effectiveness of assistance from



    Page 14                                      GAO/NSIAD-97-194 Foreign Assistance
B-277413




other bilateral and multilateral donors, weather conditions that affect crop
yields, and the stability of international markets. Recent mission reports
for both Bangladesh and the Philippines cited external factors as
explanations for the failure to achieve country-level development results.
The Bangladesh mission reported that political turmoil and power
shortages were the principal causes of shortfalls in expected economic
growth. The Philippines mission reported that the value of direct exports
from Mindanao was below targeted levels, primarily due to the decline in
pineapple prices and the loss of European markets for bananas.

Other observers have recently noted USAID’s difficulty in demonstrating its
impact on broad development indicators. In March 1997, USAID’s Inspector
General concluded that in most cases, USAID’s goals and objectives
exceeded the agency’s span of influence and that it was therefore
extremely difficult for USAID to take credit for improving country-level
results. The Office of Management and Budget similarly noted in 1996,
after reviewing USAID’s strategic plan and indicators, that it was very
difficult to credit USAID projects with progress in country-level indicators.

USAID  is developing common indicators that missions will use to
consistently measure progress in key areas worldwide. These measures
are intended to help the agency aggregate the results of its various
missions’ programs to show progress in achieving overall agency goals.
However, the use of common indicators will not resolve USAID’s difficulty
in attributing gains to its programs at the country level. According to a
senior agency official, a key criterion for using these common indicators,
as with the mission-specific strategic objective indicators currently used,
will be that USAID can show a “plausible association” with the results, not
that the results are attributable to USAID assistance.

The extent of USAID’s plausible association with the country-level results is
not reflected in the missions’ documents. In some cases, especially in
countries where USAID is the largest donor and projects are showing some
results, this association may be very strong, while in other cases, the
association may be tenuous, based only on token USAID involvement. The
agency has not clearly and consistently differentiated between levels of
association with development results in its mission performance reports.
USAID management acknowledged that it needs to do a better job of making
the plausible associations clear in its strategic documents.

Citing broad development indicators can result in misleading reports on
USAID’s performance. The reports we reviewed showed that when missions



Page 15                                      GAO/NSIAD-97-194 Foreign Assistance
                            B-277413




                            reported that specific strategic objective performance targets were met,
                            they rarely mentioned other factors outside of USAID’s control that
                            contributed to the results, such as assistance from other donors, actions
                            taken by the host governments independently, or favorable international
                            market developments. Furthermore, the USAID Inspector General recently
                            reported that USAID seemed to be taking credit in its 1996 Agency
                            Performance Report for some high-level impact merely because it had
                            projects in that program area.


Program Evaluations Can     Although some of the missions we visited had done program evaluations
Demonstrate Effectiveness   that could be used to link project results with country-level development
but Are Not Routinely       indicators, USAID’s current performance measurement system relies largely
                            on indicators to assess the agency’s impact. However, in our recent report
Done                        on GPRA implementation, we noted that demonstrating the impact of a
                            government program on outcome indicators is difficult and a common
                            problem among federal agencies that are implementing performance
                            measurement systems.7 The most difficult aspect of analyzing and
                            reporting performance data is separating the impact of a program from the
                            impact of external factors to measure the program’s effect. We noted that
                            in these cases “it may be important to supplement performance
                            measurement data with impact evaluation studies to provide an accurate
                            picture of program effectiveness.”

                            Systematic evaluations of how a program was implemented can provide
                            managers with important information about a program’s success or failure.
                            For example, in Honduras, a USAID contractor’s evaluation of a basic
                            education project confirmed that USAID activities in that country,
                            implemented from 1986 to 1995, had a significant impact on key
                            country-level indicators of improvement of the educational system in that
                            country relative to other factors.

                            Before October 1995, missions were required to evaluate each project
                            upon completion and submit the results to USAID’s Center for Development
                            Information and Evaluation. However, current USAID guidance allows
                            missions to decide whether they will do performance evaluations of their
                            activities. In one mission we visited, a senior USAID official said that the
                            mission would rely on performance indicators, not on evaluations, to
                            monitor its projects’ progress and results. He said that evaluations would
                            be done primarily when implementation problems arise. At other missions,

                            7
                            Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138,
                            May 30, 1997).



                            Page 16                                                GAO/NSIAD-97-194 Foreign Assistance
                      B-277413




                      USAID officials also suggested that evaluations would probably be done less
                      often than in the past.

                      According to the Director of USAID’s Center for Development Information
                      and Evaluation, the agency’s revised policy on evaluation was not
                      necessarily intended to reduce the frequency of evaluations at the
                      missions, but to better target them to the management needs of the
                      mission. He said that missions are expected to do adequate evaluations to
                      support statements about performance included in mission reports.
                      However, documents we reviewed rarely cited evaluations to support their
                      descriptions of performance in economic growth strategic objectives.
                      Officials from the Center and a mission we visited told us that they
                      believed the agency needed to reassess its policies on evaluation and
                      mission practices to ensure that the agency is sufficiently and
                      appropriately using evaluations for effective management.

                      According to USAID management, as missions improve performance
                      monitoring, there will be less need for routine impact evaluations at the
                      activity level. Eventually, fully operational performance monitoring will
                      result in an adequate assessment of the impact of USAID’s activities in a
                      country. In addition, missions could supplement performance
                      measurement data by obtaining reviews of other donor programs and
                      host-country efforts. Nevertheless, periodic, independent evaluations of
                      USAID’s funded activities can be a useful tool to validate program
                      accomplishments.


                      USAID has begun using program performance in its resource allocation
Mission Performance   process in a more systematic manner. However, the process continues to
Is Being Linked to    evolve, and resource allocation decisions are affected primarily by foreign
Resource Allocation   policy considerations of the executive branch and congressional priorities.
                      Each of USAID’s regional bureaus used different formulas and assigned
                      different weights for performance when constructing their position on
                      USAID’s fiscal year 1998 budget request. However, guidance for the fiscal
                      year 1999 budget cycle attempts to standardize the bureaus’ assessment
                      processes, including the weight accorded to performance in budget
                      deliberations.




                      Page 17                                     GAO/NSIAD-97-194 Foreign Assistance
                            B-277413




Revised Resource            A basic principle underlying USAID’s reform efforts is to divide its
Allocation Process          assistance program funding among its strategic goals in line with carefully
Focuses on Program          crafted strategic plans and systems for performance measurement. USAID
                            now includes more explicit and systematic consideration of program
Contributions Within Each   performance in its resource allocation process.
Agency Goal
                            Mission officials told us USAID’s emphasis on performance and results was
                            good, but expressed mixed views on USAID’s allocation process. Officials
                            generally said political and foreign policy considerations continue to
                            dominate the budget process. While mission officials said the process is
                            more transparent than in the past, some said that USAID should be more
                            consultative with the missions regarding the allocation of earmarked
                            and/or directed funding. One mission director said there is little linkage
                            between the mission’s performance, as measured against the mission’s
                            “results contract” with USAID, and the allocation of resources to the
                            mission.

                            USAID’s process for assessing performance and allocating resources
                            continues to evolve, and modifications continue to be made in reporting
                            requirements and procedures for weighing program results. Performance
                            has been a factor in allocating resources. For example, when we observed
                            headquarters technical sessions for rating performance, we learned that
                            USAID was eliminating an agricultural loan program in Guinea due to low
                            rates of loan disbursements. We also learned that USAID was cutting
                            assistance to El Salvador’s electoral commission due to its poor
                            performance in implementing needed reforms. Performance assessments
                            may become more meaningful in the process as missions provide more
                            complete performance information. In the Asia and Near East Bureau, the
                            number of activities that reported complete performance information rose
                            from 31 percent in 1995 to 59 percent in 1996.

                            USAID allowed regional bureaus latitude in how they implemented the
                            reporting process for the 1998 fiscal year budget request, as well as for
                            identifying priorities for fiscal year 1997 resource allocations. The Asia and
                            the Near East Bureau scored and ranked, within each of the agency’s four
                            sustainable development goals, each strategic objective by performance,
                            contribution to agency and bureau priorities, contribution to foreign policy
                            goals, and the extent to which the host country had been a good
                            development partner for that objective. The ranking became a starting
                            point for further discussions on what activities should be funded. The
                            Bureau for Africa also used a similar approach, scoring and ranking
                            strategic objectives within each sector on performance, pipeline,



                            Page 18                                      GAO/NSIAD-97-194 Foreign Assistance
B-277413




host-country performance in the sector, sectoral need and magnitude of
the problem, and contribution to regional initiatives and priorities for that
sector. The Bureau for Latin America and the Caribbean scored and
ranked each strategic objective by performance, then scored and ranked
countries by performance, foreign policy interest (such as funding Haitian
democratization efforts or the Guatemalan peace process), severity of
need, and commitment to free market policies and democratic
governance. These scores were used to place countries in four funding
categories; those in the highest category received funding priority, after
allowing for funding needs based on analysis of the pipeline and for
meeting targets for earmarks and directives.

The USAID Administrator issued guidance in January 1997 to standardize
the bureaus’ resource requests, performance assessments, and resource
allocation processes for the fiscal year 1999 budget request. Among other
things, resource requests and allocations are expected to be based on
strategic objective performance and its significance to the strategic plan.
Bureaus are to use three common factors and common weights to
measure performance and allocate resources: performance (35 percent),
contribution to agency goals (30 percent), and contribution to
development initiatives (35 percent). While this guidance provides a more
standard ranking and scoring format, these groupings still allow wide
latitude for including factors that are still considered in the allocation
process. For example, contribution to agency goals includes foreign policy
objectives, and contribution to development initiatives includes bureau
initiatives, country and/or sectoral need, and the quality of the
development partnership in general and within specific goal areas.

We attended several USAID program review meetings during this year’s
deliberations on the fiscal year 1999 budget at the Africa, Asia/Near East,
and Latin America bureaus. Generally, the initial discussions on strategic
objective performance were chaired by technical staff from the regional
bureaus with varying representation from the Global, Management, and
Policy and Program Coordination bureaus. The Asia/Near East and Latin
America bureaus held several levels of reviews in Washington for each
country, often with senior mission management and mission program
staff. Due to the large number of missions in Africa, the Washington teams
conducted the initial assessment based on the submissions and, in some
cases, additional information sent from the field. The Africa Bureau held
subsequent 1-day program reviews with staff from full sustainable
development missions and with missions and operating units with
upcoming strategy reviews. Following the rounds of technical and country



Page 19                                      GAO/NSIAD-97-194 Foreign Assistance
                 B-277413




                 meetings, bureaus conducted regional wrap-up sessions to reach resource
                 allocation decisions and prepare for budget reviews with agency
                 management.

                 At the technical meetings we attended, we noted that the teams had
                 reviewed the documents, and, in some cases, received additional
                 information from the missions. Team members that had been given
                 guidance on the criteria for scoring performance appeared well prepared,
                 usually supporting their scores with specific examples of how strategic
                 objectives were or were not meeting targets or making an impact. In some
                 cases, team members discussed the quality and relevance of performance
                 indicators and targets and the extent to which impact could be attributed
                 to USAID assistance. We also heard some limited discussion on the quality
                 of the data used to measure performance.

                 This is only the second year of USAID’s revised budget process, and USAID
                 officials acknowledged that modifications will continue to be made as best
                 practices are identified. We noted at several meetings that the team
                 members and mission staff were confused about some of the mission
                 reporting. For example, the guidance asked missions to report on the
                 intermediate results that were most central to achieving the goals of the
                 strategic objectives. However, there was discussion about whether the
                 reported information was the best reflection of performance. Some team
                 members cabled to missions for additional information and used this in
                 their scoring; others did not. The quality of the reports and the extent to
                 which they provide an accurate and complete story of each strategic
                 objective are key factors in performance assessment.


                 Although USAID will have spent about $100 million by the end of fiscal year
New Management   1998 to develop its New Management System (NMS), the system does not
System Is Not    work as intended and has created serious problems in mission operations
Working          and morale. The system is key to successful information sharing required
                 under reengineering for accountability and control. The agency deployed
                 the system worldwide, knowing that it was not fully operational or
                 adequately tested. Indeed, USAID’s Inspector General reported 6 months
                 after implementation of the system that it was not complete, had not been
                 demonstrated to work effectively, and had not been adequately tested.8
                 Because of problems with the system, USAID suspended use of part of the



                 8
                 Audit of the Worldwide Deployment of the New Management System (NMS) (USAID/OIG,
                 A-000-97-004P, Mar. 31, 1997).



                 Page 20                                              GAO/NSIAD-97-194 Foreign Assistance
                             B-277413




                             system in the missions in April 1997 and is now taking corrective steps
                             recommended by the Inspector General.


System Is Key to USAID       Since 1994, USAID has been developing the NMS to support its organizational
Reforms                      reforms. The system is designed to consolidate USAID’s accounting, budget,
                             personnel, procurement, program operations, and property management
                             into a single, integrated network that the agency’s missions and offices
                             worldwide can access and to aid in the effective management and
                             monitoring of its programs.9 The purpose of the system is to

                         •   make financial management more efficient by streamlining business
                             processes, eliminating paper forms, ensuring its compliance with federal
                             accounting and financial management requirements, and providing
                             managers with information needed to make appropriate decisions and
                             reliably report the status of USAID projects;
                         •   facilitate missions’ program delivery by providing the means for missions
                             and offices to share information on-line about program management; and
                         •   empower missions and provide USAID management with a means for
                             maintaining accountability.

                             The system was activated at USAID headquarters in July 1996 and at the
                             missions in October 1996. As of October 1996, four of the six system
                             modules—the accounting, budget, operations, and procurement
                             modules—were operational, with some limitations, at USAID headquarters.
                             The personnel and property management modules are still in
                             development. According to USAID’s Office of the Inspector General, the
                             agency will have invested about $89 million by the end of fiscal year 1997
                             and about $100 million by the end of fiscal year 1998 to develop the
                             system.


Dysfunctional NMS Has        The four missions we visited could not routinely use NMS to successfully
Hampered Operations at       execute financial management functions such as obligating funds and
Missions                     recording procurement actions. According to mission officials, having a
                             dysfunctional system limits the benefits of the missions’ reengineered
                             operations. The Honduras mission director told us that “the reform
                             process has now reached a point where some reforms as contemplated
                             simply cannot go much further without a functioning NMS.”

                             9
                              See Foreign Assistance: Status of USAID’s Reforms (GAO/NSIAD-96-241BR, Sept. 24, 1996); Interim
                             Report on the Status of USAID’s New Management System (USAID/OIG, A-000-96-001-S, Sept. 27,
                             1996); and Interim Report on the Cost of USAID’s New Management System (USAID/OIG,
                             A-000-96-002-S, Sept. 27, 1996).



                             Page 21                                                  GAO/NSIAD-97-194 Foreign Assistance
B-277413




For several months, the mission in Honduras could not obligate allocated
funds for disaster relief on the country’s north coast due to problems in
the system’s accounting and procurement modules. Also, because of the
system’s long response times, entering information into the system was
often very time-consuming and frustrating for mission staff. Officials in
Honduras attempted to run some procurement actions while we observed:
the actions took from 2 to 6 hours to process, and none could be
successfully completed. In fact, because of these problems, some mission
officials we visited had two personal computers on their desks: one for
using NMS and one to do other work. In El Salvador, some mission
personnel told us that all other work frequently stopped while NMS was in
use and the system was having a negative impact on implementation of
mission projects.

In addition to citing excessive system response times as a major problem,
the agency’s Chief of Staff noted potential problems with the system
development effort. He said that because each module was independently
developed, the database used for NMS had some inconsistencies. For
example, in one module, for some data fields the system will accept eight
characters, while another module will accept only six characters for the
same information. Such inconsistencies can cause problems when
attempting to transfer data from one module to another.

In March 1997, USAID’s Inspector General reported that the system’s
premature deployment had increased the risk of fraud and abuse, had not
met users’ needs, and did not meet basic federal financial system
requirements. These findings were consistent with our observations. The
Inspector General made a number of recommendations to USAID
management, including suspension of part of the system until many of its
problems were resolved.

In April 1997, following the Inspector General’s report and comments from
system users and us, USAID suspended the NMS accounting and
procurement modules and left the budget and operations modules
operational in the missions. These systems are expected to remain shut
down until at least fiscal year 1998. All four modules will still be used by
headquarters offices, which have not had as much trouble in using NMS as
have the missions.

USAID   said it is taking the following steps to address problems with the NMS:




Page 22                                       GAO/NSIAD-97-194 Foreign Assistance
                  B-277413




              •   A new Director of Information Resources Management, who will focus
                  exclusively on managing information resource activities, has been hired.
              •   The Administrator has announced that he will designate a senior official as
                  the agency’s Chief Information Officer, who will be certified by the Office
                  of Management and Budget as qualified for the position and report directly
                  to the Administrator.
              •   A full-time NMS project manager has been selected to supervise the
                  development efforts until the system is operational.
              •   Efforts have been undertaken to analyze the technical and implementation
                  problems that currently limit NMS from achieving its full potential.
              •   A goal to achieve level 2 of the Software Engineering Institute’s Capability
                  Maturity Model (CMM) for software development has been established.10
              •   A statement of work has been developed to assess the accounting and
                  procurement components of NMS and their quality and to identify the risks
                  and opportunities in the application code that the components use.
              •   A transition to performance-based contracting for NMS and a reduction in
                  both the number of contract entities and contractors are underway.
              •   Development activities are limited to the portions of NMS that are still
                  operational and needed to establish the core functionality in the
                  accounting module.


                  It is too early to discern the full extent of the impact of reengineering
Conclusions       efforts at USAID missions, as they have only recently made the investment
                  and operational changes necessary to bring about long-term change.
                  However, USAID’s reengineering efforts have produced some tangible
                  benefits in the areas of planning, implementing, and monitoring assistance
                  projects. Missions are adapting to the strategic approach to planning, but
                  missions’ progress in changing their portfolios of projects has been
                  constrained by factors such as funding limitations and obligation
                  authority. The extent to which USAID can capitalize on operational changes
                  will depend on continued vigilance by agency management to ensure that
                  these changes take hold and continue. The ultimate test will be whether
                  USAID can achieve sustained efficiencies in project planning and
                  implementation and demonstrate the impact of its projects. With limited
                  resources available for foreign assistance, USAID’s ability to target its
                  assistance effectively and deliver it efficiently is especially important.

                  10
                    The CMM for Software is a framework of key elements of an effective software process. The CMM
                  shows an evolutionary improvement path for software organizations that consists of 5 levels with
                  level 1 (termed “Initial”) representing an immature process. Level 2 (termed “Repeatable”) is defined
                  as basic project management processes that have been established to track cost, schedule, and
                  functionality. In addition, the necessary process discipline is in place to repeat earlier successes on
                  projects with similar applications. USAID is also requiring in its new software development contracts
                  that the firms be independently judged as operating at a minimum of CMM level 2.



                  Page 23                                                      GAO/NSIAD-97-194 Foreign Assistance
                  B-277413




                  Since information-sharing is critical to USAID’s new operating system,
                  diligence in properly deploying NMS is key to the reform process. While NMS
                  promises to aid in more efficient management of USAID resources, many of
                  the efficiencies expected to result from this system have been delayed and
                  offset by unsuccessful implementation. USAID has taken positive steps to
                  address the concerns discussed in this report and the recommendations in
                  its Inspector General’s report. However, whether these actions will correct
                  the NMS’ problems will depend on the successful implementation of these
                  efforts. In addressing the problems with NMS, it is USAID management’s
                  responsibility to ensure that NMS facilitates, rather than impedes, the
                  reform process.


                  In commenting on a draft of this report, USAID stated that the report was “a
Agency Comments   fair, balanced, and thoughtful assessment of the state of overseas
                  missions’ reengineering efforts.” USAID comments provided some suggested
                  clarifications to the draft report and additional information on agency
                  efforts to address problems associated with the New Management System,
                  which we have incorporated in the text of our report, as appropriate.
                  USAID’s letter, without attachments, is reprinted in appendix IV.



                  As requested, we examined how USAID missions have reformed their
Scope and         operations and developed a results-oriented program. In making our
Methodology       selections for overseas fieldwork, we asked USAID to recommend five
                  countries in each of its regions where reengineering has progressed to the
                  point that we could see results and the economic growth objective was a
                  priority. USAID provided us program budget data on all countries in which it
                  operates and five country recommendations per region, along with
                  explanations for their selections. Following discussions with USAID officials
                  and congressional Committee staff, we selected El Salvador, Honduras,
                  Bangladesh, and the Philippines for fieldwork. We based our country
                  selections on the size of the USAID program, importance of the economic
                  growth strategic objective in the overall program, and pipeline size. We
                  had done fieldwork in the Dominican Republic for the first part of the
                  assignment. We focused on Latin America and Asia because of the greater
                  emphasis on economic growth in these regions; we excluded Central
                  Europe and the former Soviet Union region from consideration since these
                  countries’ programs are relatively new, transitional, and expected to end
                  soon.




                  Page 24                                     GAO/NSIAD-97-194 Foreign Assistance
B-277413




To assess (1) how USAID missions have reengineered their operations and
(2) how USAID monitors and evaluates the results of its projects, we
interviewed USAID officials in the Management, Global, and regional
bureaus and the Bureau for Policy and Program Coordination, including
the Center for Development Information and Evaluation in Washington,
D.C.; reviewed agency policies, practices, and procedures; and analyzed
agency documentation, including evaluations, reporting documents, and
Inspector General reports. At the missions visited, we interviewed USAID
management and staff, held discussions with foreign service nationals on
the impact of reforms and reengineering, and collected and analyzed
relevant documentation. We also interviewed representatives of
nongovernmental and private voluntary organizations and USAID
contractors that implement USAID programs and officials of international
financial institutions, other donor countries, and the host governments.

To assess how reengineering has affected the delivery of development
assistance, we interviewed USAID officials, partners, and customers about
reengineered program design and delivery and reviewed documentation
on mission strategies, plans, and projects. In reviewing portfolio changes
at the missions we visited, we assessed the active, completed, terminated,
or initiated projects for each year between fiscal year 1994 and 1997.

To determine how USAID allocates funds for its projects, we interviewed
USAID officials, attended regional bureau budget sessions, examined the
budget process, and reviewed recent and current budget documents and
agency guidance. We assessed how the budget process relates the various
factors that the agency uses to measure and rank its performance to
provide some insight as to whether the budget process is meeting
expectations.

To assess how USAID’s NMS supports mission operations, we met with
officials of USAID’s Office of the Inspector General and reviewed their
reports on NMS.11 We reviewed agency documentation, including agency
cable traffic and e-mails in Washington, D.C., and the field missions
visited; USAID’s general notice system and policy directives, guidance, and
statements; and congressional testimony by the Administrator.12 We
interviewed USAID officials and agency staff who use NMS in Washington,

11
 See Interim Report on the Status of USAID’s New Management System (USAID/OIG, A-000-96-001-S,
Sept. 1996); Interim Report on the Cost of USAID’s New Management System (USAID/OIG,
A-000-96-002-S, Sept. 1996); and Audit of the Worldwide Deployment of the New Management System
(NMS) (USAID/OIG, A-000-97-004P, Mar. 1997).
12
 J. Brian Atwood, Administrator, USAID, before the House Appropriations Committee, Subcommittee
on Foreign Operations, March 19, 1997.



Page 25                                                 GAO/NSIAD-97-194 Foreign Assistance
B-277413




D.C., and in the field missions and observed mission officials’ and staff
members’ use and/or tests of NMS. We reviewed our prior reports: a
December 1992 report identifying weaknesses in federal information
resources management that frequently led to costly projects that show
disappointing results,13 numerous reports and testimonies since 1993 on
information management overall,14 and a February 1997 report noting that
failure to follow disciplined management and system development
practices could lead to costly failures.15

We performed our work from October 1996 to May 1997 in accordance
with generally accepted government auditing standards.


We are providing copies of this report to the Chairmen and Ranking
Minority Members of the House and Senate Committees on
Appropriations, the Senate Committee on Foreign Relations, the House
Committee on Government Reform and Oversight, and the Senate
Committee on Governmental Affairs, and to the Ranking Minority Member
of the House Committee on International Relations. We are also sending
copies to the Administrator, U.S. Agency for International Development,
and to the Director, Office of Management and Budget. Copies will be
made available to others upon request.

Please contact me at (202) 512-4128 if you or your staff have any questions
about this report. Major contributors to this report are listed in
appendix V.

Sincerely yours,




Benjamin F. Nelson
Director, International Relations and
  Trade Issues

13
 See Information Management and Technology Issues (GAO/OCG-93-5TR, Dec. 1992) and Information
Resources: Summary of Federal Agencies’ Information Resources Management Problems
(GAO/IMTEC-92-13FS, Feb. 1992).
14
 See Information Technology Investment: Agencies Can Improve Performance, Reduce Costs, and
Minimize Risks (GAO/AIMD-96-64, Sept. 1996); Information Management Reform: Effective
Implementation Is Essential for Improving Federal Performance (GAO/T-AIMD-96-132, July 1996);
Executive Guide: Improving Mission Performance Through Strategic Information Management and
Technology (GAO/AIMD-94-115, May 1994); Foreign Assistance: AID Strategic Direction and Continued
Management Improvements Needed (GAO/NSIAD-93/106, June 1993).
15
  See Information Management and Technology (GAO/HR-97-9, Feb. 1997).
Page 26                                                 GAO/NSIAD-97-194 Foreign Assistance
Page 27   GAO/NSIAD-97-194 Foreign Assistance
Contents



Letter                                                                                           1


Appendix I                                                                                      30

USAID Obligations,
Fiscal Years 1995-97
Appendix II                                                                                     31

Directed and
Undirected Funds for
Development
Assistance and
Related Accounts,
Fiscal Years 1995-97
Appendix III                                                                                    32

Chronology of USAID
Reforms
Appendix IV                                                                                     33

Comments From the
U.S. Agency for
International
Development
Appendix V                                                                                      34

Major Contributors to
This Report

                        Abbreviations

                        CMM       Capability Maturity Model
                        GPRA      Government Performance and Results Act
                        NMS       New Management System
                        USAID     U.S. Agency for International Development


                        Page 28                                 GAO/NSIAD-97-194 Foreign Assistance
Page 29   GAO/NSIAD-97-194 Foreign Assistance
Appendix I

USAID Obligations, Fiscal Years 1995-97



Dollars in thousands
                                                                              Fiscal year
Sustainable development
goal areas and operating           1995                                          1996                                  1997 (est.)
expenses                   Obligation                 Percent          Obligation              Percent          Obligation           Percent
Economic growth            $2,398,897                     40.1         $1,980,349                  38.4         $1,996,967              38.0
Democracy                    436,506                          7.3         388,805                    7.5           398,777               7.6
PHNa                        1,125,784                     18.8            934,700                  18.1            968,510              18.4
Environment                  495,098                          8.3         377,945                    7.3           397,499               7.6
Otherb                      1,533,031                     25.6          1,476,065                  28.6          1,495,656              28.4
Total program funds         5,989,316                    100.0          5,157,864                 100.0          5,257,409             100.0
Operating expensesc          612,164                            d
                                                                          561,732                       d
                                                                                                                   608,311                 d

                                                                d                                       d                                  d
Total                      $6,601,480                                  $5,719,596                               $5,865,720
                                        a
                                        Population, health, and nutrition.
                                        b
                                            The amounts for “Other” include the $1.2 billion provided to Israel each year.
                                    c
                                     Includes the U.S. Agency for International Development (USAID) and USAID Office of Inspector
                                    General operating expenses and Foreign Service Retirement and Disability funds.
                                        d
                                            Not applicable.

                                        Sources: Program funds: USAID, Bureau for Management, Office of the Budget. Operating
                                        expenses: USAID Congressional Presentation, Summary Tables, Fiscal Year 1998.




                                        Page 30                                                       GAO/NSIAD-97-194 Foreign Assistance
Appendix II

Directed and Undirected Funds for
Development Assistance and Related
Accounts, Fiscal Years 1995-97

              Dollars in thousands
                                                                              Fiscal year
              Account                                              1995                 1996             1997
              Child survival                                  $321,092             $352,992          $502,000a
              Basic child’s education                           104,406               88,000           98,000
              Transfers                                                0              56,500           51,500
              U.S. Telecommunications                                380                  400             500
              Training Institute earmark
              Other directivesb                                 302,084              259,173          233,798
              Population                                        514,724              356,000          355,000
              Total directed                                  1,242,686            1,113,065         1,240,798
              Undirected funds available for                    834,382              561,935          540,702
              economic growth, democracy,
              and environment
              Total development assistance                  $2,077,068           $1,675,000         $1,781,500
              Undirected funds as                                   40.2                 33.5             30.4
              percentage of development
              assistance
              a
              Includes $100 million grant to United Nations Childrens Fund.
              b
                  Includes directives reported by USAID as contributing to economic growth goals.

              Source: USAID, Bureau for Management, Office of the Budget.




              Page 31                                                      GAO/NSIAD-97-194 Foreign Assistance
Appendix III

Chronology of USAID Reforms



                                                                         Downsizing                      Fiscal year
                                                                         during                          obligations
       Reform milestones                                                 fiscal year                     and staffing

1993     October: USAID reorganization plan announced by Administrator                                     $6.8 billion
                                                                                                           11,150
         November: Planned closing of 21 missions announced                                                employees


1994     March: Strategies for sustainable development announced          5 missions closed                $7.0 billion
                                                                          Staff reduced by 1,448           9,702
         October: Reengineered program operations processes tested                                         employees
         in 10 missions


1995     February: Operations business area analysis completed             10 missions closed              $6.6 billion
                                                                           Staff reduced by 550            9,152
         March: Overseas reinvention impact review completed                                               employees

         March: Agency strategic implementation guidelines issued

         September: One-year experiment on reengineered program
         operation processes completed

         September: Agency strategic framework and indicators for FY
         1995-96 issued

         October: New program operations procedures issued

         October: Agency Automated Directives System implemented

         October: Partial deployment of New Managment System
         (NMS)


1996     February: Results Review and Resource Request (R4) process        6 missions closed               $5.7 billion
         implemented                                                       Staff reduced by 1,126          8,026
                                                                                                           employees
         March: Reengineering best practices report issued (based on
         test of reengineered processes in 10 missions)

         July: NMS activated in Washington, D.C.

         October: NMS deployed worldwide
                                       Source: USAID.



1997     February: REFORM Advisory Group visits selected missions to       3 mission closings planned      $5.9 billion
         assist with reengineering                                         Staff reduced by 417 (as of     (estimated)
                                                                           6/30/97)                        7,609
         April: Two modules of NMS suspended at missions                                                   employees
                                                                                                           (as of
                                                                                                           6/30/97)



                                           Page 32                                           GAO/NSIAD-97-194 Foreign Assistance
Appendix IV

Comments From the U.S. Agency for
International Development




              Page 33         GAO/NSIAD-97-194 Foreign Assistance
Appendix V

Major Contributors to This Report


                        Jess T. Ford
National Security and   Lawrence L. Suda
International Affairs   John D. DeForge
Division, Washington,   Ann L. Baker
                        Bruce L. Kutnick
D.C.                    James B. Michels
                        Nancy L. Ragsdale
                        Audrey E. Solis
                        James M. Strus


                        John C. Martin
Accounting and
Information
Management Division,
Washington, D.C.
                        Donna J. Byers
General Government
Division, Washington,
D.C.
                        Ernie E. Jackson
Office of General
Counsel, Washington,
D.C.




(711231)                Page 34             GAO/NSIAD-97-194 Foreign Assistance
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested