oversight

Public Financial Management: Improvements Needed in USAID's and Treasury's Monitoring and Evaluation Efforts

Published by the Government Accountability Office on 2012-09-13.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to the Ranking Member,
                 Committee on Foreign Relations, U.S.
                 Senate


September 2012
                 PUBLIC FINANCIAL
                 MANAGEMENT
                 Improvements Needed
                 in USAID’s and
                 Treasury’s Monitoring
                 and Evaluation Efforts




GAO-12-920
                                             September 2012

                                             PUBLIC FINANCIAL MANAGEMENT
                                             Improvements Needed in USAID's and Treasury's
                                             Monitoring and Evaluation Efforts
Highlights of GAO-12-920, a report to the
Ranking Member, Committee on Foreign
Relations, U.S. Senate




Why GAO Did This Study                       What GAO Found
Effective management of public               To develop programs to strengthen developing countries’ Public Financial
resources can play an important role in      Management (PFM) systems, the U.S. Agency for International Development
a country’s development. In recent           (USAID) and the U.S. Department of the Treasury (Treasury) rely on
years, developing countries committed        assessments of the host country government’s systems. In 2011, USAID
to strengthen their PFM systems and          implemented new processes that place a greater emphasis on PFM in its
donors committed to use those                development efforts as the agency aims to increase its use of country systems to
systems as much as possible. The             deliver assistance. The agency traditionally included PFM capacity-building
United States provides assistance to         efforts only as components of broader programs, as it identified relevant
strengthen PFM systems primarily
                                             weaknesses during the country assessment or program design process. USAID’s
through USAID and Treasury. USAID
                                             new strategy and program development processes include a mandatory
conducts capacity building activities to
strengthen PFM systems as part of its
                                             assessment of a country’s institutional capacity, including its financial systems,
development programs and has also            and a requirement to consider the use of country systems to deliver assistance.
set a target to obligate 30 percent of its   Most USAID country offices are required to develop a strategy using the new
annual assistance through local              guidance by the end of fiscal year 2013. Treasury’s process for developing
systems by 2015. Treasury provides           programs begins with an initial assessment of the host country’s capabilities.
technical assistance through advisors        Treasury staff then draft objectives for the program. For example, a Treasury
who work in country, typically with the      program in Honduras set four objectives, including improving operational
finance ministry.                            efficiency and enhancing accountability by strengthening the organization of the
                                             ministry of finance. Once in country, the advisor develops an annual workplan,
GAO was asked to examine the
                                             outlining more specific goals aimed at meeting the overall objectives.
processes U.S. agencies use to (1)
develop programs to strengthen PFM           USAID and Treasury use several processes to monitor and evaluate their PFM
systems and (2) monitor and evaluate         assistance, but weaknesses exist. USAID uses its regular procedures, which
those programs. GAO reviewed                 may include performance management plans, periodic progress reporting, site
agency guidance and program                  visits, and evaluations, to monitor and evaluate its PFM-related programs. Prior
documents, interviewed U.S. agency           reports by USAID’s Inspector General and GAO have found weaknesses in
officials, and selected case studies to      USAID’s implementation of its monitoring procedures in other programs,
serve as illustrative examples of PFM-       including programs from the USAID offices that provide PFM assistance. In
related programs.
                                             addition, USAID is currently unable to monitor overall progress toward its target
What GAO Recommends                          to obligate 30 percent of its program funds through local systems by 2015.
                                             USAID, and GAO in prior reports, have identified a number of weaknesses in
USAID should improve its capacity to         evaluation practice. To address weaknesses the agency had identified, USAID
measure its use of local systems and         adopted a new evaluation policy in January 2011 that states that all large
ensure adequate monitoring of its PFM        projects are required to have an external evaluation, 3 percent of program
programs. Treasury should implement          budgets should be devoted to external evaluation, and evaluations must use
additional controls to improve the
                                             methods that generate the highest quality evidence. Treasury’s processes for
process for computing program-wide
                                             monitoring and evaluating its programs include monthly reports, annual
annual performance measures and
fully implement its requirement to           quantitative performance measures, voluntary customer feedback surveys, and
evaluate the impact of its completed         on-site management reviews, but Treasury does not fully evaluate the
assistance. USAID and Treasury both          performance of its completed technical assistance programs. In addition,
concurred with GAO’s                         Treasury’s quantitative performance measures have been a useful project-level
recommendations.                             indicator of performance but have not been a useful indicator of overall
                                             performance due in part to inherent challenges associated with summarizing
                                             program performance and errors introduced when aggregating the performance
                                             data. Furthermore, a senior Treasury official reported that Treasury had not yet
View GAO-12-920. For more information,       fully implemented a requirement to conduct independent postproject evaluations
contact Thomas Melito at (202) 512-9601 or   of its technical assistance programs.
melitot@gao.gov.

                                                                                     United States Government Accountability Office
Contents


Letter                                                                                     1
               Background                                                                  3
               USAID’s and Treasury’s Processes for Developing PFM Programs
                 Involve Consultation with Stakeholders and Rely on
                 Assessments of Host-Country Systems                                     13
               USAID and Treasury Use Certain Processes to Monitor and
                 Evaluate PFM-Related Programs, but Implementation Has Some
                 Weaknesses                                                              21
               Conclusions                                                               32
               Recommendations for Executive Action                                      33
               Agency Comments and Our Evaluation                                        34

Appendix I     Scope and Methodology                                                     36



Appendix II    Several International Organizations Reported Results of Their Public
               Financial Management Diagnostic Tools                                     39



Appendix III   State Department Conducts Fiscal Transparency Reviews and
               Administers the Domestic Finance for Development Initiative               48



Appendix IV    Examples of Public Financial Management-related Projects in Six
               Countries                                                                 51



Appendix V     Comments from the U.S. Agency for International Development               59



Appendix VI    Comments from the Department of the Treasury                              63



Appendix VII   GAO Contact and Staff Acknowledgments                                     65




               Page i                                  GAO-12-920 Public Financial Management
Tables
          Table 1: Selected PFM-Related Commitments in the Paris
                   Declaration and the Accra Agenda                                   8
          Table 2. Status of Countries in the PFM Risk-Assessment
                   Framework Process                                                18
          Table 3: USAID’s Kosovo Growth and Fiscal Stability Initiative:
                   Selected Objectives and Expected Results                         52
          Table 4: USAID’s Liberia GEMS: Selected Objectives,
                   Subobjectives, and Expected Results                              53
          Table 5: USAID’s Peru ProDecentralization 2: Selected Objectives
                   and Expected Results                                             54
          Table 6: Treasury’s Technical Advisory Services to Cambodia on
                   Budgeting: Selected Objectives and Activities                    55
          Table 7: Treasury’s Technical Advisory Services on Honduran
                   Budgeting: Selected Objectives and Action Plan Activities        56
          Table 8: Treasury’s Technical Advisory Services to South Africa on
                   Budgeting: Selected Objectives and Activities                    57


Figures
          Figure 1: Illustrative Example of a Country’s Budget Cycle                  4
          Figure 2: Countries Receiving U.S. PFM-Related Assistance since
                   2007                                                             10
          Figure 3. Select Components of USAID’s New Strategy
                   Development and Project Design Processes That May
                   Identify the Need for PFM Capacity Building                      15
          Figure 4: Open Budget 2010 Index Scores                                   43
          Figure 5: Transparency International 2010 Corruption Perceptions
                   Scores                                                           45
          Figure 6. Case Study Country Percentile Rankings on PFM
                   Diagnostic Tools                                                 47




          Page ii                                 GAO-12-920 Public Financial Management
Abbreviations

DF4D              Domestic Finance for Development
DG                Democracy and Governance Office, U.S. Agency for
                   International Development
EG                Economic Growth Office, U.S. Agency for International
                    Development
IMF               International Monetary Fund
OECD              Organization for Economic Cooperation and Development
OTA               Office of Technical Assistance, U.S. Department of the
                   Treasury
PEFA              Public Expenditure and Financial Accountability Program
PFM               Public Financial Management
PFMRAF            Public Financial Management Risk Assessment
                    Framework
STATE             U.S. Department of State
TREASURY          U.S. Department of the Treasury
USAID             U.S. Agency for International Development

This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page iii                                          GAO-12-920 Public Financial Management
United States Government Accountability Office
Washington, DC 20548




                                   September 13, 2012

                                   The Honorable Richard G. Lugar
                                   Ranking Member
                                   Committee on Foreign Relations
                                   U.S. Senate

                                   Dear Senator Lugar:

                                   A country’s development is influenced, in part, by how effectively its
                                   government raises, manages, and expends public resources. In 2005, as
                                   part of the Paris Declaration on Aid Effectiveness, developing countries
                                   committed to strengthen their public financial management (PFM)
                                   systems and donors committed to use those systems to the maximum
                                   extent possible. More transparent and effective PFM systems can make
                                   governments more accountable and give donors more assurance that
                                   funds will not be misspent if they deliver more aid through recipient
                                   countries’ budgets and finance systems. Highlighting the need for
                                   stronger PFM, international organizations have raised concerns about low
                                   levels of transparency in countries’ budgets and high levels of corruption.

                                   The United States provides assistance to strengthen other countries’ PFM
                                   systems primarily through the U.S. Agency for International Development
                                   (USAID) and the U.S. Department of the Treasury (Treasury). USAID has
                                   set a target of obligating 30 percent of the agency’s total annual
                                   assistance, or program funds, through local systems, including both
                                   partner country PFM systems and local not-for-profit and for-profit
                                   organizations, by fiscal year 2015. 1 Over the last decade, USAID has
                                   provided most of its assistance through agreements with large
                                   international implementing partners. USAID assesses countries’ PFM
                                   systems to determine their ability to effectively receive and manage U.S.
                                   assistance directly. USAID primarily provides PFM capacity-building
                                   assistance as a component of a broader development program, whereas
                                   the Treasury’s Office of Technical Assistance (OTA) provides PFM-
                                   related technical assistance. Since 2007, USAID and Treasury have
                                   provided PFM assistance to 70 countries. The U.S. Department of State


                                   1
                                    Since USAID is working to commit significantly more to partner country systems in
                                   Pakistan and Afghanistan, USAID excludes these two countries from the target as they
                                   would significantly distort the agency’s overall figures.




                                   Page 1                                          GAO-12-920 Public Financial Management
(State) assesses the budget transparency of countries that receive U.S.
assistance and helps to leverage existing PFM assistance, but has not
funded PFM assistance.

In this context, you asked us to examine: (1) the processes U.S. agencies
use to develop programs to strengthen PFM systems in developing
countries and (2) the processes U.S. agencies use to monitor and
evaluate their programs to strengthen PFM systems.

To address these objectives, we reviewed USAID, Treasury, and State
guidance and program documents and interviewed U.S. officials in
Washington, D.C. To address the first objective, we reviewed relevant
documents, including assessment reports, agency guidance, and
assistance agreements. 2 To address the second objective, we reviewed
agency and Organization for Economic Cooperation and Development
(OECD) 3 guidance and program documents, including assistance
agreements, progress reports, evaluations, work plans, and performance
management plans; and quantitative performance data. We spoke with
USAID and Treasury officials about their development, monitoring, and
evaluation of PFM-related programs. We also selected six case study
countries in which Treasury or USAID has relevant ongoing or recently
completed projects focused on strengthening PFM systems. We chose
these case study countries for their geographic and income level
diversity, as illustrative rather than generalizable examples. For Treasury,
we selected Cambodia, Honduras, and South Africa; and for USAID,
Kosovo, Liberia, and Peru. For each of our case study countries, we
interviewed U.S. officials and reviewed program development and
monitoring and evaluation documents related to PFM-focused projects.
Finally, we interviewed officials at the World Bank and International
Monetary Fund to discuss the tools that they use to assess PFM systems.

We conducted this performance audit from October 2011 through
September 2012 in accordance with generally accepted government


2
 Assistance agreements include all contracts, cooperative agreements, and grant
agreements between USAID and its implementing partners.
3
 OECD is is an international economic organization of 34 countries, including the United
States, founded in 1961 to stimulate economic progress and world trade. It is a forum of
countries committed to democracy and the market economy, providing a platform to
compare policy experiences, seek answers to common problems, identify good practices,
and coordinate the domestic and international policies of its members.




Page 2                                          GAO-12-920 Public Financial Management
                         auditing standards. Those standards require that we plan and perform the
                         audit to obtain sufficient, appropriate evidence to provide a reasonable
                         basis for our findings and conclusions based on our audit objectives. We
                         believe that the evidence obtained provides a reasonable basis for our
                         findings and conclusions based on our audit objectives. See appendix I
                         for a more detailed discussion of our scope and methodology.



Background
PFM Focuses on a         The broad objectives of PFM are to achieve overall fiscal discipline,
Country’s Budget Cycle   allocation of resources to priority needs, and efficient and effective
                         allocation of public services, according to OECD. While donors may use
                         different definitions of PFM, most definitions focus on a country’s budget
                         cycle, the process used to manage public resources. The budget cycle
                         centers around four main phases: (1) budget formulation, (2) budget
                         execution, (3) accounting and reporting, and (4) external oversight (see
                         fig. 1). OECD states that PFM includes all components of a country’s
                         budget cycle.




                         Page 3                                   GAO-12-920 Public Financial Management
Figure 1: Illustrative Example of a Country’s Budget Cycle




A budget cycle starts with the budget formulation process, in which the
government, often with legislative oversight, plans for the use of the
coming year’s resources in accordance with policy priorities. 4 After the
government approves the budget and the new fiscal year begins,
programming agencies and the ministry of finance, or appropriate entity,
are responsible for executing the budget. They use the resources


4
  The steps and entities involved in the budget cycle may differ between countries. Also,
some entities, such as the ministry of finance, may be referred to by different names, such
as the budget directorate, depending on the country.




Page 4                                            GAO-12-920 Public Financial Management
                           allocated to them for items such as salaries for public servants, operating
                           costs for their offices, and goods and services delivered to their
                           beneficiaries. The ministry of finance, or equivalent, manages the flow of
                           funds and monitors and makes in-year adjustments to help ensure
                           compliance with the budget and PFM rules. Throughout the fiscal year,
                           each programming agency is to account for and record its expenditures.
                           The ministry of finance centrally consolidates these accounts. At the end
                           of the fiscal year, the ministry of finance, or equivalent, issues an
                           accounting report that demonstrates how the budget was implemented.
                           External or independent entities, such as a country’s supreme audit
                           institution, may review this report. 5 The audit institution reviews the
                           government’s revenue collection and spending and issues its own
                           statement on the execution of the budget and the strength of the PFM
                           systems. In many countries, the institution presents this audit report to an
                           appropriate government entity for further scrutiny and action.

                           PFM processes involve a number of governmental entities. While the
                           ministry of finance is generally the focus of a country’s PFM system, PFM
                           extends to all public ministries that are charged with delivering services or
                           have spending authority. Civil society, donors, and oversight institutions,
                           such as a country’s supreme audit institution, also help to ensure the
                           proper management of public funds through external scrutiny and review.


Assessment Tools           Broad PFM assessments conducted by international organizations report
Highlight Both             improvements in countries’ PFM systems, as well as continued
Improvements and           weaknesses. Several international organizations, including the Public
                           Expenditure and Financial Accountability (PEFA) program, World Bank,
Weaknesses in Developing   International Budget Partnership, and Transparency International, have
Countries’ PFM Systems     developed assessment tools to assess broad aspects of PFM systems. 6
                           These assessments have highlighted varying levels of progress in
                           improving PFM systems around the world, and the last two have identified
                           relatively low levels of transparency in many countries’ budgets and high
                           levels of corruption, respectively. More specifically:



                           5
                            Supreme audit institution refers to any organization that sets standards for audit work.
                           GAO is the supreme audit institution for the United States.
                           6
                            The PEFA program grew out of a multidonor partnership that formed in 2001 to
                           strengthen the ability of aid recipients and donors to assess and improve country public
                           expenditure, procurement, and financial accountability systems.




                           Page 5                                            GAO-12-920 Public Financial Management
•    PEFA: A 2010 monitoring report on the PEFA program, released by
     the PEFA Secretariat, found that while PFM systems were improving
     overall, progress varied among elements of countries’ systems. 7 The
     PEFA secretariat’s analysis, based on a comparison of 33 repeat
     PEFA assessments from 2005 through 2010, showed that more
     countries had a higher number of improved versus worsened scores,
     indicating a broad trend of PFM improvement across the countries
     surveyed. According to the analysis, country actions taken to
     strengthen PFM features in the earlier stages of the budget cycle,
     such as planning, are more likely to improve or maintain a high score
     than actions taken later in the budget cycle, such as control and
     oversight of actual spending.

•    World Bank: According to our analysis of the World Bank’s quality of
     budgetary and financial management indicator scores from 2005
     through 2010, slightly more than one-third of the countries assessed
     showed improvements in the quality of their PFM systems, while one-
     quarter of the countries showed a worsening in the quality of their
     PFM systems. 8

•    International Budget Partnership: 9 In its 2010 open budget survey
     of 94 countries, the International Budget Partnership concluded that
     the state of budget transparency was poor. 10 Only about 21 percent of
     the countries surveyed had open budgets while 44 percent of
     countries provided limited to no budget information. Nonetheless, the
     survey found the trend toward open budgets was favorable based on
     substantial improvements in budget transparency, especially among



7
 PEFA Secretariat, PFM Performance Measurement Framework: Monitoring Report 2010
(Washington, D.C.: May 19, 2011).
8
 The World Bank conducts Country Policy and Institutional Assessments annually to rate
the quality of a country’s policy and institutional framework against 16 indicators, including
the quality of budgetary and financial management. The World Bank only publishes the
indicator scores of countries that are eligible for funding from the World Bank’s
International Development Association, which provides interest-free loans and grants to
governments of the poorest countries. See World Bank’s Country Policy and Institutional
Assessments 2010 Assessment Questionnaire.
9
 The International Budget Partnership is a nongovernmental organization that collaborates
with civil society around the world to reduce poverty and improve the quality of
governance by reforming government budget systems and influencing budget policies.
10
 International Budget Partnership, The Open Budget Survey 2010, (Washington, D.C.:
October 2010).




Page 6                                             GAO-12-920 Public Financial Management
                                  countries that had provided little information in the past. Some of
                                  these governments achieved improvements by simply making budget
                                  documents available on their websites.

                             •    Transparency International: 11 The organization’s Corruption
                                  Perceptions Index showed that nearly three quarters of the 178
                                  countries in the index scored below 5, on a scale of 10 (very clean) to
                                  0 (highly corrupt), indicating what it categorizes as a serious
                                  corruption problem. 12

                             For further information on each organization’s assessment tools and
                             selected results, including the percentile rankings of our six case study
                             countries in selected PFM diagnostic tools, see appendix II.


International Attention to   Donors and recipient governments have increased their attention to
Strong and Effective PFM     strengthening PFM systems, recognizing that strong and effective PFM
Systems Has Increased        systems underpin fiscal and macroeconomic stability, guide the allocation
                             of public resources to national priorities, support the efficient delivery of
since 2003                   services for poverty reduction and economic development, and make
                             possible the transparency and scrutiny of public funds. In 2003, OECD’s
                             Development Assistance Committee established a Working Party on Aid
                             Effectiveness, which has played a critical role in establishing an initial
                             international donor coordination framework and setting goals for donors
                             and aid recipient countries to strengthen PFM. The working party has
                             sponsored four global development forums since 2003. The second
                             forum, held in Paris in 2005, resulted in the Paris Declaration on Aid
                             Effectiveness which included broad commitments by recipient
                             governments and donors to strengthen PFM systems and use those
                             systems as appropriate. More than 100 countries and aid agencies,
                             including the United States, endorsed the declaration. By signing the
                             declaration, recipient governments made clear commitments to


                             11
                               Transparency International is a global civil society organization focused on the fight
                             against corruption. Through more than 90 chapters worldwide and an international
                             secretariat in Berlin, the organization raises awareness of the damaging effects of
                             corruption and works with partners in government, business, and civil society to develop
                             and implement effective measures to lessen it.
                             12
                               Transparency International defines corruption as the abuse of entrusted power for
                             private gain, and its definition encompasses corrupt practices in both the public and
                             private sectors. See Transparency International, Corruption Perceptions Index 2010
                             (Berlin, Germany: October 2010).




                             Page 7                                            GAO-12-920 Public Financial Management
strengthen their systems to the maximum extent possible, and donor
governments made clear commitments to use those systems wherever
possible. These commitments were renewed and refined in the
subsequent forum in Accra, Ghana, in 2008. The most recent forum was
held in Bussan, South Korea, in 2011. (See table 1.)

Table 1: Selected PFM-Related Commitments in the Paris Declaration and the Accra
Agenda

 Partner countries committed to:
 •   Carry out diagnostic reviews and undertake reforms to ensure national systems,
     institutions, and procedures for managing resources are effective, accountable, and
     transparent
 •   Intensify efforts to mobilize domestic resources, strengthen fiscal sustainability, and
     create an enabling environment for public and private investments
 •   Publish timely, transparent, and reliable reporting on budget execution
 •   Take leadership of PFM reform process
 Donors committed to:
 •   Use country systems and procedures to the maximum extent possible. Where use is
     not feasible, establish safeguards and measures in ways that strengthen rather than
     undermine country systems
 •   Provide reliable indicative commitments of aid over a multi-year framework and
     disburse aid in a timely and predictable manner according to agreed schedules
 •   Rely to the maximum extent possible on transparent partner government budget and
     accounting mechanisms
 Partner countries and donors jointly committed to:
 •   Implement harmonized diagnostic reviews and performance assessment frameworks
     in PFM
Source: GAO analysis of Paris Declaration on Aid Effectiveness, 2005, and the Accra Agenda for Action, 2008.




Page 8                                                                  GAO-12-920 Public Financial Management
U.S. Agencies Provide a   USAID and Treasury are the two main U.S. agencies providing PFM-
Variety of PFM-Related    related assistance. 13 Combined, the two agencies have PFM-related
Assistance                projects in 70 countries in all regions of the world, as shown in figure 2.
                          State also conducts some PFM-related activities, although it has not
                          funded programs. See appendix III for details on State’s PFM activities.




                          13
                            Other U.S. agencies provide some PFM-related assistance. For example, GAO provides
                          capacity-building support to fellow audit institutions through its membership in the
                          International Organization of Supreme Audit Institutions. In addition, the Millennium
                          Challenge Corporation (MCC) has minimum financial accountability requirements for
                          countries to be eligible to enter into assistance compacts. Deficiencies in these
                          requirements may be addressed through MCC’s threshold program, which is typically
                          administered by USAID.




                          Page 9                                        GAO-12-920 Public Financial Management
Figure 2: Countries Receiving U.S. PFM-Related Assistance since 2007




                                        a
                                         Only countries where assistance is at least partially funded by OTA are listed. OTA projects funded
                                        completely by other agencies are not included.




                                        Page 10                                                 GAO-12-920 Public Financial Management
        b
         U.S. appropriations law requires State to evaluate the fiscal transparency of foreign governments
        before providing certain U.S. assistance. If State determines it is in the U.S. national interest, it
        processes waivers for countries deemed nontransparent thereby allowing U.S. agencies to provide
        assistance to them. See Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, § 668(e), 121
        Stat. 1844, 2353 (2007) and Consolidated Appropriations Act, 2012, Pub. L. No. 112-74, § 7031(b),
        125 Stat. 786, 1211 (2011).


USAID   USAID provides PFM capacity-building activities through its development
        programs and is seeking to provide more of its assistance through
        recipient countries’ financial systems. Capacity-building activities to
        strengthen PFM systems are typically part of broader democracy and
        governance (DG) or economic growth (EG) programs. 14 PFM activities
        that are included as components of DG programs typically address the
        areas of legislative function and processes, public sector executive
        function, local government decentralization, and anticorruption reforms.
        PFM activities included as components of EG programs typically address
        fiscal and monetary policy issues. USAID has identified DG and EG
        programs with PFM components in over 60 countries since 2007.
        However, according to a USAID official, USAID cannot determine the total
        funding for PFM activities because it does not collect data at a sufficiently
        detailed level to precisely identify PFM activities. Therefore, the official
        reported that USAID is unable to separate PFM assistance from other
        assistance in larger DG and EG development programs. In 2011, total
        funding for DG programs was $1.7 billion, and for EG programs, $4.2
        billion. 15 For illustrative examples of USAID PFM-related projects, see
        appendix IV.

        In addition to PFM capacity-building programs, USAID is seeking to
        increase the use of recipient country PFM systems to deliver assistance.
        As part of its implementation and procurement reforms under its
        institutional reform agenda, known as USAID Forward, USAID has
        announced an agency target of obligating 30 percent of its annual



        14
          USAID reports that its DG programs support more legitimate, inclusive, and effective
        governments that are responsive to the needs of their people. DG’s stated goals include
        strengthening and promoting more transparent and accountable governance. Further,
        USAID reports that its EG programs work to help developing countries achieve rapid,
        sustained, and broad-based economic growth. Reported activities include developing well-
        functioning markets and strengthening the framework of policies, institutions, and public
        goods, including helping poor countries adopt better financial management systems that
        allow for stronger oversight of government spending and reduce chances for corruption.
        15
          Funding totals for 2011 do not include amounts that may have been administered by
        USAID from the Assistance to Europe, Eurasia, and Central Asia account.




        Page 11                                                 GAO-12-920 Public Financial Management
           assistance through local systems, including both partner country PFM
           systems and local not-for-profit and for-profit organizations, by fiscal year
           2015.

Treasury   Treasury’s OTA provides technical assistance through advisors who work
           in-country within the finance ministry, the central bank, or other
           government entities. OTA’s program consists of five core areas, as
           follows:

           •   Budget and Financial Accountability helps foreign governments
               reform and strengthen their PFM systems in order to promote control,
               accountability, and transparency over resources, and to improve a
               country’s overall financial condition.
           •   Banking and Financial Services promotes strong financial sectors in
               which institutions are well regulated, stable, and accessible; serve as
               efficient intermediaries between savers and investors; and are
               resistant to criminal activity.
           •   Government Debt Issuance and Management helps host countries
               implement sound public debt management practices and develop
               markets through which the government can finance itself.
           •   Economic Crimes helps counterpart governments build their capacity
               to prevent, detect, investigate, and prosecute complex financial
               crimes.
           •   Revenue Policy and Administration provides assistance to
               ministries of finance and other relevant organizations that strengthens
               their ability to serve the country and its people through the efficient
               and responsible collection of revenues.

           OTA provides both long-term resident and intermittent advisors. Long-
           term resident advisors provide advice and training to ministers of finance,
           central bank governors, and other government officials. Country
           engagement typically lasts between 2 and 10 years, according to an OTA
           official. Intermittent advisors provide highly specialized technical
           assistance in support of long-term projects, projects requiring several
           different specialties, and projects of short duration. According to an OTA
           official, they currently have about 70 advisors in roughly 50 countries. In
           2011, OTA’s total funding was $44.5 million, with $25.5 million in direct
           OTA appropriations and $19 million in transfers from other agencies,
           including State, USAID, and the Millennium Challenge Corporation. For
           illustrative examples of OTA technical assistance projects, see appendix
           IV.




           Page 12                                   GAO-12-920 Public Financial Management
USAID’s and
Treasury’s Processes
for Developing PFM
Programs Involve
Consultation with
Stakeholders and Rely
on Assessments of
Host-Country Systems

USAID’s Processes       USAID is implementing new processes for developing programs that
                        reflect new agency reform priorities to increase the use of country
                        systems to deliver U.S. assistance. USAID’s work in PFM has traditionally
                        involved capacity building under broader programs designed to improve
                        fiscal management and promote good governance. According to agency
                        officials, USAID has increased its attention to PFM issues. USAID’s new
                        country strategy development and project design processes include
                        various analyses and assessments that may identify opportunities to
                        strengthen and use countries’ PFM systems, as prioritized by USAID’s
                        reform agenda. In these new processes, USAID may identify the need for
                        PFM assistance during countrywide development assessments or
                        through other required assessments. Furthermore, according to USAID
                        guidance, country offices are to develop these assistance programs in
                        collaboration with country stakeholders throughout the program
                        development process. USAID’s new processes are similar in structure
                        and approach to prior processes, but, according to USAID, aim to
                        incorporate more analytical rigor at all stages of the strategic planning
                        framework so that USAID’s efforts are better aligned with the recipient
                        country’s development efforts.

                        USAID may identify the need for PFM assistance during the country
                        strategy development process. According to USAID, most country offices
                        are required to develop a Country Development Cooperation Strategy, a
                        5-year strategy document, by the end of fiscal year 2013. 16 USAID’s latest


                        16
                          Exceptions include country offices implementing a single-sector program and country
                        offices that are phasing out or closing down by fiscal year 2014.




                        Page 13                                         GAO-12-920 Public Financial Management
draft of the guidance for developing the country strategy, released in
September 2011, states that the strategy should demonstrate how it is
integrating the goals of USAID’s assistance reform effort, USAID
Forward, such as working through host-country systems and developing
the capacity of civil society and private sector partners. In developing the
country strategy document, the USAID country office is to consult with
various country stakeholders and conduct several assessments to
understand the development context and develop goals and objectives.
According to the guidance, the country office is required to develop the
strategy document with a focus on maximizing the impact of USAID
resources and build the capacity of specific institutions and related
governance systems at the national, regional, and local levels. For
example, if a USAID country office determines that a nontransparent and
inefficient financial system is a key obstacle to economic growth, the
country office is to work with the host government to improve its capacity
for sound financial management and equitable allocation of resources.
According to USAID officials, one indicator of need for a PFM project
would be an assessment that the country has a major fiscal imbalance
that needs to be corrected by a combination of increased revenue
mobilization or reduction of budget expenditures. Another indicator would
be a determination, arrived at through repeated reporting on corruption in
the media, or internal and external publications, and supported by
stakeholder interviews, that anticorruption programs would improve PFM.

The goals and objectives outlined in the country strategy are to provide
the basis for project design, monitoring, and evaluation. As of June 2012,
USAID stated it had approved 15 country strategies, and 73 USAID
country offices are scheduled to complete a strategy by October 2013.
See figure 3 for highlights of the elements of USAID’s new strategy and
project design processes that could identify the need for PFM capacity
building.




Page 14                                   GAO-12-920 Public Financial Management
Figure 3. Select Components of USAID’s New Strategy Development and Project
Design Processes That May Identify the Need for PFM Capacity Building




Note: USAID’s new processes have several steps to identify the need for PFM capacity-building
programs. We included the components that pertain most directly to PFM.


According to the new USAID guidance on project design, after USAID
approves the strategy document and identifies the need for PFM-related
assistance, USAID country offices are to begin the project design phase.
This phase is to begin with the formation of a design team that is
responsible for the project’s development from planning to
implementation. 17 Overall, the project design phase is to comprise three
stages, as follows:

•    Conceptual stage. During the conceptual stage, the project design
     team is to conduct stakeholder outreach, assessments, analysis, and
     implementation planning. This stage results in a concept paper, which
     provides a summary of a proposed project that country-office
     management can review to assess how the project aligns with the
     country strategy, the likelihood of success, and the assumptions


17
  USAID country offices are required to follow the new project design guidance starting in
July 2012. However, to facilitate transition to the new USAID project design process, full
application of the new project design guidance will only apply to a limited number of new,
priority projects for fiscal years 2012 and 2013 for USAID country offices with an approved
country strategy, or an approved Feed the Future strategy for Feed the Future focus
countries. Priority projects include those that intend to use government systems.




Page 15                                               GAO-12-920 Public Financial Management
    underlying project success. After USAID management approves the
    concept paper, the design team transitions to the analytical stage.

•   Analytical stage. During the analytical stage, the design team seeks
    to understand the problem or constraints identified during the
    conceptual stage, and identify and assess critical assumptions. The
    team is to conduct a series of targeted assessments, including
    required gender, environmental, and sustainability analyses and other
    social, political, and institutional analyses. Of these analyses, we
    identified two that pertain directly to PFM systems, as follows:
    •   Sustainability analysis. The sustainability analysis assesses the
        partner government’s ability to manage the continuation of the
        project after the project has concluded. According to USAID
        guidance, to build sustainability into a project, the design should
        consider how the country office will increase the skills and
        capacity of local stakeholders involved in maintaining
        development gains after the project ends—as well as how USAID
        will ensure that relevant activities or services are gradually tied to
        sustainable financing models through private sector participation
        or through sustainable, publicly managed arrangements and
        government processes.
    •   Institutional analysis. An institutional analysis is an in-depth
        assessment of the local institutions and systems most critical to
        the implementation of the project’s development interventions,
        including an assessment of the quality of their leadership,
        structure, and staff; and identification of their administrative and
        financial management strengths and weaknesses. Using the
        analysis, USAID is to develop a plan for project activities that are
        necessary and sufficient to bring these institutions up to the level
        of performance or engagement as partners appropriate for their
        roles in the project’s implementation and their eligibility for direct
        USAID funding.

    According to USAID guidance, the analytical stage results in a project
    authorization document that will be the basis for project
    implementation, adaptation, and evaluation, which includes a
    summary of the analyses that underlie the rationale for the project
    design.

•   Approval stage. A successful project design results in an approved
    project authorization, which enables a project to move from the
    planning stage to implementation. The project authorization sets out
    the purpose and duration of the project, defines fundamental terms
    and conditions of the assistance when a partner country agreement is


Page 16                                    GAO-12-920 Public Financial Management
    anticipated, and approves an overall total budget level, subject to the
    availability of funds.

In addition to the above analyses, USAID has developed a PFM risk-
assessment framework (PFMRAF) to measure the fiduciary risk, or the
risk of funds being misspent or mismanaged, when USAID plans to
provide aid through the country’s finance systems. USAID guidance
commits country offices to offer, if appropriate, a USAID assessment of
partner country PFM systems with the goal of providing funding for project
implementation through the use of those systems. If the offer is accepted,
the assessment must be carried out using the PFMRAF. Whenever
possible, USAID should begin conducting the PFMRAF during the
conceptual stage of the project design process. The PFMRAF consists of
the following five stages:

1. The Rapid Appraisal provides a high-level snapshot of country-level
   fiduciary risks associated with the use of partner country PFM
   systems and helps inform decisions on whether to undertake a more
   rigorous, formal risk assessment.
2. The Risk Assessment, Analysis, Mitigation, and the Approval for
   Use of Partner Country Systems establishes the baseline level of
   risk corresponding to contemplated funding levels and identifies
   vulnerabilities of the partner country PFM sector in which USAID is
   considering use of the system for project implementation. USAID
   determines whether any systemic risk can be reasonably mitigated
   and, if so, what kind of mitigating measures might be introduced to
   reduce the risk. USAID establishes an accountability framework with a
   set of conditions that would, if complied with, constitute formal
   approval for the use of a partner country PFM system.
3. The Project Design, Approval, Designation of Responsibilities,
   and Selection of the Funding Mechanism incorporates the approval
   of use of country systems into the project design, includes risk
   mitigation measures, such as capacity-building technical assistance,
   concurrent audits, and disbursements in tranches, as appropriate, and
   the appropriate funding mechanism.
4. Negotiating and Preparing the Bilateral Project Agreement with
   the Partner Country Government involves the preparation of the
   bilateral agreement in accordance with the risk mitigation measures
   outlined in the approval for use of partner country systems and in
   collaboration with the partner country government.
5. Implementation, Monitoring, and Evaluation occurs once both
   countries agree to the bilateral agreement and as outlined in the
   project design.



Page 17                                   GAO-12-920 Public Financial Management
                                          According to USAID, the country office is to incorporate recommendations
                                          on actions to mitigate identified risks to providing assistance directly to
                                          the host government into various stages of the project design. According
                                          to one USAID official, the country office’s decision to implement the
                                          recommended steps for mitigating risk may depend on the availability of
                                          funds. According to USAID, if risks cannot be mitigated, USAID is not to
                                          deliver assistance through the partner country’s financial systems.
                                          Mitigation steps may include adding a PFM component to a project, such
                                          as technical assistance to improve an aspect of the financial system. As
                                          of June 2012, USAID had completed the rapid assessment in 20
                                          countries and was in the process of finalizing the report for an additional
                                          12. For 4 of the countries for which USAID had completed the rapid
                                          assessment, USAID had decided to either delay or not to proceed to
                                          stage 2 for various reasons, including funding cuts and the political
                                          situation in the country. For the second stage of the PFMRAF, USAID had
                                          completed the risk assessment for 3 countries and was in the process of
                                          conducting the assessment for 11 others. USAID had completed four
                                          stages of the PFMRAF for 3 countries. Table 2 contains a summary of the
                                          status of countries where USAID has begun the PFMRAF process.

Table 2. Status of Countries in the PFM Risk-Assessment Framework Process

             Stage 1                  Stage 2                           Stage 3                            Stage 4                Stage 5
 Complete        In Progress   Complete       In Progress    Complete        In Progress        Complete        In Progress     Complete
         a                                                                                                                               b
    20                 12         3               11              3                 0                3                    0          0
                                          Source: USAID.
                                          a
                                          Stage 2 on hold or no plan to continue to stage 2 in four of these countries.
                                          b
                                          The three countries that have completed stage 4 of the process are pending obligation of funds,
                                          which will move them to stage 5.




Treasury’s Processes                      Treasury OTA’s processes for developing PFM programs involve
                                          collaboration with country government officials whom Treasury has
                                          determined have a strong commitment to reform and are seeking to
                                          develop strong PFM, as well as OTA’s assessment of the country system.
                                          OTA receives requests for assistance from foreign governments, other
                                          U.S. government agencies, U.S. embassies, international organizations,
                                          and OTA advisors and other donors already working in countries on other
                                          projects. According to OTA officials, OTA receives two to three requests
                                          for assistance per month, but due to the agency’s limited financial
                                          resources and the financial commitment required for each project, it




                                          Page 18                                                 GAO-12-920 Public Financial Management
selectively chooses its projects. OTA’s principles for providing assistance
are as follows:

•   work with partners committed to ownership of their reform;
•   advocate self-reliance by providing countries with knowledge and
    skills required to generate and manage their own resources and
    reduce dependence on international aid; and
•   work with ministry staff daily to build capacity through mentoring and
    on-the-job training on PFM practices.

After receiving the request, OTA begins a preliminary assessment
process to identify weaknesses and areas for assistance, according to
OTA officials. The preliminary assessment is primarily a review of
available information on the country’s PFM system, which may include
information from the U.S. embassy, as well as relevant country reports
and assessments prepared by international financial institutions, other
bilateral assistance providers, and nongovernmental organizations. After
reviewing documents, if it decides to proceed with the assessment, OTA
completes a more in-depth, in-country assessment of PFM, which
consists of meetings with U.S. embassy staff, other donors, and relevant
ministry officials, including the minister of finance or the head of the
central bank, as well as high-level policy staff and mid-level supervisors.
According to OTA officials, the purpose of the in-country assessment is to
review the structural issues related to the country system and determine
whether the country has dedicated staff committed to working toward their
reform efforts. At any point during the assessment process, OTA may
determine that OTA assistance is not suitable for the country’s needs.
OTA officials told us that in one such case, government officials
requested that OTA manage their budget for them, which is against
OTA’s principles because doing so would not promote self-reliance or
help the country develop the capacity to generate and better manage its
own revenues. Finally, OTA officials told us that their decision on whether
to provide assistance depends on whether sufficient funding is available.

The in-country assessment typically results in a draft of official terms of
reference, which identifies weaknesses to be addressed during OTA’s
engagement. According to OTA officials, the terms of reference describes
the broad goals of the project and represents agreement between
Treasury and the host-country counterpart, the ministry staff with whom
the advisor will work on a daily basis. For example, the 2010 terms of
reference for a budget project in Honduras identified the following four
agreed-upon areas for assistance:




Page 19                                  GAO-12-920 Public Financial Management
•   improve operational efficiency and enhance accountability by
    strengthening the organization of the ministry of finance;
•   move toward compliance with International Public Sector Accounting
    Standards;
•   enhance capacity to conduct fiscal analysis and produce more reliable
    macroeconomic estimates and revenue and expenditure projections;
    and
•   conduct workshops on basic finance methods and terminology.

The terms of reference also identifies the advisor’s host-country
counterparts and agreements regarding each party’s responsibilities.
According to OTA officials, after OTA fully vets a project and secures
funding, the parties finalize and sign the terms of reference. However,
OTA and its counterparts may seek to revise the terms of reference if the
central purpose of the technical assistance project changes or during
times of transition, such as when a finance minister changes. In
Honduras, the 2010 terms of reference replaced an existing OTA project
following a gap in OTA assistance during a political crisis in the Honduran
government in 2009.

The OTA advisor assigned to the country uses the broad goals expressed
in the terms of reference to develop a work plan. The work plan contains
specific objectives, milestones, and planned completion dates designed
to work toward goals agreed upon in the terms of reference, and the plan
must be developed within 60 days of the advisor arriving in-country. Each
work plan covers a 1-year period and is the primary basis for regular
monthly progress reports to OTA headquarters. OTA considers work
plans, and the monthly reports on which they are based, to be dynamic
documents that reflect the project’s progress in real time, and advisors
can and should change or modify the work plan during the course of a
project as appropriate and in consultation with the ministry staff and OTA
management. Moreover, according to OTA officials, a 1-week in-country
assessment may not provide all of the information needed and may
require that the advisor rewrite the plan after a few months in the country.
While the work plan is considered a management tool to monitor projects,
OTA officials told us that in recent years it has also become a joint
document between the OTA advisor and ministry staff as a shared tool to
monitor and discuss progress.




Page 20                                  GAO-12-920 Public Financial Management
USAID and Treasury
Use Certain Processes
to Monitor and
Evaluate PFM-Related
Programs, but
Implementation Has
Some Weaknesses

USAID’s Processes
USAID Uses Multiple Tools to     To monitor PFM-related programs, USAID develops performance
Monitor Its PFM Assistance but   management plans, reviews periodic progress reports, and conducts site
Lacks a Process for Tracking     visits, among other activities, but reviews of USAID programs have
Use of Country Systems           identified agency-wide weaknesses in implementation, including using
                                 unreliable baseline data and inaccurate reporting of results. USAID
                                 develops the monitoring and evaluation framework during the project
                                 design, and it generally is to include the following:

                                 •     A performance management plan: This is a tool to plan and
                                       manage the process of monitoring, evaluating, and reporting progress
                                       toward achieving the project’s development objectives. The
                                       performance management plan includes performance indicators and
                                       targets that link to the project objectives. 18

                                 •     A work plan: This specifies activities to be undertaken and the
                                       proposed schedule for these activities during the life of the project. 19


                                 18
                                   According to USAID guidance, a performance indicator is a particular characteristic or
                                 dimension used to measure actual results compared to expected results to determine
                                 progress toward the objective at three levels: output, outcome, and impact. Output is an
                                 immediate and intended product or consequence of an activity within USAID’s control,
                                 such as number of personnel trained or new technologies developed. Outcome is a
                                 higher-level or end result, or purpose, such as the development objective to develop local-
                                 level capacity to create reliable financial institutions. Impact generally refers to higher-level
                                 medium- and long-term impacts, which can be intended or unintended, positive or
                                 negative, such as growth in national income or the size of the budget deficit.
                                 19
                                      Our three USAID case-study countries’ agreements required annual work plans.




                                 Page 21                                              GAO-12-920 Public Financial Management
                                       •   Periodic progress reports: USAID assistance agreements with
                                           nongovernmental organizations generally also require implementing
                                           partners to submit periodic progress reports, the frequency of which
The following illustrates the key
elements that are built into the           vary depending on the assistance agreement, but these reports may
performance management plan                not be required more frequently than quarterly or less frequently than
and the work plan for USAID’s              annually. The progress reports should generally contain a comparison
Growth and Fiscal Stability
Initiative Program in Kosovo.              of actual accomplishments with the goals and objectives established
                                           for the period and reasons why established goals were not met, if
                                           appropriate. Implementing partners should immediately notify USAID
Agreement: The USAID/Kosovo                of developments that have a significant impact on the award-
program established 3 objectives:
                                           supported activities.
Objectives:
1. Build professionalism and           •   Site visits: USAID guidance states that site visits are conducted as
expertise in sound PFM in
municipalities.                            needed by the technical officers, who are responsible for monitoring.
2. Support private sector
participation through public-private
partnerships.                          USAID works with the implementing partner to produce and approve the
3. Develop and implement an action-
oriented government strategy for       work plan and the performance management plan. USAID’s technical
profitable private sector growth.      officer approves the plans. USAID guidance also requires staff to conduct
The performance management
                                       at least one portfolio review each year that covers all activities included in
plan incorporated these objectives,    their various programs. These reviews determine, among other things,
along with indicators and targets.     whether the activities are achieving the desired results.
For example:
                                       Prior reviews of USAID programs have identified challenges in the
Objective 1: Build professionalism
and expertise in sound PFM in
                                       agency’s implementation of its monitoring processes across many types
municipalities.                        of programs, including DG and EG programs. In its fiscal year 2011
Indicator: Number of PEFA              memo on management and performance challenges, USAID’s Inspector
assessments conducted by
municipalities.                        General identified specific monitoring-related weaknesses as one of
Target: Conduct 5 assessments in       USAID’s most serious management and performance challenges. The
year 1 and 10 in year 2.
                                       USAID Inspector General reported problems with assistance planning in
The work plan linked tasks,            25 out of 80 performance audits conducted in fiscal year 2011.
activities, and timelines to each of   Assistance planning is important because it provides the means for
the objectives from the performance
management plan.                       program implementers to track progress toward program objectives and
                                       helps to ensure that USAID assistance programs achieve planned results.
For example:
                                       In addition, 37 of the 80 audit reports the Office of the Inspector General
Objective 1: Build professionalism     issued in fiscal year 2011 identified cases in which USAID operating units
and expertise in sound PFM in
municipalities
                                       or their implementing partners reported misstated, unsupported, or
Task: Implement the PEFA               unvalidated results. In recent reports, we also identified deficiencies in
framework methodology for self-        USAID’s monitoring practices, including the lack of an integrated plan for
assessments by municipalities.
Activity: Conduct inventory of
current PFM practices through PEFA
self-assessment.
Timeline: First 3 months.




                                       Page 22                                    GAO-12-920 Public Financial Management
monitoring and evaluating nonemergency food aid, 20 monitoring practices
that do not correspond to agency performance guidelines, 21 difficulties in
developing meaningful outcome indicators related to building trade
capacity, 22 undocumented site visits for assistance programs in Burma, 23
and lack of performance targets and baseline data for indicators related to
PFM efforts in Afghanistan. 24

Moreover, in our review of the fiscal year 2011 and 2012 Inspector
General’s audits of USAID’s DG and EG programs, we found monitoring
weaknesses cited in 20 of the 32 audit reports. 25 The audit reports cited
monitoring weaknesses, including unreliable or nonexistent baseline data;
performance data weaknesses, such as results that were not reported,
lack of data verification, or inaccurate reporting of data; lack of a current
performance management plan; and inadequate monitoring of program
activities, including lack of regular submission of progress reports.

With regard to our three case study countries, we found that the country
offices generally applied USAID’s monitoring processes in all three PFM-
related programs; however, we did find some conditions that could make
monitoring more difficult, as described below. Our review was not
intended to be comprehensive or applicable to all USAID PFM-related
programs.




20
   GAO, International Food Assistance: USAID Is Taking Actions to Improve Monitoring
and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede
Efforts, GAO-09-980 (Washington, D.C.: September 28, 2009).
21
  GAO, Foreign Assistance: The United States Provides Wide-ranging Trade Capacity
Building Assistance, but Better Reporting and Evaluation Are Needed, GAO-11-727
(Washington, D.C.: July 29, 2011).
22
  GAO, Foreign Assistance: The United States Provides Wide-ranging Trade Capacity
Building Assistance, but Better Reporting and Evaluation Are Needed, GAO-11-727
(Washington, D.C.: July 29, 2011).
23
  GAO, Burma: UN and U.S. Agencies Assisted Cyclone Victims in Difficult Environment,
but Improved U.S. Monitoring Needed, GAO-11-700 (Washington, D.C.: July 26, 2011).
24
 GAO, Afghanistan Governance: Performance-Data Gaps Hinder Overall Assessment of
U.S. Efforts to Build Financial Management Capacity, GAO-11-907 (Washington, D.C.:
Sept. 20, 2011).
25
     USAID’s DG and EG offices are the primary offices providing PFM-related assistance.




Page 23                                           GAO-12-920 Public Financial Management
•    Performance management plans: All three programs were clearly
     defined in terms of overall objectives, project objectives, tasks,
     expected results or outcomes, and required plans and reports. All
     three programs—the Kosovo Growth and Fiscal Stability Initiative, the
     Peru ProDecentralization Project, and Liberia’s Governance and
     Economic Management Support Program—had performance
     management plans containing objectives and indicators. Two of the
     three performance management plans also included targets.
     However, the targets for Liberia’s program had not been determined 9
     months into the program. 26

•    Work plans: All three programs also had work plans that specified
     tasks and timelines. However, Liberia’s Governance and Economic
     Management Support Program initially operated without an approved
     work plan, although it did establish an action plan to guide its
     activities. 27

•    Periodic progress reports: Assistance agreements for all three
     USAID programs required quarterly progress reports, and the
     implementing partners submitted all reports according to agreed-upon
     time frames. In our review of the three programs, we found that the
     language used to describe the objectives and results in the quarterly
     progress reports did not correspond to the language in the work plans,
     which could make it difficult to track progress against the work plans’
     objectives. For example, the quarterly reports for the Kosovo Growth
     and Fiscal Stability Initiative Program describe progress on five
     separate lines of effort under the objective to improve fiscal
     stewardship, while the statement of work and work plan discuss only
     three lines of effort under the same objective.

•    Site visits: USAID officials responsible for monitoring progress on all
     three projects said that they do not rely solely on progress reports to
     track progress, and that additional monitoring tools included weekly
     progress meetings, frequent site visits, and telephone and e-mail
     communications with implementers. For example, the USAID official



26
  The Liberia program’s performance management plan indicates that it will be updated in
the fourth quarter of fiscal year 2012.
27
  The lack of an approved work plan was due to the ongoing negotiations between the
Government of Liberia and the USAID/Liberia country-office staff, according to a program
progress report.




Page 24                                          GAO-12-920 Public Financial Management
                                       in Kosovo reported that he works on a daily basis with the
                                       implementing partners who produce the quarterly and annual reports
                                       and that they exchange e-mails and phone conversations daily.

                                  See appendix IV for descriptions of USAID PFM-related assistance
                                  programs from our case study countries.

                                  In addition to USAID’s reported program monitoring weaknesses, we
                                  found that the agency does not have a process to fully identify and
                                  measure its use of country systems. USAID has set an agency target of
                                  obligating 30 percent of the agency’s annual assistance through local
                                  systems, including both partner country PFM systems and local not-for-
                                  profit and for-profit organizations, by fiscal year 2015. 28 However, USAID
                                  currently cannot track progress toward its 30 percent target because its
                                  accounting system cannot identify the full range of such assistance, which
                                  includes a variety of implementing mechanisms from host-country
                                  contracting to direct cash transfers. The project data in USAID’s
                                  accounting system does not provide sufficiently detailed information, such
                                  as the location of the organization receiving assistance, to identify
                                  qualifying assistance. According to a senior USAID official, the agency is
                                  working on a system whereby it will tag each entity receiving assistance
                                  with an identifier, such as government or not-for-profit organization, as
                                  well as a vendor location signifying U.S.-based or non-U.S. based.
                                  According to the official, this tagging process will facilitate identifying use
                                  of country systems. However, USAID’s financial system currently cannot
                                  distinguish whether non-U.S. based not-for-profit and for-profit
                                  organizations are located in the host country or in a third-party country.

USAID Uses Evaluations to         USAID’s process to assess the effectiveness of its PFM-related programs
Assess the Effectiveness of Its   involves independent evaluations, but weaknesses in the agency’s overall
Programs, but Weaknesses in       evaluation practices have been reported. USAID may use independent
Its Practices Have Been           external evaluations in the middle or at the end of a program based on
Reported                          the need to inform program decisions. USAID adopted a new evaluation
                                  policy in January 2011 and updated its guidance in February 2012 to
                                  reflect it. The new evaluation policy requires that all large projects have at
                                  least one evaluation and that evaluations use methods that generate the
                                  highest-quality and most credible evidence, including experimental



                                  28
                                    According to a senior USAID official, USAID directed 3 percent of its funds through
                                  country systems in 2007.




                                  Page 25                                          GAO-12-920 Public Financial Management
methods. 29 The policy also states that 3 percent of program budgets
should be devoted to external evaluation and classifies two types of
evaluations:

•    performance evaluations, which focus on descriptive and normative
     questions and other questions pertinent to program design,
     management, and operational decision making; and
•    impact evaluations, which define a counterfactual (what would have
     happened had the program not been implemented) to control for
     external factors and measure the change in a development outcome
     that is attributable to a defined intervention.

USAID reported that experimental methods generate the strongest
evidence for impact evaluations; however, experimental methods and
impact evaluations may be difficult to apply to PFM capacity-building
efforts. 30 For example, USAID evaluations of PFM programs have
highlighted difficulties associated with conducting impact evaluations,
including lack of data or resources, and too short a time period to identify
impact. 31 A February 2012 USAID report on the first year of
implementation of the new evaluation policy noted that USAID had not yet
completed an evaluation under the new policy. 32




29
  In experimental methods research, subjects are randomly assigned to either a treatment
or control group, data are collected both before and after the intervention, and results from
the treatment group are benchmarked against results from the control group.
30
  The Inter-American Development Bank, an international financial institution composed of
48 member governments, has reported that techniques for using impact evaluations to
assess institutional capacity building have not been fully developed. Furthermore, if a
program targets a ministry, such as the finance ministry, it may be difficult to establish a
control or comparison group equivalent to the ministry (i.e., a counterfactual). Capacity
building at the subnational (regional or municipal) level may be more amenable to
experimental or quasi-experimental methods if program assistance is provided to some
regions or municipalities but not others.
31
 Democracy International Inc., Kosovo Effective Municipalities Initiative Mid-Term
Evaluation, a special report prepared at the request of the U.S. Agency for International
Development, May 2009; and ARD Inc., Prodecentralization Project External Mid Term
Review, a special report prepared at the request of the U.S. Agency for International
Development, September 2010.
32
  U.S. Agency for International Development,USAID Evaluation Policy: Year One. First
Annual Report and Plan for 2012 and 2013, PD-ACS-999 (Washington, D.C.: February
2012).




Page 26                                            GAO-12-920 Public Financial Management
USAID adopted the new policy to address a number of weaknesses it had
identified in its evaluation practices. USAID reported that the number of
evaluations submitted to USAID’s Development Experience
Clearinghouse—its main repository for agency evaluations—decreased
from nearly 500 in 1994 to approximately 170 in 2009 despite an almost
three-fold increase in program dollars managed. Furthermore, according
to USAID, the majority of evaluations in recent years relied heavily on
anecdotal information and expert opinion, rather than systematically
collected evidence. We have also reported weaknesses in USAID
evaluation practices in several areas in recent years, including not
planning for evaluations or not using evaluation results. 33 Furthermore,
the weaknesses in performance indicators that the USAID IG identified in
DG and EG programs could create difficulties for evaluating PFM
programs. For example, weaknesses in baseline data or performance
indicators would make it difficult to use quantitative measures to evaluate
these programs.

We reviewed external evaluations from our case study countries to
illustrate how evaluations have been applied to PFM programs and noted
a number of program accomplishments as well as examples of
methodological weaknesses. 34 For example, an external team conducted
a final evaluation of the Governance and Economic Management
Assistance Program in Liberia in 2010 and found that the program made
corrupt practices more difficult, improved the accuracy of the budget, and
exerted central control over governmental processes. 35 However, the
evaluation team was hampered by insufficient time for data collection and
analysis and a disorganized monitoring and evaluation framework. In
addition, in Kosovo in 2009, an external team conducted a midterm


33
  GAO, Foreign Assistance: The United States Provides Wide-ranging Trade Capacity
Building Assistance, but Better Reporting and Evaluation Are Needed, GAO-11-727
(Washington, D.C.: July 29, 2011); Afghanistan Development: Enhancements to
Performance Management and Evaluation Efforts Could Improve USAID’s Agricultural
Programs, GAO-10-368 (Washington, D.C.: July 14, 2010); and International Food
Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of
Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts,
GAO-09-980 (Washington, D.C.: Sept. 28, 2009).
34
  These evaluations were not subject to USAID’s new evaluation policy, which was issued
in January 2011.
35
  Sibley International LLC, Final Evaluation of USAID GEMAP Activities (Governance and
Economic Management Assistance Program), a special report prepared at the request of
the U.S. Agency for International Development, June 2010.




Page 27                                         GAO-12-920 Public Financial Management
                              evaluation of the Economic Management for Stability and Growth
                              program and found that the program improved the government’s
                              institutional capacity in fiscal and monetary policy. 36 However, the team
                              did not document its evaluation methodology.


Treasury’s Processes
Treasury’s OTA Uses Various   Treasury’s OTA uses various processes to monitor PFM-related
Processes to Monitor PFM-     programs, including narrative monthly and trip reports, annual quantitative
Related Programs              performance measures, and customer feedback surveys, but weaknesses
                              exist in some of its evaluation processes. The work plan is the key
                              document OTA uses to monitor a project. OTA uses monthly reports to
                              document and monitor progress toward the milestones identified in the
                              annual work plan, including changes to milestones or timelines, and to
                              facilitate communication between the resident advisors and headquarters.
                              OTA advisors may also share monthly reports with host-country
                              counterpart institutions, U.S. embassy staff, other Treasury bureaus and
                              offices, and other interested partners such as USAID, the Millennium
                              Challenge Corporation, and relevant international financial institutions to
                              monitor progress and coordinate donor assistance as relevant. In addition
                              to updates on work-plan progress, the monthly reports may also include
                              other information such as activities completed; significant meetings held;
                              and important political, social, and economic developments. OTA also
                              uses monthly reports to track changes made to the work plan agreed to
                              by the advisor and his or her counterpart. For example, an OTA budget
                              project in Cambodia listed implementation of program budgeting as one
                              its primary objectives and improving budget classification as one its
                              secondary objectives. In the monthly reports, the resident advisor
                              documented the need to refocus the reform efforts on budget
                              classification to better align with the Cambodian government’s ongoing
                              PFM reform efforts. Further, in the case of OTA’s 2011 Honduras budget
                              project to implement international public sector accounting standards, the
                              objectives remained unchanged, but milestones were changed in the
                              monthly reports. According to the advisor, some milestones had to be
                              extended due to overly ambitious initial expectations that were purposely
                              established to encourage the ministry staff to undertake reforms. See


                              36
                                AECOM International Development, Evaluation of the Economic Management for
                              Stability and Growth Program, a special report prepared at the request of the U.S. Agency
                              for International Development, April 2009.




                              Page 28                                          GAO-12-920 Public Financial Management
appendix IV for descriptions of OTA-funded PFM projects in our case
study countries.

OTA headquarters also maintains regular communication with its
advisors. Senior OTA officials monitor project performance through
regular contact with advisors in the field by e-mail, telephone, and site
visits. One advisor told us that their director would already be aware of
any issues raised in the monthly reports, given the frequency of their
communications. Resident advisors also told us that senior Treasury staff
visited Cambodia and Honduras to review OTA projects and talk with
counterparts. Following site visits, OTA officials are required to prepare
trip reports. For example, in a trip report from November 2011, a senior
OTA official reported that he met with host-country counterparts to
discuss the four current OTA engagements in Cambodia, including the
budget project. The visit confirmed the reported difficulties in
implementing the government financial management information system
as a result of a procurement delay, but also affirmed OTA’s role in its
eventual implementation. OTA also uses voluntary customer surveys to
collect feedback on OTA projects from the host-country counterpart. For
example, a Honduran government official who completed the voluntary
customer feedback survey for the budget project indicated that the project
met expectations and made a significant contribution by strengthening the
host-country staff’s technical capacity through the training sessions that
OTA staff offered.

We found that OTA applied its monitoring processes to all three of our
case study country projects, but monthly reports lacked some details on
progress. We reviewed OTA’s monitoring activities in three PFM-related
projects to examine how OTA applied its monitoring process to these
projects. Our review was not intended to be comprehensive or applicable
to all OTA projects. We found the projects to be clearly defined with
specific objectives, milestones, proposed completion dates, and regularly
submitted monthly reports. In addition, in all three cases, the resident
advisor reported frequent communication on project progress with
headquarters. However, in two of the case study countries, Cambodia
and South Africa, work plans had major changes in objectives, but only
Cambodia documented its change in a monthly report. Additionally, our
review found that the level of detail of the monthly report narratives on
progress varied, with a few reports having little detail, which could make
tracking progress for these specific projects difficult without other
communication between the advisor and OTA management. However, as
noted above, OTA headquarters maintained frequent communication with
advisors in the field.


Page 29                                 GAO-12-920 Public Financial Management
Treasury OTA’s Performance     OTA’s quantitative performance measures have been a useful project-
Measures Are Useful Project-   level indicator of performance but have not been a useful indicator of
Level Indicators but Have      OTA-wide performance because of conceptual problems and errors
Weaknesses                     introduced by OTA when aggregating the performance data. OTA
                               advisors and management score technical assistance projects annually
                               based on project-specific indicators under four main categories. 37 The
                               projects are scored on a scale of one (lowest) to five (highest) for
                               “traction,” or the degree of engagement with host-country counterparts,
                               and “impact,” or the results of project activities that bring about change in
                               counterpart law, systems, processes, and procedures. OTA officials told
                               us that the traction and impact measures also reflect the language and
                               values of OTA, and are useful to management, advisors, and host-
                               country counterparts. The primary purpose of the quantitative
                               performance scores was to respond to an Office of Management and
                               Budget reporting requirement for evaluation data. While OTA reported
                               that it met or exceeded targets for traction and impact set in 2008, in
                               every year from 2009 through 2011, OTA officials acknowledged that the
                               aggregate values associated with its annual goals were of limited value
                               due to lack of comparability across programs and over time. In previous
                               work on the Office of Management and Budget, we highlighted the
                               difficulty of representing program performance with a single rating. 38
                               Using a single rating can force agencies to simplify more nuanced and
                               complex performance results, a circumstance similar to OTA’s
                               aggregation of traction and impact scores. A senior official told us that
                               OTA, in complying with the OMB requirement, designed the measures to
                               be as useful as possible at the project level. For example, OTA uses the
                               project-level traction and impact measures in setting expectations and
                               discussing progress with both the host-country counterpart and the
                               advisor. In addition, the official reported that OTA is continually looking
                               for additional ways to use the traction and impact scores. One example
                               of how researchers have used PFM performance data is by developing
                               analytical approaches to identify the determinants of strong PFM systems




                               37
                                 The four main categories for project assessment are (1) international standards,
                               processes, and participation; (2) progress toward country goals; (3) building human and
                               systems capacity; and (4) whole-of-government and private sector effectiveness.
                               38
                                GAO, Performance Budgeting: Observations on the Use of OMB’s Performance
                               Assessment Rating Tool for the Fiscal Year 2004 Budget, GAO-04-174 (Washington,
                               D.C.: Jan. 30, 2004).




                               Page 30                                          GAO-12-920 Public Financial Management
                              and elements of successful PFM reform efforts, which can help to control
                              for important differences across countries. 39

                              Our analysis of OTA performance data found several errors that OTA
                              introduced when aggregating project-level data; these errors collectively
                              raised questions about the reliability of the instrument used to aggregate
                              OTA’s quantitative performance measures and, thus, suggested there
                              may be limitations in its ability to provide insight into performance across
                              OTA projects. OTA uses this instrument to calculate the annual
                              performance averages which are compared against annual targets and
                              provided to OMB and to Congress in annual reports. Our analysis of the
                              spreadsheet containing these data, and limited spot checking of some
                              underlying data for our three case-study countries, identified a number of
                              errors, including those introduced when transcribing data to the
                              spreadsheet. In one instance, OTA listed a performance score under the
                              wrong country, which resulted in inaccurate information for two countries’
                              projects.

OTA Does Not Fully Evaluate   OTA has not yet fully implemented its requirement to conduct
Technical Assistance after    independent evaluations—that is, evaluations conducted by someone
Completion                    other than the resident advisor—of its completed technical assistance.
                              OTA guidance for project reporting and documentation includes
                              requirements for end-of-tour reports when a resident advisor leaves and
                              end-of-project reports when a technical assistance project is completed.
                              The end-of-project report is a postproject evaluation whose purpose is to
                              (1) compare accomplishments with the initial objectives of the project and
                              (2) improve the planning and execution of future projects. A senior OTA
                              official said that the requirement for the end-of-tour report was fully
                              implemented and that OTA intended to fully implement the end-of-project
                              report requirement by the end of 2012. According to the guidance, the
                              evaluation must be conducted independently; the report is the
                              responsibility of the associate director and can be delegated to a senior
                              advisor, but it should not be prepared by an advisor who prepared an
                              end-of-tour report. Both OECD guidance and USAID guidance also



                              39
                                See, for example, Paolo de Renzio, Matt Andrews, and Zac Mills, “Does Donor Support
                              to Public Financial Management Reform in Developing Countries Work? An Analytical
                              Study of Quantitative Cross-Country Evidence.” Overseas Development Institute Working
                              Paper # 329, April 2011; or Matt Andrews, “How Far Have Public Financial Management
                              Reforms Come in Africa?” Harvard Kennedy School Faculty Working Paper Series,
                              RWP10-018, May 2010.




                              Page 31                                        GAO-12-920 Public Financial Management
              highlight the importance of independence in evaluation. In contrast to the
              guidance, OTA provided three end-of-project reports that were conducted
              by OTA staff who had been involved in providing some of the technical
              assistance being evaluated. While no comprehensive and fully
              independent end-of-project evaluation has been conducted, OTA officials
              described relevant insights from a postproject trip to one country that was
              undertaken by a senior OTA official in part to better understand the
              longer-term impact of OTA assistance. During the trip, that official
              identified some factors that significantly limited the impact of this project,
              notably a lack of counterparty commitment at the policy level to
              implement and sustain reforms in key areas identified in the terms of
              reference.


              As more donors, including the United States, seek to provide additional
Conclusions   assistance through countries’ systems, the need for strong PFM systems
              and accountability in recipient countries become even more important to
              help lessen corruption and ensure countries effectively manage
              resources. Given USAID’s stated goal of obligating 30 percent of annual
              assistance through local systems, including partner country PFM
              systems, by 2015, it must ensure that those systems are functioning
              properly and transparently. Efforts to strengthen PFM systems can lower
              the risks that assistance delivered through country systems will be
              misspent, while also increasing the capacity of the recipient country to
              effectively manage its own resources. Given concerns about
              transparency and corruption in many aid-recipient countries, achieving
              the expected benefits of using recipient countries’ PFM systems may be
              difficult without concerted efforts by the donors and countries to
              strengthen these systems. In addition, the risks to using these systems
              are increased without efforts to strengthen them. To help strengthen
              these systems, USAID provides developing countries with PFM capacity-
              building assistance. In light of this assistance, and given the difficulties
              USAID has experienced in the past with implementing monitoring and
              evaluation practices, the importance of developing efforts to strengthen
              PFM systems and effectively monitoring and evaluating these efforts has
              increased significantly. Previous difficulties USAID has experienced with
              regard to monitoring, including poor quality of reported data on projects,
              and the lack of reliable baseline data, could affect USAID’s ability to
              conduct effective evaluations of these projects, as evaluators will not
              have access to reliable data. Also, while USAID’s new evaluation policy
              places greater emphasis on impact evaluations and experimental
              methods, it may be difficult for USAID to evaluate its PFM capacity-
              building efforts using these approaches. Finally, for USAID to achieve its


              Page 32                                    GAO-12-920 Public Financial Management
                      target for use of local systems, by fiscal year 2015, it must be able to
                      identify and measure the full extent of assistance that qualifies.

                      Treasury OTA advisors provide technical assistance targeted at
                      weaknesses in countries’ PFM systems. Although OTA has taken
                      numerous steps to monitor this assistance, errors in its aggregated
                      performance data and the lack of comprehensive postproject evaluations
                      limit OTA’s ability to effectively evaluate its assistance. OTA has adapted
                      a challenging mandate from OMB to create a useful measure of its efforts’
                      performance for management, advisors, and host-country counterparts;
                      however, conceptual problems with and errors in the aggregated
                      measures undermine the measures’ reliability. With greater confidence in
                      the quality of the data, opportunities exist to better identify patterns in
                      performance across OTA programs, such as economic or institutional
                      factors that influence program performance. These patterns could help
                      OTA better understand the strengths and weaknesses of its assistance
                      programs and make appropriate changes. OTA could also use the
                      quantitative performance measurement system it has developed to
                      experiment and document the results of new approaches. Finally,
                      although OTA guidance recognizes the importance of postproject
                      evaluations by including in its guidance the requirements for an end-of-
                      project evaluation, the agency has yet to enforce this requirement.
                      Without a postproject evaluation OTA may not fully understand results of
                      its technical assistance or be able to apply lessons learned to new
                      projects.


                      To monitor progress toward USAID’s target to obligate 30 percent of its
Recommendations for   annual assistance through local systems by 2015, we recommend that
Executive Action      the Administrator of USAID direct the appropriate offices to develop a
                      process to reliably identify and track the agency’s use of local systems in
                      all countries receiving assistance.

                      To help ensure that USAID conducts effective evaluations of PFM-related
                      programs under its new policy, we recommend that the Administrator of
                      USAID direct the appropriate offices to ensure that they are establishing
                      adequate monitoring practices for PFM-related programs. Such practices
                      may include selecting proper indicators, collecting reliable baseline data,
                      and ensuring the reliability of reported results.

                      To improve the effectiveness of OTA’s technical assistance, we
                      recommend that the Secretary of the Treasury direct OTA to take the
                      following two actions to improve monitoring and evaluation:


                      Page 33                                   GAO-12-920 Public Financial Management
                     •   implement additional controls to improve the process for computing
                         OTA-wide annual performance measures, and
                     •   fully implement OTA’s existing requirement for end-of-project
                         evaluations and, consistent with its existing guidance, have an
                         independent party conduct the evaluations.

                     We provided a draft of this report to USAID, Treasury, and State for their
Agency Comments      review and comment. Both USAID and Treasury concurred with our
and Our Evaluation   recommendations in their written comments, which are reproduced in
                     appendix V and appendix VI respectively. These agencies along with
                     State provided technical comments and updated information, which we
                     have incorporated throughout this report, as appropriate.

                     In concurring with our two recommendations, USAID reported that it is in
                     the process of refining definitions that will identify and help measure the
                     assistance that qualifies to meet the agency’s target of obligating 30
                     percent of its annual assistance through local systems by 2015. USAID
                     also reported that it has implemented two complementary reforms that will
                     help ensure effective evaluations and adequate monitoring of its PFM
                     assistance. The first reform involves USAID planning for the monitoring
                     and evaluation of assistance during the early stages of project design,
                     including defining indicators and collecting baseline data. The second
                     reform requires USAID to plan final monitoring and evaluation schedules
                     during project design.

                     In concurring with our two recommendations, Treasury reported that OTA
                     has corrected several errors in the 2011 annual performance measures,
                     and has taken steps to strengthen data controls, including conducting
                     additional reviews and increasing staff resources dedicated to computing
                     the performance measures. In addition, OTA has begun to implement its
                     requirement for independent end-of-project evaluations of its technical
                     assistance and intends to fully implement the requirement by the end of
                     2012.


                     We are sending copies of this report to appropriate congressional
                     committees. We are also sending copies of this report to the
                     Administrator of USAID and the Secretaries of the Treasury and State. In
                     addition, this report is available at no charge on the GAO Web site at
                     http://www.gao.gov.




                     Page 34                                  GAO-12-920 Public Financial Management
If you or your staff have any questions about this report, please contact
me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made key contributions to this reported
are listed in appendix VII.

Sincerely yours,




Thomas Melito
Director, International Affairs and Trade




Page 35                                     GAO-12-920 Public Financial Management
Appendix I: Scope and Methodology
             Appendix I: Scope and Methodology




             To examine the process the U.S. Agency for International Development
             (USAID) uses to develop programs to strengthen public financial
             management (PFM) systems, we reviewed USAID’s official policy and
             procedures, as contained in the Automated Directives System, as well as
             USAID’s new guidance documents for developing country strategies and
             designing projects, including the Country Development Cooperation
             Strategy and Project Design Guidance. We also interviewed USAID
             officials, including officials from the democracy and governance (DG) and
             economic growth (EG) offices. Most USAID country offices are required to
             have an approved country strategy by the end of fiscal year 2013 and to
             follow the new project design guidance starting July 2012. Because
             USAID was still in the process of transitioning to these new processes
             during the course of our audit, we did not review USAID’s implementation
             of these processes.

             To examine the process the U.S. Department of Treasury’s Office of
             Technical Assistance (OTA) uses to develop programs to strengthen PFM
             systems, we interviewed senior OTA officials regarding OTA’s policies
             and procedures. We interviewed resident advisors regarding how OTA
             assessed and developed projects in our three OTA case study countries.
             We also reviewed OTA’s official policy guidance document and project-
             specific documents, including assessment reports, signed terms of
             reference, work plans, and monthly reports.

             To examine and assess the processes USAID uses to monitor and
             evaluate its PFM-related programs, we reviewed USAID program
             development guidance; monitoring and evaluation policies and
             procedures; USAID reports; and program documents, including
             assistance agreements, monitoring and evaluation plans, and progress
             reports; USAID external project evaluations; past GAO reports; and
             USAID Inspector General reports. We used the review of the documents
             for our three USAID country case studies to illustrate the implementation
             of USAID’s monitoring processes for its PFM-related programs. We also
             interviewed agency officials in Washington, D.C., and conducted
             telephone interviews and had e-mail communications with key country-
             office staff for each of our case study countries.

             To examine and assess OTA’s processes for monitoring and evaluating
             PFM-related programs, we reviewed OTA project reporting and
             documentation instructions, project work plans, and monthly reports, end-
             of-tour and end-of-project reports, annual quantitative performance data,
             Organization for Economic Cooperation and Development guidance on
             evaluation, and conducted interviews with OTA headquarters staff. We


             Page 36                                 GAO-12-920 Public Financial Management
Appendix I: Scope and Methodology




used the review of the documents for our three OTA case studies to
illustrate the implementation of OTA’s monitoring processes. We
supplemented the document review with interviews with current or former
advisors for each of the case study countries. In addition, we assessed
the reliability of the instrument OTA uses to aggregate quantitative
performance measures across projects. We examined spreadsheets
provided by OTA for consistency, examined data for outliers and missing
values, and spot-checked the transcription of data to the spreadsheet for
our case study countries. Due to a number of errors in the OTA data, we
could not determine if the aggregated performance data were sufficiently
reliable for identifying patterns in performance across projects or over
time.

In selecting our case study countries, we focused on countries in which
OTA or USAID had relevant ongoing or recently completed projects
designed to strengthen PFM systems. We selected six countries:
Cambodia, Honduras, and South Africa for OTA, and Kosovo, Liberia,
and Peru for USAID. In selecting these countries, we considered the
following two factors:

•   Geographic diversity: For each agency, we selected countries from
    three different geographical regions.

•   Country income group diversity: For each agency, we chose a country
    that the World Bank has listed as (1) lower income, (2) lower-middle
    income, and (3) upper-middle income in order to report examples from
    different income levels, which may also be associated with different
    institutional characteristics.

In cases where more than one country would be acceptable under our
decision criteria, we considered additional criteria, such as the availability
of other broad-based indicators. For OTA, we focused our selection on
countries receiving technical assistance from OTA’s Budget and Financial
Accountability team, given its focus on traditional PFM aspects. For
USAID, we selected our countries from a list of countries with significant
PFM-related programs that USAID provided. USAID excluded some
countries with PFM-related programs from the list because staff were not
available to discuss their programs with us. Our review of USAID and
OTA case study countries was not intended to be comprehensive or
applicable to all their respective programs and projects or generalizable to
all countries.




Page 37                                   GAO-12-920 Public Financial Management
Appendix I: Scope and Methodology




To describe recent trends in country PFM systems, we reviewed the data
and publications of five international organizations that conduct broad
assessments of country PFM systems. These broad assessment tools
include the Public Expenditure and Financial Accountability Program, the
World Bank’s Country Policy and Institutional Assessment’s quality of
budgetary and financial management indicator, International Budget
Partnership’s Open Budget Survey, Transparency International’s
Corruption Perceptions Index, and the International Monetary Fund’s
fiscal transparency Reports on the Observance of Standards and Codes.
We converted our six case study country scores for the three PFM
diagnostic tools for which scores were available (Open Budget Survey,
Corruption Perceptions Index, and quality of budget indicator) into
percentile rankings to illustrate each country’s performance as measured
by the three PFM diagnostic tools. We also interviewed officials at the
World Bank and International Monetary Fund to discuss their PFM-related
diagnostic tools.

To describe the Department of State’s PFM-related efforts, we conducted
interviews with agency officials in Washington, D.C. We reviewed State
documents, including agency guidance, waiver packages, and program
documents. We also reviewed relevant appropriations laws to identify the
requirements for State’s fiscal transparency reviews.

We conducted this performance audit from October 2011 through
September 2012 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.




Page 38                                GAO-12-920 Public Financial Management
Appendix II: Several International
                           Appendix II: Several International
                           Organizations Reported Results of Their Public
                           Financial Management Diagnostic Tools


Organizations Reported Results of Their Public
Financial Management Diagnostic Tools
                           Five international organizations — the Public Expenditure and Financial
                           Accountability Program, the World Bank, International Budget
                           Partnership, Transparency International, and the International Monetary
                           fund — have developed tools to assess various aspects of countries’
                           public financial management (PFM) systems, and some have published
                           recent findings or results. We illustrate the percentile ranking for each of
                           our case study countries in three broad PFM diagnostic tools – the Open
                           Budget Survey, the Corruption Perceptions Index, and the World Bank’s
                           quality of budgetary and financial management indicator.


Public Expenditure and     In December 2001, a multiagency partnership founded the Public
Financial Accountability   Expenditure and Financial Accountability Program (PEFA) to strengthen
Program’s PFM              the ability of aid recipients and donors to assess and improve country
                           public expenditure, procurement, and financial accountability systems. 1
Performance Measurement
                           In 2005, the program developed the PFM Performance Measurement
Framework                  Framework, known as the PEFA Framework, to provide a measure of the
                           strengths and weaknesses of a country’s PFM system. The PEFA
                           Framework incorporates a PFM performance report, which includes an
                           assessment of the evaluated country’s PFM performance along six core
                           dimensions of PFM. The six dimensions are the following:

                           •   Credibility of the budget: The budget is realistic and is implemented
                               as intended.
                           •   Comprehensiveness and transparency: The budget and the fiscal
                               risk oversight are comprehensive, and fiscal and budget information is
                               accessible to the public.
                           •   Policy-based budgeting: The budget is prepared with due regard to
                               government policy.
                           •   Predictability and control in budget execution: The budget is
                               implemented in an orderly and predictable manner, and arrangements
                               for the exercise of control and stewardship in the use of public funds
                               exist.
                           •   Accounting, recording, and reporting: Adequate records and
                               information are produced, maintained, and disseminated to meet
                               decision-making control, management, and reporting purposes.



                           1
                            The partnership included the World Bank, the European Commission, the United
                           Kingdom’s Department for International Development, the Swiss State Secretariat for
                           Economic Affairs, the French Ministry of Foreign Affairs, the Royal Norwegian Ministry of
                           Foreign Affairs, and the International Monetary Fund.




                           Page 39                                          GAO-12-920 Public Financial Management
Appendix II: Several International
Organizations Reported Results of Their Public
Financial Management Diagnostic Tools




•   External scrutiny and audit: Arrangements for scrutiny of public
    finances and follow up by executive are operating.

The six core dimensions include 28 high-level indicators, each of which is
assigned a letter score from A to D. The initial assessment helps
establish performance baselines, while repeat assessments help in
monitoring performance progress over time.

In 2011, the PEFA secretariat released the 2010 PEFA Monitoring
Report. Based on a comparison of 33 repeat PEFA assessments over the
2005-2010 period, the analysis shows that more countries had a higher
number of highest or improved scores (23 countries) than lowest or
worsened scores (8 countries), indicating a broad trend of PFM
improvement across the countries surveyed. 2 According to the analysis,
while PFM systems are improving overall, systems features vary
significantly. Formal PFM features where progress can be achieved
through adopting a new law, regulation, or technical tool, or focusing on
no more than a few agencies, or at an early stage in the budget cycle are
more likely to improve or maintain a high score than functional PFM
features where progress requires actually implementing a new law or
regulation, or coordinating the work of many agencies, or working later in
the budget cycle.

Starting July 2012, the PEFA partners are extending the PEFA program
by 5 years and conducting a comprehensive review of the PEFA
Framework, the first since it was launched in 2005. One of the objectives
is to improve confidence in PEFA assessments through an endorsement
process that provides an incentive to ensure adherence to PEFA good
practices in undertaking an assessment.




2
  Highest or improved scores are combined because it is not possible to improve on an A
score; lowest or worsened scores are combined because it is not possible to be worse
than a D score. Five of the eight countries were small island states with challenges not
typical of the larger sample, including high vulnerability to external shocks such as
hurricanes and volatile tourism earnings, severe capacity constraints with high emigration
of skilled human resources, and political instability. The other three were small African
states with many of the same challenges.




Page 40                                           GAO-12-920 Public Financial Management
                          Appendix II: Several International
                          Organizations Reported Results of Their Public
                          Financial Management Diagnostic Tools




World Bank’s Quality of   The World Bank undertakes annual Country Policy and Institutional
Budgetary and Financial   Assessments to assess the quality of a country’s present policy and
Management Indicator      institutional framework. One of the 16 indicators the World Bank uses to
                          assess the performance of a country’s current policies and institutions is
                          the quality of budgetary and financial management. This criterion
                          assesses the extent to which the country has a comprehensive and
                          credible budget linked to policy priorities; effective financial management
                          systems to ensure that the budget is implemented as intended in a
                          controlled and predictable way; and timely and accurate accounting and
                          fiscal reporting. Over the 5-year period from 2005 through 2010, 26 out of
                          73 countries, or slightly more than one-third, showed improvements in the
                          quality of their PFM systems, while 19 countries’ scores worsened. 3 Most
                          countries, 62 percent in 2010, are clustered in the mid-range. 4

                          In addition to the quality of budgetary and financial management
                          indicator, the World Bank uses other broad PFM diagnostic tools,
                          including the Country Financial Accountability Assessments and the
                          Public Expenditure Reviews. The objective of the Country Financial
                          Accountability Assessments is to support the World Bank’s development
                          objectives by identifying strengths and weaknesses in country PFM
                          systems. The assessments are intended to help identify priorities and
                          inform the design and implementation of capacity-building programs. The
                          assessments also describe and analyze financial management and
                          expenditure controls, including expenditure monitoring, accounting and
                          financial reporting, internal controls, internal and external auditing, and
                          legislative review. Information obtained from the assessments, taken
                          together with that obtained from other World Bank diagnostic products
                          and other sources, supports the preparation of an integrated fiduciary
                          assessment. The results of these assessments inform the preparation of
                          the World Bank’s Country Assistance Strategy, particularly the sections
                          dealing with the size of the support program, the sectors to be supported,
                          selection of lending instruments, and approaches to risk management.
                          The assessments are particularly important where World Bank resources


                          3
                            The World Bank does not publish the indicator scores of countries that are not eligible
                          for funding from the World Bank’s International Development Association, which provides
                          interest-free loans and grants to governments of the poorest countries. An average of 76
                          countries received a rating for this indicator from 2005 through 2010.
                          4
                            The World Bank rates each country on a scale of 1 (low) to 6 (high) in half-point
                          increments (0.5). A score of 1 corresponds to a very weak performance; a score of 6, to a
                          very strong performance. Mid-range includes scores of 3 and 3.5.




                          Page 41                                          GAO-12-920 Public Financial Management
                            Appendix II: Several International
                            Organizations Reported Results of Their Public
                            Financial Management Diagnostic Tools




                            are managed by the country’s own PFM system, as in the case of budget
                            support.

                            The Public Expenditure Review’s objectives are to strengthen budget
                            analysis and processes to achieve a better focus on growth and poverty
                            reduction, and to assess public expenditure policies and programs to
                            provide governments with an external review of their policies. Public
                            Expenditure Reviews may also address the incentives and institutions
                            needed to improve the efficacy of public spending in major sectors such
                            as health and education.


International Budget        The International Budget Partnership’s Open Budget Survey assesses
Partnership’s Open Budget   the availability in each country of eight key budget documents, as well as
Survey                      the comprehensiveness of the data contained in these documents. 5 The
                            survey also examines the extent of effective oversight provided by
                            legislatures and supreme audit institutions, as well as the opportunities
                            available to the public to participate in national budget decision-making
                            processes.

                            In its 2010 survey of budget transparency in 94 countries, Open Budget
                            concluded that the state of budget transparency is poor. Only a minority
                            of countries, 20 out of the 94, or 21 percent, can be considered to have
                            open budgets, while a larger number of countries, 41, or 44 percent,
                            provide minimal to scant budget information. The following figure includes
                            the 2010 Open Budget scores for 5 of our 6 case study countries. 6 (See
                            fig. 4.)




                            5
                              The Open Budget Index is an external view of transparency, not of a country’s systems
                            and processes.
                            6
                                The 2010 Open Budget Survey did not include Kosovo.




                            Page 42                                          GAO-12-920 Public Financial Management
Appendix II: Several International
Organizations Reported Results of Their Public
Financial Management Diagnostic Tools




Figure 4: Open Budget 2010 Index Scores




Note. The Open Budget Index assigns countries covered by the survey a transparency score on a
100-point scale. Scores assigned to questions in the Open Budget Survey are used to determine an
overall transparency score for each country surveyed. These scores are then compiled to create the
Open Budget Index, providing a ranking of each country’s relative level of transparency.


Nonetheless, the survey finds the trend toward open budgets is favorable
based on substantial improvements in budget transparency, especially
among countries that provided little information in the past. Some of these
governments, especially those that scored very low in earlier rounds of
the Open Budget Index, largely achieved these improvements by taking
one basic and inexpensive step: They began to make available on their
web-sites the budget documents that they previously produced but had
made available only to internal government audiences or to donors.
According to the Open Budget Survey, countries performing poorly on the
Open Budget Index tend to share certain characteristics, such as low
levels of income, low levels of democracy, and geographical location in
Africa and the Middle East.




Page 43                                               GAO-12-920 Public Financial Management
                             Appendix II: Several International
                             Organizations Reported Results of Their Public
                             Financial Management Diagnostic Tools




Transparency                 Transparency International’s Corruption Perceptions Index ranks
International’s Corruption   countries according to the perception of corruption in the public sector. 7
Perceptions Index            The Corruption Perceptions Index is an aggregate indicator that combines
                             different sources of information about corruption, making it possible to
                             compare countries. Broadly speaking, the surveys and assessments used
                             to compile the index include questions relating to bribery of public
                             officials, kickbacks in public procurement, embezzlement of public funds,
                             and questions that probe the strength and effectiveness of public sector
                             anticorruption efforts.

                             Transparency International’s Corruption Perceptions Index shows that
                             nearly three-quarters of the 178 countries in the index score below 5, on a
                             scale from 10 (very clean) to 0 (highly corrupt). These results, shown in
                             figure 5, indicate a serious corruption problem, according to Transparency
                             International. Figure 5 includes the scores of our six case study countries.




                             7
                              Transparency International defines corruption as the abuse of entrusted power for private
                             gain; its definition encompasses corrupt practices in both the public and private sectors.




                             Page 44                                          GAO-12-920 Public Financial Management
Appendix II: Several International
Organizations Reported Results of Their Public
Financial Management Diagnostic Tools




Figure 5: Transparency International 2010 Corruption Perceptions Scores




According to Transparency International, the Corruption Perceptions
Index is not suitable for trend analysis or for monitoring changes in the
perceived levels of corruption over time for all countries as year-to-year
changes in a country’s or territory’s score can result from a change in the
perceptions of performance, a change in the ranking provided by original
sources, or changes in the methodology resulting from Transparency
International’s efforts to improve the index. However, using specific
criteria, Transparency International determined that nine countries




Page 45                                          GAO-12-920 Public Financial Management
                          Appendix II: Several International
                          Organizations Reported Results of Their Public
                          Financial Management Diagnostic Tools




                          showed an improvement, and seven countries showed deterioration from
                          2009 to 2010. 8

International Monetary    The International Monetary Fund (IMF) developed a Code of Good
Fund’s Fiscal             Practices on Fiscal Transparency (the Code) in 1998 to help ensure that
Transparency Reports on   governments are providing a clear picture of the structure and finances of
                          government. The Code, updated in 2007, focuses particularly on
the Observance of         transparency and accountability, and is based on four principles: clarity of
Standards and Codes       roles and responsibilities, open budget processes, public availability of
                          information, and assurances of integrity. IMF’s fiscal transparency
                          Reports on the Observance of Standards and Codes is a voluntary fiscal
                          transparency assessment program intended to assess member countries’
                          observance of the good fiscal transparency practices in the Code and
                          recent achievements in fiscal transparency; identify and prioritize reforms
                          necessary to enhance the openness of accountability of public
                          institutions; and in resource-rich countries, assess and advise on revenue
                          transparency issues.


Case Study Country        Figure 6 illustrates the percentile ranking for each of our six case study
Performance Based on      countries in three broad PFM diagnostic tools: the Open Budget Survey,
PFM Diagnostic Tools      the Corruption Perceptions Index, and the World Bank’s quality of
                          budgetary and financial management indicator. Percentiles measure
                          position from the bottom, so the percentile rank indicates the percent of
                          cases falling at or below that score. A higher percentile represents a
                          better relative standing. For example, Kosovo’s ranking on the Corruption
                          Perceptions Index is at the 36th percentile, meaning that Kosovo’s
                          standing is as high or higher than 36 percent of the other 178 countries in
                          the index.




                          8
                           Countries showing improvement include Bhutan, Chile, Ecuador, former Yugoslav
                          Republic of Macedonia, Gambia, Haiti, Jamaica, Kuwait, and Qatar. Countries showing
                          deterioration include the Czech Republic, Greece, Hungary, Italy, Madagascar, Niger, and
                          the United States.




                          Page 46                                          GAO-12-920 Public Financial Management
Appendix II: Several International
Organizations Reported Results of Their Public
Financial Management Diagnostic Tools




Figure 6. Case Study Country Percentile Rankings on PFM Diagnostic Tools




Note: Kosovo was not in the 2010 Open Budget Survey. World Bank budgetary and financial
management indicator scores are not available for Peru and South Africa because the World Bank
does not disclose Country Policy and Institutional Assessment scores for countries that are not
eligible for funding from the World Bank’s International Development Association, which provides
interest-free loans and grants to governments of the poorest countries.




Page 47                                               GAO-12-920 Public Financial Management
Appendix III: State Department Conducts Fiscal
                           Appendix III: State Department Conducts
                           Fiscal Transparency Reviews and Administers
                           the Domestic Finance for Development
                           Initiative
Transparency Reviews and Administers the
Domestic Finance for Development Initiative
State Department           The State Department (State) does not directly fund public financial
Conducts Two Public        management (PFM) programs, but its Office of Monetary Affairs (OMA) is
Financial Management-      responsible for two PFM-related activities. First, since 2008, U.S.
                           appropriations laws have required State to evaluate the fiscal
Related Activities         transparency of foreign governments receiving U.S. assistance. 1 When
                           State determines it is important to the United States’ national interest, it
                           may approve waivers for countries that are deemed to be nontransparent
                           that allow U.S. agencies to provide assistance to these countries in
                           accordance with the legal prohibition against providing such assistance
                           without waivers. State processed 28 such waivers in 2011.

                           Second, State supports the presidential initiative called Domestic Finance
                           for Development (DF4D), which President Obama announced in 2011.
                           Under this initiative, State is to help countries use their own resources
                           and leverage other donor resources to meet development goals. State is
                           piloting this initiative in five countries: El Salvador, Honduras, Kyrgyzstan,
                           Tunisia, and Zambia. The goal of DF4D is to strengthen the commitment
                           to reform within partner countries; provide technical assistance in partner
                           countries, such as taxation expertise, including through innovative public-
                           private partnerships; and elevate the importance and interrelation of
                           domestic resource mobilization, fiscal transparency, and anticorruption
                           efforts in public finance as key components for sustainable economic
                           development. Because State has not funded programs in the past for this
                           initiative, State has reached out to other organizations operating in these
                           countries, including international and bilateral organizations, to leverage
                           their programs and resources.


State Processes for PFM    State has developed processes for carrying out its PFM-related activities
Activities Derive from     required by legislative mandate and presidential initiative. Under the
Legal Requirements and     Fiscal Transparency Review Process (FTRP), OMA reviews central
                           governments expected to receive bilateral economic assistance and
Presidential Initiatives   international security assistance for several dimensions of transparency:
                           publicly disclosed budget, budget breakdown by ministry, standards for



                           1
                            Since 2008, the annual Department of State, Foreign Operations, and Related Programs
                           Appropriations Acts have prohibited certain U.S. foreign assistance to the central
                           government of any country that fails to make its national budget publicly available on an
                           annual basis. The law permits the Secretary of State to waive this restriction on individual
                           countries by reporting to Congress that a waiver is important to the national interest of the
                           United States. See Pub. L. No. 110-161.




                           Page 48                                            GAO-12-920 Public Financial Management
Appendix III: State Department Conducts
Fiscal Transparency Reviews and Administers
the Domestic Finance for Development
Initiative




awarding natural resource contracts, and timely and accurate
documents. 2 Each year, OMA reviews all countries it deemed
nontransparent in the prior year, and countries where recent events may
have affected fiscal transparency to evaluate whether they meet the
threshold of fiscal transparency. State uses the IMF’s definition of fiscal
transparency as a guideline for the FTRP. 3 During this process, OMA
officials said they obtain information about the level of transparency of
each country by collecting information from U.S. embassy staff working in
those countries. In addition, OMA staff review publicly available reports
published by international organizations and civil society representatives,
such as the World Bank, International Monetary Fund, International
Budget Partnership, and the World Economic Forum. Because these
organizations do not always prepare country reports on an annual basis,
officials told us they use these reports as a check on their internally
generated information, rather than relying on them as primary sources of
information. For countries that State finds to be nontransparent, State can
issue a waiver that allows the country to receive otherwise restricted
assistance. As part of the process of requesting a waiver, embassy staff
in country develop action plans to assist the country in improving the level
of transparency in its budget process. The action plan should address
specific fiscal transparency issues identified in the transparency review
process and should include recommendations of short- and long-term
steps that the countries can take to improve budget transparency.
Embassies work with host governments to encourage implementation of
action plans, which may include activities such as continued daily
engagement with country officials working on budget reforms, providing
training and coordinating training on issues related to the budget process,
and funding local nongovernmental organizations to perform budget
oversight functions.




2
 Countries that are not transparent with their budgets are ineligible to receive certain U.S.
assistance, without a State issued waiver, in accordance with Pub. L. No. 112-74, §
7031(b).
3
  The IMF defines fiscal transparency as “openness toward the public at large about
government structure and functions, fiscal policy intentions, public sector accounts, and
projections. It involves ready access to reliable, comprehensive, timely, understandable,
and internationally comparable information on government activities so that the electorate
and financial markets can accurately assess the government’s financial position and the
true costs and benefits of government activities, including their present and future
economic and social implications.”




Page 49                                            GAO-12-920 Public Financial Management
                           Appendix III: State Department Conducts
                           Fiscal Transparency Reviews and Administers
                           the Domestic Finance for Development
                           Initiative




                           For OMA’s DF4D initiative, State officials, working with other agencies
                           and organizations, helped develop programs based on needs expressed
                           by country government officials that built on existing reform efforts and
                           development priorities. As an interagency effort, OMA helps identify and
                           leverage existing programs and resources, such as those of multilateral
                           organizations and other donors, such as USAID’s Innovation Fund and
                           the Global Health Initiative. For example, USAID is implementing a $7.6
                           million program to advance El Salvador’s fiscal reform agenda by building
                           capacity and improving systems for public expenditure and management
                           and tax revenue mobilization, promoting private sector engagement, and
                           creating a $2 million Revenue Challenge Fund to support improved tax
                           collection at the municipal level. With five countries piloting DF4D, State
                           plans to proceed by selecting additional partner countries based on
                           performance against quantitative DF4D-related measures, consultations
                           with posts, and expressions of interest from ministers of finance and other
                           economic leaders. Moreover, State plans to use action plans developed
                           for FTRP for countries to be considered for DF4D. State encourages
                           posts to report on PFM issues and opportunities to mobilize domestic
                           resources, raise the issues with relevant stakeholders in the public and
                           private sectors, and work with OMA staff to further the objectives of the
                           DF4D initiative.


State Is Taking Steps to   State is beginning to conduct additional monitoring as part of its fiscal
Monitor Progress of        transparency evaluations. Starting in 2012, State is requiring benchmarks
Nontransparent Aid         in its action plans for nontransparent countries so that it can compare
                           progress annually. According to State officials, some country action plans
Recipients                 had benchmarks but the quality of the benchmarks varied. The action
                           plan attempts to capture all the steps necessary to improve a country’s
                           fiscal transparency and includes more than just State actions. Lastly,
                           appropriations law for 2012 requires State to evaluate whether or not the
                           country has made progress toward improved fiscal transparency included
                           in any waiver request submitted. 4 State guidance reports that progress
                           toward implementation of embassy action plans will factor into its decision
                           of whether to renew waivers.




                           4
                               Consolidated Appropriations Act, 2012, Pub. L. No. 112-74, § 7031(b).




                           Page 50                                            GAO-12-920 Public Financial Management
Appendix IV: Examples of Public Financial
                      Appendix IV: Examples of Public Financial
                      Management-related Projects in Six Countries



Management-related Projects in Six
Countries
                      To provide illustrative examples of U.S. projects to strengthen public
                      financial management (PFM) systems, we chose six case study
                      countries. For the U.S. Agency for International Development (USAID),
                      we selected projects in Kosovo, Liberia, and Peru and for the U.S.
                      Department of the Treasury (Treasury), we selected projects in
                      Cambodia, Honduras, and South Africa. We used the following criteria to
                      select the three countries for each agency:

                      •   The agency has relevant ongoing or recently completed projects
                          focused on strengthening PFM systems.

                      •   The countries represent different geographic regions.

                      •   The countries have different income levels, which may be associated
                          with different levels of government capabilities. For each U.S. agency,
                          we selected a country that the World Bank has classified as lower
                          income (Cambodia and Liberia), lower-middle income (Honduras and
                          Kosovo), and upper-middle income (Peru and South Africa).

                      For one PFM-related project in each country we summarized the project’s
                      background, and, for selected objectives, we summarized some of
                      USAID’s expected results and some of Treasury’s activities. These
                      examples are meant to be illustrative and not generalizable.


Selected USAID PFM–   Tables 3 through 5 summarize selected USAID PFM-related projects in
Related Projects      Kosovo, Liberia, and Peru.

                      Kosovo Growth and Fiscal Stability Initiative

                      Time period: July 2010-July 2013
                      Award amount: $1,051,208

                      Description: Since 1999, USAID’s Kosovo economic policy and
                      institutional strengthening programs have focused on establishing key
                      central economic institutions and an enabling environment for private
                      sector growth. USAID is to adjust the focus of technical assistance as the
                      Growth and Fiscal Stability Initiative builds upon the experience and
                      lessons learned from the creation of reliable financial institutions in the
                      central government to address the fiscal stewardship challenges faced by
                      subnational governments. The initiative is to work with municipalities in
                      areas that are directly linked with the Ministry of Finance and Economy,
                      such as budget, treasury, property tax, and public private partnerships.



                      Page 51                                        GAO-12-920 Public Financial Management
                                            Appendix IV: Examples of Public Financial
                                            Management-related Projects in Six Countries




Table 3: USAID’s Kosovo Growth and Fiscal Stability Initiative: Selected Objectives and Expected Results

Objective: Build professionalism and expertise in sound PFM to enable municipal institutions to autonomously and
efficiently stimulate local economic development while providing a high standard of public services
Expected results
•   Identify PFM strengths and weaknesses and develop a PFM reform action plan
•   Share best practices and lessons learned with other municipalities through outreach and training
•   New municipalities use the main Kosovo financial management systems daily
Objective: Strengthen the Office of the Auditor General’s ability to effectively review and assess municipalities’ fiscal
operations in a more timely manner
Expected results
•   Office has adequate budget resources to enable annual audits of all municipalities
•   International audit firms perform some municipal audits and train Auditor General staff
•   Municipalities address audit findings and improve fiscal operations
Objective: Assist the Ministry of Finance and Economy to (1) develop and implement a plan to enable data sharing among
information systems; (2) assess priority capacity-building needs in budget and treasury functions; and (3) ensure that the
property tax department and supporting systems can meet municipalities’ need to generate revenue
Expected results
•   Install a single, shared physical network with all ministry organizations linked to it
•   Ministry budget and treasury departments function effectively and transparently without external assistance
•   Municipalities administer property tax as a source of revenue
                                            Source: USAID Growth and Fiscal Stability Initiative Assistance Agreement.




                                            Liberia: Governance and Economic Management Support Program
                                            (GEMS)

                                            Time period: July 2011-June 2016
                                            Contract amount: $44,902,679

                                            Description: USAID-GEMS is to address significant governance
                                            challenges remaining after USAID completed its previous capacity-
                                            building program in 2010. The program is to strengthen human and
                                            institutional capacity within selected ministries, agencies, and
                                            commissions. USAID-GEMS is to develop and maintain systems that
                                            increase transparency and accountability, increase efficiency, reduce
                                            expenditures, increase revenue, and limit corruption.




                                            Page 52                                                                 GAO-12-920 Public Financial Management
                                             Appendix IV: Examples of Public Financial
                                             Management-related Projects in Six Countries




Table 4: USAID’s Liberia GEMS: Selected Objectives, Subobjectives, and Expected Results

Objective: Broadened capacity building for ministries and agencies
Subobjective: Management systems of targeted government institutions meet international good practices and standards
Expected results
•   Improve management systems at selected state-owned enterprises
•   Participating ministries and agencies conform with international good practice standards
•   Targeted ministries and agencies ready to manage U.S. government and other donor funds
Objective: Provide financial management and information technology training
Subobjective: Liberia Institute of Public Administration provides sustainable, quality training, and advisory services to its
clients in financial management and information technology
Expected results
•   Design, deliver, and monitor training that responds to the clients’ needs
•   Provide training and testing for selected international certifications in programs such as computer technician
Objective: Effective concessions monitoring and management
Subobjective: Government of Liberia manages mining, agriculture and forestry concessions effectively
Expected results
•   Establish transparent legal and regulatory framework
•   Use information and communications technology-based systems for management and oversight
Objective: Assistance to the Central Bank of Liberia
Subobjective: National payment system meets West African Monetary Zone requirements
Expected results
•   Implement payment and accounting systems effectively
•   Complete transition to direct deposit payments
                                             Source: USAID Governance and Economic Management Support Program Assistance Agreement and first-year work plan.




                                             Peru: USAID ProDecentralization 2

                                             Time period: August 2008-July 2012
                                             Award amount: $10,644,432

                                             Description: The USAID/Peru ProDecentralization project is in the
                                             second phase. The first phase targeted national, regional, and municipal
                                             institutions responsible for implementing the decentralization process.
                                             The second phase seeks to continue to improve the policy framework at
                                             the national level and strengthen the institutional capabilities of regional
                                             and municipal governments. The national project aims to improve the
                                             legal and policy framework for decentralization, including fiscal policies
                                             that support a more equitable distribution of public resources. Regional
                                             and local-level activities are to target technical assistance and training to
                                             the diverse needs of Peru’s regional and municipal governments in



                                             Page 53                                                            GAO-12-920 Public Financial Management
                                             Appendix IV: Examples of Public Financial
                                             Management-related Projects in Six Countries




                                             effectively administering resources and responding to citizens’ increasing
                                             expectations. Under this project, existing services offered to subnational
                                             governments in planning, budgeting, accountability, and institutional
                                             strengthening are supplemented by new training and technical assistance
                                             programs.

Table 5: USAID’s Peru ProDecentralization 2: Selected Objectives and Expected Results

Objective: Improved legal and policy framework
Expected results
•   Assess legal and policy framework
•   Design a legal and policy framework tool for progress that identifies legislation and policies necessary to further the
    decentralization process
•   Provide technical assistance on critical legal and policy issues
•   Produce an annual report on the status of decentralization
Objective: Strengthened local government capacity to effectively govern
Expected results
•   Reinforce and reward good governance practices
•   Improve implementation of national administrative systems
•   Improve efficiency of subnational governments
•   Innovate strategies to collect taxes at municipal level
Objective: Strengthened good governance in context of regional and local elections
Expected results
•   Discuss at least two priority subjects of the legal and policy framework for decentralization at public hearings
•   Provide all municipalities in the project with clear and updated guidance on the transfer of managerial competencies
                                             Source: USAID ProDecentralization 2 Performance Management Plan and second-year work plan.




Selected Treasury PFM–                       Tables 6 through 8 summarize selected Treasury Office of Technology
Related Projects                             Assessment (OTA) PFM-related projects in Cambodia, Honduras, and
                                             South Africa.

                                             Cambodia: Technical Advisory Services on Budgeting

                                             Time period: October 2009–September 2010

                                             Description: The Cambodian government has been implementing a plan
                                             to create a credible budget and improve accountability. In the current
                                             phase of the plan, OTA and Cambodian government actions are focused
                                             on decentralizing the reform effort to the line ministries and subnational
                                             levels. These actions include expanding the use of strategic and program



                                             Page 54                                                             GAO-12-920 Public Financial Management
                                               Appendix IV: Examples of Public Financial
                                               Management-related Projects in Six Countries




                                               budgeting, implementing a PFM information system, improving
                                               macroeconomic forecasting, enhancing the linkage between the capital
                                               and operating budgets, and improving overall financial accountability. The
                                               overall aim is to improve the ability of the budget to be an instrument for
                                               policy delivery and to support effective and efficient service delivery.

Table 6: Treasury’s Technical Advisory Services to Cambodia on Budgeting: Selected Objectives and Activities

Objective: Improve and expand program budgeting
Action plan activities
•    Review and improve the implementation of program budgeting, including integrating poverty reduction into the Budget Strategy
     Plan and Program Budget guidelines
•    Develop strategy to expand program budgeting
•    Continue capacity development through the further training on program budgeting
Objective: Further improve budget comprehensiveness
Action plan activity
•    Further incorporate off-budget expenditures and revenues into the annual budget, including increasing information on donor funds
Objective: Improve budget integration
Action plan activities
•    Develop a policy on the integration of the budget. Continue to improve integration of the recurrent and capital budgets through
     ongoing initiatives such as the Medium Term Expenditure Framework, Budget Strategy Plan, and Program Budget.
Objective: Provide advisory support as necessary and requested by budget entities
Action plan activity
•    Improve lines of accountability by clarifying roles, functions, and responsibilities
Objective: Improve instruments for encouraging responsible financial management and accountability
Action plan activity
•    Establish a list of flexibilities to be given to those budget entities that meet agreed criteria on satisfactory resource management
Objective: Improve budget classification
Action plan activity
•    Further improve the implementation of new budget classifications and chart of accounts
                                               Source: OTA Terms of Reference and 2010 work plans for Cambodia.




                                               Honduras: Technical Advisory Services on Budgeting

                                               Time Period: January 2011-December 2013

                                               Description: OTA’s technical assistance focuses on implementing
                                               international public sector accounting standards.




                                               Page 55                                                            GAO-12-920 Public Financial Management
                                            Appendix IV: Examples of Public Financial
                                            Management-related Projects in Six Countries




Table 7: Treasury’s Technical Advisory Services on Honduran Budgeting: Selected Objectives and Action Plan Activities

Objective: Assist in the preparation of year-end financial statements
Action plan activities
•    Review financial statements sent to Congress in April 2011
•    Assist with the creation of a consolidated balance sheet
•    Prepare a Consolidated Statement of Cash Flows for the entire government in accordance with international accounting
     standards
Objective: Review laws and regulations and recommend changes to the legislative framework
Action plan activities
•    Review Organic Budget Law, Organic Law of the Supreme Court of Accounts, Public Administrative Law, and other relevant
     regulations
•    Evaluate current application of laws and regulations
•    Provide written recommendations to finance minister
Objective: Identify government business enterprises as defined by the international public sector accounting standards and
evaluate the impact of implementing the required standards
Action plan activities
•    Identify government business enterprises and determine which ones are using appropriate standards
•    Provide assistance to integrate these enterprises into the financial management information system
Objective: Develop regulations and manuals including procedures to identify the data needed and entity responsible for
gathering and recording data and templates for financial statements
Action plan activities
•    Review existing regulations and manuals for the central government and recommend changes
•    Review and provide assistance with the development of manuals and regulations for relevant government entities as they are
     integrated into the financial management information system
Objective: Provide training and materials to Accountant General and staff
Action plan activities
•    Evaluate skill level of accounting staff
•    Develop training materials for accounting staff and assist with training
•    Develop specialized training materials for central administration and assist with training
•    Review financial information provided by other relevant government entities to determine training needs
Objective: Review functionality of the country’s financial management information system. Identify changes needed for the
system to serve as the primary source of financial information throughout the finance ministry.
Action plan activities
•    Review current reporting requirements of the finance ministry
•    Evaluate the financial management information system’s ability to meet reporting requirements of finance ministry directorates
                                            Source: Treasury OTA 2011 Work Plan for Honduras.




                                            Page 56                                             GAO-12-920 Public Financial Management
                                            Appendix IV: Examples of Public Financial
                                            Management-related Projects in Six Countries




                                            South Africa: Technical Advisory Services on Budgeting

                                            Time Period: February 2007—January 2009

                                            Description: Treasury OTA has conducted technical assistance projects
                                            in South Africa in the areas of budget formulation, intergovernmental
                                            finance, infrastructure budgeting, public finance training, and others since
                                            1997. In 2006, the National Treasury of South Africa requested a new
                                            OTA budget project to focus on the organization of an expenditure
                                            program performance evaluation unit, performance information and cost
                                            analysis, and support for the Collaborative Africa Budget Reform
                                            Initiative. The central mission of the resident advisor was to establish an
                                            expenditure program performance review system.

Table 8: Treasury’s Technical Advisory Services to South Africa on Budgeting: Selected Objectives and Activities

Objective: Assist in the development of a performance information management framework and support the implementation
of performance-based budgeting practices
Work plan activities
•  Facilitate agreement among different staff at the National Treasury on the core principles of performance information
   management
•  Gather and provide input with the aim to publish a Performance Information Management Framework
•  Identify opportunities for performance-based budgeting practices (outlined in the framework) to be integrated into existing
   budgeting systems in conjunction with Public Finance and Budget Office
•  Facilitate the implementation of a pilot for performance-based budgeting into the structure of the 2008-2009 budget
Objective: Develop performance review capacity at the National Treasury
Work plan activities
•  Identify the goal, role and primary objectives of a performance review function in conjunction with the Public Finance and Budget
   Offices
•  Assess and make recommendations on the organizational and technical capacity of the National Treasury to conduct
   performance reviews. Work with the Public Finance and Budget Offices to select a performance review(s) related to the following
   potential issues:
•  Reform the budget process for interdepartmental information technology systems
•  Review the efficiency and effectiveness of labor-based programs such as public works
•  Review the efficiency and effectiveness of infrastructure maintenance and rehabilitation program
•  Review the efficiency and effectiveness of training and skills development and other government capacity-building projects
•  Review state-owned enterprises with large accumulated surpluses
•  Assist in conducting a performance review(s) to identify critical interventions, as support for the establishment of a performance
   review function.
•  Assist in the establishment of a permanent performance review function




                                            Page 57                                           GAO-12-920 Public Financial Management
                                            Appendix IV: Examples of Public Financial
                                            Management-related Projects in Six Countries




Objective: Support the oversight role of parliament through the provision of more timely and actionable in-year financial and
performance information
Work plan activities
•  Clarify and obtain agreement on the information needs and expectations of the Joint Budget Committee in Parliament
•  Identify and map the in-year financial and performance information on national departments currently available to the National
   Treasury
•  Develop strategies to collect and report necessary information to the Joint Budget Committee and design report format and
   processes
•  Pilot report format and processes with the Joint Budget Committee
•  Improve and finalize report format and processes
Objective: Support efforts to enhance professionalism and sharing of effective practices between the countries of Africa via
the Collaborative Africa Budget Reform Initiative
Work plan activity
•  Identify possible opportunities for assistance
                                            Source: Treasury OTA 2007 Resident Advisor Work Plan.




                                            Page 58                                                 GAO-12-920 Public Financial Management
Appendix V: Comments from the U.S. Agency
             Appendix V: Comments from the U.S. Agency
             for International Development



for International Development




             Page 59                                     GAO-12-920 Public Financial Management
Appendix V: Comments from the U.S. Agency
for International Development




Page 60                                     GAO-12-920 Public Financial Management
Appendix V: Comments from the U.S. Agency
for International Development




Page 61                                     GAO-12-920 Public Financial Management
Appendix V: Comments from the U.S. Agency
for International Development




Page 62                                     GAO-12-920 Public Financial Management
Appendix VI: Comments from the
             Appendix VI: Comments from the Department
             of the Treasury



Department of the Treasury




             Page 63                                     GAO-12-920 Public Financial Management
Appendix VI: Comments from the Department
of the Treasury




Page 64                                     GAO-12-920 Public Financial Management
Appendix VII: GAO Contact and Staff
                  Appendix VII: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Thomas Melito, (202) 512-9601 or melitot@gao.gov
GAO Contact
                  Cheryl Goodman (Assistant Director), Michael Maslowski, Shirley Min,
Staff             RG Steinman, Michael Hoffman, Debbie Chung, Grace Lui, David
Acknowledgments   Dayton, and Etana Finkler made key contributions to this report.




(320870)
                  Page 65                                GAO-12-920 Public Financial Management
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.