oversight

International Financial Crises: Challenges Remain in IMF's Ability to Anticipate, Prevent, and Resolve Financial Crises

Published by the Government Accountability Office on 2003-06-16.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Chairman, Committee on
             Financial Services, and to the Vice
             Chairman, Joint Economic Committee,
             House of Representatives

June 2003
             INTERNATIONAL
             FINANCIAL CRISES
             Challenges Remain in
             IMF’s Ability to
             Anticipate, Prevent,
             and Resolve Financial
             Crises




GAO-03-734
             a
                                               June 2003


                                               INTERNATIONAL FINANCIAL CRISES

                                               Challenges Remain in IMF's Ability to
Highlights of GAO-03-734, a report to          Anticipate, Prevent, and Resolve
Chairman, House Committee on Financial
Services and Vice Chairman, House Joint        Financial Crises
Economic Committee




Building on reform initiatives                 While the Fund’s new vulnerability framework is more comprehensive than
instituted after the Mexican                   its previous efforts, it is too early to assess whether it will improve Fund
financial crisis, the IMF                      efforts to anticipate crises. The new framework uses the Fund’s major
implemented new initiatives in the             forecasting tools, the World Economic Outlook (WEO) and the Early
mid-1990s to better anticipate,                Warning System (EWS), which have not performed well in anticipating prior
prevent, and resolve sovereign
financial crises. GAO was asked to
                                               crises. The forecasting of crises has been historically difficult for all
assess (1) the IMF’s framework for             forecasters.
anticipating financial crises, (2) the
status of key IMF reform initiatives           The Fund, with the World Bank, has made progress in implementing
to prevent financial crises, and (3)           initiatives to prevent crises, but several challenges remain. To obtain better
new IMF proposals to resolve                   information about country financial sector weaknesses, the Fund and Bank
future financial crises.                       introduced the Financial Sector Assessment Program (FSAP) to report on
                                               member countries’ financial sectors and the Reports on the Observance of
                                               Standards and Codes (ROSC) to assess member countries’ adherence to 12
GAO recommends that the                        standards. Assessments have not been completed in some major emerging
Secretary of the Treasury instruct             market countries primarily because participation is voluntary, and use of this
the U.S. Executive Director of the             information has been mixed. For example, some private sector market
Fund to work with other Executive              participants have found the reports untimely, outdated, and dense.
Board members to encourage the
Fund to                                        Participation Gaps by 33 Major Emerging Market Countries in Key Assessments, 1999-2003
•   improve the timeliness of                  Number of assessments
    FSAP and ROSC reports;
•   expand the coverage,
    frequency, and publication of
    updates of participants’
    implementation of FSAP and
    ROSC recommendations;
•   improve the FSAP and ROSC
    reports’ readability; and
•   increase participation in the
    FSAP and all standards of the
    ROSC and consider making
    participation mandatory.

Treasury, IMF, and the World Bank
generally agreed with the report’s
recommendations. The IMF stated
that we gave WEO and EWS                       The Fund is considering two approaches to restructuring unsustainable
forecasts greater importance than              sovereign debt; however, there are significant challenges to implementing
is warranted in anticipating crises.
                                               them. One approach involves creating an international legal framework that
However, we focused on the only
mature and quantifiable elements               would allow a specified majority of a country’s external creditors to
of the vulnerability framework.                restructure most private sector loans. Under the second approach, the Fund
                                               is encouraging members to include renegotiation clauses in individual
www.gao.gov/cgi-bin/getrpt?GAO-03-734.         bonds. Many private sector representatives wish to maintain the existing
                                               process in which the Fund assists resolution by providing loans to some
To view the full report, including the scope
and methodology, click on the link above.      eligible members. In response to concerns that its resources may have
For more information, contact Joseph A.        unintended negative impacts during a crisis, the Fund has clarified and
Christoff at (202) 512-8979 or                 strengthened its criteria for lending to members experiencing crises.
christoffj@gao.gov.
Contents



Letter                                                                                                      1
                              Background                                                                    2
                              Results in Brief                                                              4
                              Too Early to Determine If Fund’s New Process for Anticipating
                                Financial Crises Improves on Past Efforts                                   7
                              Some Progress in Implementing Crisis Prevention Initiatives, but
                                Challenges Remain                                                          15
                              The Fund’s Efforts to Better Resolve Future Financial Crises Face
                                Significant Challenges                                                     24
                              Conclusion                                                                   30
                              Recommendations for Executive Action                                         31
                              Agency Comments and Our Evaluation                                           31


Appendixes
               Appendix I:    Objectives, Scope, and Methodology                                           34
               Appendix II:   Private Sector Structured Interview Participants                             37
              Appendix III:   Assessment of IMF Forecasting                                                38
                              GAO Analysis Used Standard Econometric Techniques for Forecast
                                Evaluation                                                                 38
              Appendix IV:    Eighty-seven Emerging Market Countries                                       50
               Appendix V:    Standards, Codes, and Principles Assessed under IMF and
                              World Bank Reports on the Observance of Standards and
                              Codes                                                                        53
              Appendix VI:    Update on the International Monetary Fund’s Safeguards
                              Assessments                                                                  56
                              The Fund Introduces Safeguards Assessments for Member
                                Countries That Currently Borrow                                            56
                              Implementation of the Fund’s Assessment Recommendations                      57
             Appendix VII:    Fund/World Bank FSAP and ROSC Process                                        59
             Appendix VIII:   Country Participation in and Publication of FSAPs and
                              ROSCs                                                                        60
              Appendix IX:    Fund Resources Provided under the Supplemental Reserve
                              Facility                                                                     64
               Appendix X:    Comments from the Department of the Treasury                                 65
              Appendix XI:    Comments from the International Monetary Fund                                67
                              GAO Comments                                                                 70



                              Page i                                 GAO-03-734 International Financial Crises
                           Contents




          Appendix XII:    Comments from The World Bank                                                72
                           GAO Comments                                                                74
          Appendix XIII:   GAO Contacts and Staff Acknowledgments                                      75
                           GAO Contacts                                                                75
                           Staff Acknowledgments                                                       75


Tables                     Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial
                                    Crises, 1990–2001                                                  12
                           Table 2: Forecast Quality for 87 Emerging Market Countries, 1990–
                                    2001                                                               43
                           Table 3: Forecast Quality for G-7 Countries, 1990–2001                      45
                           Table 4: Comparison of Year-Ahead WEO Forecasts for Program
                                    and Nonprogram Countries, 1990–2001                                46
                           Table 5: Comparison of Program and WEO Year-Ahead Forecasts,
                                    for 52 Countries in the Initial Years on Program,
                                    1990–2001                                                          47
                           Table 6: IMF Supplemental Reserve Facility Loans, 1997–2002                 64


Figures                    Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve
                                     Financial Crises                                                   4
                           Figure 2: IMF Vulnerability Assessment Framework                             8
                           Figure 3: Participation of 33 Major Emerging Markets in Key ROSC
                                     Assessments, 1999–2003                                            20
                           Figure 4: FSAP and ROSC Participation and Publication by Major
                                     Emerging Market Countries                                         61
                           Figure 5: FSAP and ROSC Participation and Publication by Other
                                     Countries                                                         62



                           Abbreviations

                           BCP               Basel Core Principles
                           BMA               Bond Market Association
                           CAC               Collective Action Clauses
                           CCFF              Compensatory Contingency Finance Facility
                           CPI               consumer price index
                           CPSS              Committee on Payments and Settlements Systems
                           DCSD              Developing Country Studies Division
                           EFF               Extended Fund Facility
                           EMCA              Emerging Market Creditors Association




                           Page ii                               GAO-03-734 International Financial Crises
Contents




EMTA                  Emerging Market Traders Association
EWS                   Early Warning System
FATF                  Financial Action Task Force
FSA                   Financial Sector Assessment
FSAP                  Financial Sector Assessment Program
FSSA                  Financial System Stability Assessment
GDDS                  General Data Dissemination Standard
GDP                   gross domestic product
IAIS                  International Association of Insurance Supervisors
IASB                  International Accounting Standards Board
IFAC                  International Federation of Accountants
IMF                   International Monetary Fund
IOSCO                 International Organization of Securities Commissions
IPMA                  International Primary Market Association
ISMA                  International Securities Market Association
KLR                   Kaminsky, Lizondo, and Reinhart
MSE                   mean square error
OECD                  Organization for Economic Cooperation and Development
RMSE                  root mean square error
ROSC                  Reports on the Observance of Standards and Codes
RSS                   Residual Sum of Squares
SBA                   Stand-By Arrangements
SDDS                  Special Data Dissemination Standard
SDRM                  Sovereign Debt Restructuring Mechanism
SEC                   U.S. Securities and Exchange Commission
SIA                   Securities Industry Association
SRF                   Supplemental Reserve Facility
STF                   Systemic Transformation Facility
UNCITRAL              United Nations Commission on International Trade Law
WEO                   World Economic Outlook




 This is a work of the U.S. Government and is not subject to copyright protection in the
 United States. It may be reproduced and distributed in its entirety without further
 permission from GAO. It may contain copyrighted graphics, images or other materials.
 Permission from the copyright holder may be necessary should you wish to reproduce
 copyrighted materials separately from GAO’s product.




Page iii                                         GAO-03-734 International Financial Crises
A
United States General Accounting Office
Washington, D.C. 20548



                                    June 16, 2003                                                                     Lert




                                    The Honorable Michael G. Oxley
                                    Chairman
                                    Committee on Financial Services
                                    House of Representatives

                                    The Honorable Jim Saxton
                                    Vice Chairman
                                    Joint Economic Committee
                                    House of Representatives

                                    In May 2000, in the aftermath of the Asian financial crisis, the newly
                                    appointed Managing Director of the International Monetary Fund (IMF or
                                    the Fund) announced his plans to strengthen the Fund’s reform initiatives
                                    to enable the Fund to more effectively safeguard the stability of the
                                    international financial system. Building on reform initiatives instituted in
                                    the mid-1990s, after the Mexican financial crisis of 1994–95, these new
                                    initiatives were designed to better anticipate, prevent, and resolve
                                    sovereign financial crises. A sovereign financial crisis can occur when a
                                    country is unable or unwilling to honor its debt obligations and investors
                                    lose confidence in that country’s financial markets. Recent crises have
                                    occurred primarily in “emerging market” countries, larger and more
                                    economically advanced developing countries such as Argentina, Brazil, and
                                    Turkey. According to World Bank (the Bank) estimates, the financial costs
                                    to countries that experienced crises in the 1980s and 1990s exceeded $1
                                    trillion—greater than the total amount of all donors’ assistance to
                                    developing countries in the 1980s and 1990s.

                                    This letter responds to your request that we examine the Fund’s efforts to
                                    better safeguard the stability of the international financial system. In this
                                    report, we assess (1) IMF’s framework for anticipating financial crises, (2)
                                    the status of key IMF initiatives to prevent financial crises, and (3) new IMF
                                    proposals to resolve future financial crises. In addition, we analyzed the
                                    quality of IMF’s forecasts produced by the World Economic Outlook
                                    (WEO), the Fund’s primary forecasting tool (see appendix III).

                                    As part of our assessment, we reviewed documents from the U.S.
                                    government, international organizations, and private firms, including
                                    testimonies, reports, and relevant laws. We used statistical models that we
                                    developed to assess the Fund’s WEO forecasts and program projections for
                                    selected emerging market countries. We interviewed key officials at the



                                    Page 1                                    GAO-03-734 International Financial Crises
             Fund, World Bank, Department of the Treasury (Treasury), and emerging
             market bond and equity funds. We also interviewed some representatives
             of private sector firms that specialize in international finance, economics,
             and law. Appendix I provides a more detailed description of our objectives,
             scope, and methodology. Appendix II provides a list of the private sector
             firms we interviewed.



Background   In 1994–95, Mexico faced a severe financial crisis when a shift in market
             sentiment led to sudden large capital outflows. Investors also temporarily
             removed their funds from other emerging market countries, an effect
             known as contagion. In response to the crisis, Mexico quickly adopted a
             strong and ultimately successful program of adjustment and reform. To
             support the program, the Fund approved a loan of $17.8 billion to Mexico—
             one of the largest loan commitments it had ever made to a country. One of
             the major reasons cited for the crisis was the lack of timely, reliable, and
             publicly available economic, financial, and sociodemographic data for
             Mexico. Beginning in 1996, to correct this weakness, the Fund created data
             standards to guide countries in disseminating better data to the public.
             However, as we reported in 1997, the Fund needed to address a number of
             other financial, economic, and political challenges, in addition to data
             limitations, to better anticipate, prevent, and resolve financial crises.1

             Before the Fund could fully address these challenges, the Asian financial
             crisis of 1997–98 occurred. After the Asian financial crisis, the Fund
             assessed the effects of its responses to the crisis and reassessed its role in
             safeguarding the stability of the international financial system, including
             rethinking its core mission, operations, and lending activities. The Fund
             also recognized that it needed to improve its ability to anticipate financial
             crises; monitor countries’ activities; and increase public awareness,
             particularly that of the investment community. Recognizing its inability to
             anticipate past crises, the Fund instituted a quarterly vulnerability
             assessment framework in 2001 to identify countries that may be
             susceptible to crisis.




             1
              U.S. General Accounting Office, International Financial Crises: Efforts to Anticipate,
             Avoid, and Resolve Sovereign Crises, GAO/GGD/NSIAD-97-168 (Washington, D.C.: July 7,
             1997).




             Page 2                                         GAO-03-734 International Financial Crises
To improve its ability to prevent future crises, the Fund and the World Bank
in 1999 began performing joint assessments of member countries’ financial
sectors to help identify and monitor existing and potential weaknesses. In
addition, the Fund and the World Bank began to work with countries to
promote adherence to voluntary standards to reassure the international
community that the countries’ policies and practices conform to standards
and codes of good practice. These include standards to improve
transparency in government economic data; fiscal, monetary, and financial
policies; and guidelines on strengthening the financial and corporate
sectors.

The Fund acknowledges that it would be almost impossible to anticipate or
prevent all crises. According to the Fund, past efforts to resolve financial
crises during the 1990s were lengthy and very costly to debtor countries.
The Fund is encouraging the adoption of agreements that would allow a
quicker, more orderly, and predictable restructuring of countries’ debts.
The Fund’s ultimate goal is to maintain investor confidence and stability in
the international financial system. Figure 1 shows the Fund’s key initiatives
for better anticipating, preventing, and resolving financial crises. Treasury
through the U.S. Executive Director to the Fund has the lead responsibility
for monitoring the IMF’s progress in addressing these issues.




Page 3                                   GAO-03-734 International Financial Crises
                   Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve Financial Crises




Results in Brief   While it is too early to assess the effectiveness of the Fund’s new
                   vulnerability framework for anticipating crises, this framework builds on
                   key elements that have been unable to anticipate past financial crises. The
                   new framework is more comprehensive than the Fund’s previous efforts,
                   bringing together country-specific knowledge and financial expertise
                   within the Fund to better identify weaknesses in emerging market
                   economies that could lead to a crisis. However, the new framework uses
                   the Fund’s major forecasting tools, the WEO and the Early Warning System
                   (EWS), which have not performed well in anticipating prior crises. The
                   WEO has not successfully anticipated past financial crises, and the Fund’s
                   EWS models have had a high false alarm rate, having predicted many crises
                   that did not occur. The forecasting of crises has been historically difficult
                   for all forecasters due to complex underlying factors, including concerns
                   about the reliability of important macroeconomic data on emerging market
                   countries.




                   Page 4                                        GAO-03-734 International Financial Crises
The Fund, in collaboration with the World Bank, has made progress in
implementing initiatives to prevent financial crises, but several challenges
remain. In the late 1990s, the Fund and the World Bank adopted two key
initiatives designed to assist four parties: Fund staff, World Bank staff,
member country governments, and private sector participants. First,
financial sector assessments were introduced to provide reports on aspects
of member countries’ financial sectors such as banking systems and crisis
management capacity. Second, the standards initiative was adopted to
assess member countries’ adherence to 12 standards in areas such as
banking supervision and economic data dissemination. Use of the
information provided by these two initiatives by various parties has been
mixed, and several significant challenges remain. Fund staff frequently
incorporate information from the assessments into their policy advice to
member countries. However, assessments have not been completed in
some important emerging market countries, such as China and Thailand,
primarily because participation is voluntary. World Bank staff’s use of the
assessments to inform country development assistance programs is also
affected by the lack of participation by member countries and by borrower
countries’ competing development needs. Member country governments
sometimes use the assessments in prioritizing their reform agendas but
often find that the reforms are too difficult to implement. Some private
sector participants find the published reports outdated, untimely, and too
dense to be useful in making investment decisions. The Fund and the World
Bank acknowledge that the assessments cannot prevent all crises because
reforms require many years to be fully implemented, and crises can be
caused by factors outside the scope of the reforms.

The Fund is considering two approaches to achieve a more orderly and
predictable restructuring of unsustainable debt between a country and its
creditors; however, there are significant challenges to implementing both
approaches. Fund officials assert that the current process for renegotiating
the terms of member countries’ loans with external private sector creditors
is lengthy and costly. The Sovereign Debt Restructuring Mechanism
(SDRM), is a proposed international legal framework that would allow a
majority of a country’s external creditors to quickly restructure most
private sector loans. The Fund is also encouraging members to include
Collective Action Clauses (CACs) in individual bonds, which would allow a
majority of bondholders to renegotiate the bond’s terms. Although some
elements of both approaches are acceptable to the private sector and to
governments, a number of political, legal, and technical challenges stand in
the way of implementing the SDRM; it seems unlikely that these issues will
be resolved in the immediate future. While private sector officials expect



Page 5                                   GAO-03-734 International Financial Crises
that many restructurings need only involve the private sector and the
debtor country, under some circumstances, voluntary debt restructurings
will not adequately resolve all financial crises. In those cases, they said the
Fund should provide loans to eligible countries to help fill their external
financing gaps. However, some financial experts and government
representatives have raised concerns that such loans have the potential to
increase the probability of future crises. In response, the Fund has clarified
and strengthened its policy of lending in crisis situations.

The Fund’s crisis prevention initiatives are hindered by several factors that
limit their effectiveness. To strengthen the effectiveness of these initiatives,
we are recommending that the Secretary of the Treasury instruct the U.S.
Executive Director of the Fund to work with other Executive Board
members to encourage the Fund to improve the readability, timeliness,
coverage, and frequency of updates of assessment reports. Additionally, the
Fund should pursue strategies for increasing participation in the
assessment process including the possibility of making participation
mandatory for all members of the IMF.

In responding to our draft report, Treasury, IMF, and the World Bank
generally agreed with the report’s recommendations. However, the IMF
stated that we mischaracterized the role of WEO forecasts and EWS
models in IMF crisis anticipation efforts by saying that they have a greater
importance than is warranted. We disagree with this depiction. Our
assessment examined all six components of the IMF’s vulnerability
assessment framework, including the WEO and EWS. As the only mature
and quantifiable elements of the framework, our analysis focused more
heavily on the track records of the WEO and EWS. The World Bank
expressed concern with the report’s suggestion that consideration be given
in making participation in the FSAP and ROSC assessments mandatory.
While we are not suggesting that the assessments should be made
mandatory, the voluntary nature of these assessments has posed an
obstacle to full participation by important emerging market countries.




Page 6                                     GAO-03-734 International Financial Crises
Too Early to Determine     In May 2001, the IMF implemented a new vulnerability assessment
                           framework for emerging market countries to strengthen the Fund’s ability
If Fund’s New Process      to anticipate financial crises. This framework brings together country-
for Anticipating           specific knowledge and financial expertise within the Fund to better
                           identify weaknesses in emerging market economies that could lead to a
Financial Crises           crisis. Although the new vulnerability assessment framework is more
Improves on Past           comprehensive than the Fund’s previous efforts, it is new and still evolving.
Efforts                    It is too early to tell whether this new framework will successfully
                           anticipate future crises. The new framework uses the Fund’s major
                           forecasting tools, the WEO and the EWS, which have not performed well in
                           anticipating prior crises. The WEO has not successfully anticipated past
                           financial crises, and the Fund’s EWS models have had a high false alarm
                           rate, having predicted many crises that did not occur. The forecasting of
                           crises has been historically difficult for all forecasters due to complex
                           underlying factors, including concerns about the reliability of important
                           macroeconomic data on emerging market countries.



The IMF Has Recently       The Fund has attempted for many years to identify countries vulnerable to
Implemented a New          financial crisis; however, their existing tools were insufficient to anticipate
                           the financial crises of the 1990s and led the Fund in 2001 to develop the
Vulnerability Assessment
                           new vulnerability assessment framework. This comprehensive framework
Framework                  brings together detailed, country-specific knowledge and financial
                           expertise of various IMF departments, including those with regional,
                           macroeconomic, or forecasting expertise. The new framework monitors
                           the vulnerability of key emerging market countries that borrow
                           significantly from international capital markets. This information is
                           provided in a quarterly report on crisis vulnerability. Fund staff report
                           monthly on countries identified as vulnerable and provide more frequent ad
                           hoc analyses during volatile periods.

                           To conduct the vulnerability assessment, the Fund integrates six
                           independent inputs that represent the analyses and perspectives of
                           different departments of the Fund (see figure 2).




                           Page 7                                    GAO-03-734 International Financial Crises
Figure 2: IMF Vulnerability Assessment Framework




• World Economic Outlook: The WEO is a twice yearly publication that
  provides analyses of global economic developments. Through the WEO,
  the IMF provides current and following year forecasts for countries and
  regions of key economic variables such as economic growth, inflation,
  and current account.2 According to Fund staff, WEO forecasts use the
  best available information and represent the most realistic estimate of
  key economic variables, including those that could help anticipate a
  financial crisis. The IMF uses these forecasts as an input in the
  vulnerability assessment to gauge the impact of unanticipated adverse
  changes in the global environment. For example, the WEO forecasts for
  selected countries may be recalculated to examine the impact of sudden



2
 The current account represents a country’s trade balance (exports less imports of goods
and services) plus the net interest income (or payments) on outstanding international
investments (or debts) plus net transfers (grants, workers’ remittances).




Page 8                                           GAO-03-734 International Financial Crises
   increases in oil prices or an unanticipated recession in the advanced
   economies.

• Early Warning System models: The Fund uses internal and private
  sector EWS models that compute the probability of a country having a
  crisis over the following 12 to 24 months. The model examines a series
  of vulnerability indicators, including whether a country’s real exchange
  rate is overvalued, or whether the country has significantly depleted its
  foreign exchange reserves.3 The output of the EWS models helps the
  Fund focus on specific areas of vulnerability. For example, if one
  variable, such as the exchange rate, signals a crisis, the Fund would
  more closely examine related components of the vulnerability
  assessment, such as a country’s external financing requirements.

• Country external financing requirements: On a quarterly basis, the
  Fund produces an internal assessment of a country’s ability to meet its
  total external debt obligations and estimates whether that country has
  sufficient foreign exchange to avoid a crisis. This assessment includes
  estimating a country’s ratio of foreign exchange reserves to short-term
  external debt, estimating the magnitude of its current account deficit,
  and considering whether and how it manages its exchange rate.

• Market information: On an ongoing basis, the Fund analyzes most
  countries’ cost of borrowing on the international market and whether
  the country is paying a higher interest rate than similar countries. The
  Fund uses this information to provide an internal analysis of the private
  sector’s expectations of a country’s likelihood of default on its foreign
  debt and to identify possible evidence that financial problems are
  spreading across countries.

• Financial sector vulnerability: The Fund assesses the strengths and
  weaknesses of a country’s financial sector, including the banking
  system. IMF staff evaluate the financial sector’s vulnerability to changes
  in market conditions, such as fluctuations in interest and exchange
  rates. Although the detailed results of these assessments are used
  internally by the Fund, summaries of key findings are frequently
  published.


3
 Foreign exchange reserves are those external assets that are readily available to and
controlled by monetary authorities. Reserves can be used for direct financing of payment
imbalances, for indirectly regulating the exchange rate, and/or for other purposes.




Page 9                                          GAO-03-734 International Financial Crises
                                 • Country expert perspectives: IMF country experts examine the data
                                   produced by the above analyses, supplementing those results with
                                   country-specific details such as the political risks of implementing
                                   certain government policies or the relevance of certain market
                                   information.

                                 Until 1999, the Fund used the WEO as the primary forecasting tool to help
                                 identify country risks and vulnerability to crises. The new vulnerability
                                 framework, which has been in operation for about 2 years, is a more
                                 comprehensive process. According to the Fund, the quarterly integration of
                                 detailed information from country experts who continuously monitor
                                 developments in their countries is a great strength of the new vulnerability
                                 framework. Effective analysis by Fund staff of the framework’s six
                                 elements could better allow the Fund to give timely advice to authorities in
                                 vulnerable countries. It is too early to tell whether this framework will be
                                 successful in anticipating future crises. We assess the performance of the
                                 WEO and EWS models, the Fund’s primary tools for anticipating crises
                                 prior to the implementation of the new framework in May 2001.4



Forecasting Elements of the      The new vulnerability assessment framework uses the Fund’s two major
Vulnerability Framework          forecasting tools—the WEO and the EWS models—which have not
                                 performed well in anticipating prior crises. The WEO has not successfully
Have Performed Poorly in         anticipated the severe financial crises of the past decade. The Fund’s EWS
the Past                         models have had a high false alarm rate, having predicted many crises that
                                 did not occur.

The WEO Is Not a Reliable Tool   Severe financial crises are characterized by a number of negative economic
for Anticipating Crises          outcomes, including large declines in gross domestic product (GDP), also
                                 known as recessions. We found that the WEO had a poor record of
                                 forecasting such declines in GDP, tending instead to follow existing
                                 positive economic growth trends. In addition, the IMF indicates that the
                                 current account is a key variable in explaining financial crises. We found
                                 that for the current account, the accuracy of 75 percent of WEO country
                                 forecasts was worse than simply assuming that next year’s value is the
                                 same as this year. The WEO’s difficulty forecasting both GDP and current



                                 4
                                  Data on the performance of the other four components of the framework were not made
                                 available to us because these elements are considered part of the staff level deliberative
                                 process and are not provided to the Executive Board.




                                 Page 10                                          GAO-03-734 International Financial Crises
account demonstrate poor performance in anticipating the severe financial
crises of the past decade.

WEO Has Performed Poorly at Forecasting Recessions

In most cases, countries experiencing a financial crisis also experience a
severe recession in which their GDP declines significantly. Although most
recessions do not involve a major financial crisis, successful anticipation of
recessions, especially the most severe ones, would greatly increase the
likelihood of anticipating impending financial crisis. However, we found
that the WEO has a poor track record of forecasting recessions, including
those directly associated with a financial crisis.

The WEO did not forecast most of the recessions that occurred in emerging
market countries in the last 10 years. During the 1991–2001 forecast period,
134 recessions occurred in all 87 emerging market countries. We found that
the WEO correctly forecast only 15, or 11 percent, of those recessions,
while predicting an increase in GDP in the other 119 actual recessions.5 The
WEO is considerably more likely to forecast a recession when a recession
has occurred in the prior year. However, a prior year recession did not
occur in two-thirds of the recessions that the WEO failed to forecast. Thus,
WEO forecasts generally follow the existing growth trend within a country,
making it unlikely that the WEO would correctly forecast an unanticipated
recession. Furthermore, this tendency to follow the current growth trend
makes it especially difficult for the WEO to anticipate a financial crisis
because nearly all of the crisis-related recessions of the last decade
occurred in years following positive economic growth.

Further illustrating our point, the WEO was unable to anticipate large
declines in GDP, also known as recessions, that corresponded to 14 major
financial crises of the past decade, including the Mexican and Asian
financial crises (see table 1).6




5
In one case, the WEO forecast zero growth.
6
 Several studies examining international financial crises from 1990–2001 identified 14 major
financial crises in 12 prominent emerging market countries that were considered to be
fundamentally sound before the crisis. Balance of payments problems, large GDP
contractions, and financial panic commonly characterize these crises.




Page 11                                          GAO-03-734 International Financial Crises
Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial Crises, 1990–2001

                                                             WEO forecast               Actual GDP
                                                           GDP growth rate              growth ratea
Country                                    Crisis years          (percent)                 (percent)
Argentina                                         1995                      4.0                   -4.4
Argentina                                         2001                      3.7                   -4.4
Brazil                                            1998                      4.0                    0.1
Ecuador                                           1999                      2.5                   -7.3
Indonesiab                                        1998                      6.2                 -13.7
Malaysiab                                         1998                      6.5                   -6.7
Mexico                                            1995                      4.0                   -6.9
Philippinesb                                      1998                      5.0                   -0.5
Russia                                            1998                      5.1                   -4.6
South Koreab                                      1998                      6.0                   -5.8
Thailandb                                         1998                      3.5                   -9.4
Turkey                                            1994                      3.0                   -5.5
Turkey                                            2001                      5.5                   -7.4
Venezuela                                         1994                      2.5                   -3.3
Average growth rates                                                        4.4                   -5.7
Sources: Various crisis studies and IMF.
a
Actual GDP growth rate is from the WEO published 1 year later.
b
 The Asian crisis began in Thailand in the summer of 1997 and spread to the other Asian countries in
the latter part of 1997. The full effects of the crisis on the GDP of other Asian countries and Russia
occurred in 1998. Thus we are using 1998 actual GDP growth rate data for the Asian countries—
Indonesia, Malaysia, Philippines, South Korea, and Thailand. The crisis in Brazil extended into1999.


The WEO’s failure to identify these recessions demonstrates that it did not
anticipate the corresponding financial crises. In 14 cases, we found that the
WEO forecast strong economic growth, averaging a 4.4 percent increase in
GDP, despite large declines in actual GDP in 13 of 14 cases.7 In fact, actual
GDP declined by an average of 5.7 percent during the first full year of these
14 financial crises. Indonesia presents the most startling disparity, in which
the WEO forecast a growth of 6.2 percent in its GDP, when in fact
Indonesia’s GDP declined by almost 14 percent in the first full year of its
financial crisis.



7
 Brazil was the sole exception, having experienced a small 0.1 percent GDP annual growth
rate during the 1998 financial crisis.




Page 12                                                GAO-03-734 International Financial Crises
WEO Does a Poor Job in Forecasting the Current Account

According to the Fund, a country’s current account (primarily exports
minus imports of goods and services) is a key variable in anticipating
crises. Crises are associated with problems of external financing that result
from a country having difficulty obtaining foreign exchange. Since exports
are an important source of foreign exchange for developing countries,
projections of a country’s current account balance provide information
about the country’s ability to earn foreign exchange and to service its
external debt.8 According to the Fund, an unsustainably large current
account deficit can contribute to or precipitate a crisis.9

We found that WEO forecasts for current account were inaccurate most of
the time. Our analysis for the 87 emerging market countries shows that, for
more than 75 percent of the countries, the WEO current account forecasts
were less accurate than if the Fund had simply assumed that the next year’s
current account would be the same as this year’s. The results are even more
dramatic for G-7 countries; a forecast of no change was a better predictor
than the WEO forecast for six of the seven countries. This demonstrates
that, even in stable economies with excellent data, the WEO has done a
poor job of forecasting this key crisis anticipation variable. (See appendix
III for more detailed explanation on our methodology and findings).




8
 A deficit in the current account can also be financed by capital inflows, such as foreign
private investment, a drawing down of a country’s international reserves, by bilateral or
multilateral loans, or by provision of exceptional finance, such as debt service relief.
9
 Fund studies indicate that the current account deficit, as a percentage of GDP, is
significantly larger before a crisis than during relatively stable periods, which suggests that
unsustainable current account deficits tended to be part of the general overheating of the
economy before a crisis.




Page 13                                            GAO-03-734 International Financial Crises
The Fund’s Early Warning      Since 1999, the Fund has analyzed the results from internal and private
System Models Indicate Many   sector EWS models in its crisis anticipation efforts. The Fund’s internal
Crises That Do Not Occur      efforts focused on two EWS models to systematically identify countries
                              vulnerable to crises: the Kaminsky, Lizondo, and Reinhart (KLR) model,
                              which monitors a set of 15 monthly variables that signal a crisis whenever
                              any cross a certain threshold; and the Developing Country Studies Division
                              (DCSD) model, which uses five variables to compute the probability of a
                              crisis occurring in the next 24 months.10 The Fund’s models use a variable-
                              by-variable approach that allows economists to determine which variables
                              are signaling the crisis. Internal assessments of the Fund’s EWS models
                              show that they are weak predictors of actual crises. While the models
                              worked reasonably well in anticipating Turkey’s recent financial crisis, they
                              did not successfully anticipate Argentina’s financial crisis in 2002.
                              According to the Fund, the models’ most significant limitation is that they
                              have high false alarm rates; that is, they predict many crises that did not
                              occur. In about 80 percent of the cases where a crisis was predicted over
                              the next 24 months, no crisis occurred. Furthermore, in about 9 percent of
                              the cases where no crisis was predicted, there was a crisis.11



Forecasting Crises Has        Financial crises have been historically difficult to anticipate because of a
Been Historically Difficult   number of complex underlying factors. Economic outcomes are often
                              influenced by unanticipated events such as conflicts and natural disasters.
for All Parties               Many factors, in addition to weaknesses in a country’s financial structure,
                              can lead to a crisis. These include economic disturbances, such as an
                              unanticipated drop in export prices, political events, and changes in
                              investor sentiment leading to sudden withdrawals of foreign capital.
                              Furthermore, data may be inadequate, particularly in developing countries
                              where data are often not timely and are of poor quality.




                              10
                                More specifically, the DCSD model is a multivariate probit regression. The five variables
                              used are real exchange rate overvaluation, current account, foreign exchange reserve
                              losses, export growth, and the ratio of short-term debt to foreign exchange reserves. Also,
                              the model defines a currency crisis as the weighted average of 1-month changes in exchange
                              rate and reserves more than 3 (country-specific) standard deviations above country average.
                              11
                                The DCSD model had a cutoff probability of 23 percent and the KLR model had a cutoff
                              probability of 15 percent. A forecast probability above these cutoff points is deemed to
                              signal a crisis.




                              Page 14                                          GAO-03-734 International Financial Crises
                          Forecasters consistently fail to foresee crises and recessions. Forecasts
                          produced by private sector economic forecasters, governments, and
                          multinational agencies including the IMF and the Organization for
                          Economic Cooperation and Development, routinely fail to foresee the
                          coming of crises and recessions, and often fail to outperform the naive
                          model, which simply assumes that next year’s outcome will be the same as
                          this year’s. This is true even for evaluations of recent U.S. forecasts of GDP
                          and inflation. Our review of a number of forecast evaluation studies
                          confirms that the inability to predict recessions is a common feature of
                          growth forecasts for both industrialized and developing countries. The
                          studies also showed that forecast accuracy improves as the time horizon
                          gets shorter, and that there is little difference in forecast accuracy between
                          private sector and WEO forecasts.



Some Progress in          In the late 1990s, the Fund and the World Bank began implementing two
                          crisis prevention initiatives designed to assist four parties: IMF staff, World
Implementing Crisis       Bank staff, member country governments, and private sector participants.
Prevention Initiatives,   The first initiative, the financial sector assessments, provides reports on
                          aspects of member countries’ financial sectors such as banking systems
but Challenges Remain     and crisis management capacity. The second, the standards initiative,
                          assesses member countries’ adherence to 12 standards in areas such as
                          banking supervision and economic data dissemination. Parties’ use of the
                          information provided by these two initiatives has been mixed, and several
                          significant challenges remain. Fund staff frequently incorporate
                          information from the assessments into their policy advice when
                          assessments are available; however, assessments have not been completed
                          in some important emerging market countries primarily because
                          participation is voluntary. Bank staff’s use of the assessments to inform
                          country development assistance programs is also affected by gaps in the
                          completion of some assessments and by borrower countries’ competing
                          development demands. Member country governments sometimes use the
                          assessments in prioritizing their reform agendas but often find the reforms
                          too difficult to implement. Some private sector participants find the
                          published reports untimely, outdated, and too dense to be useful in making
                          investment decisions. The IMF and the Bank acknowledge that the
                          initiatives cannot prevent all crises because recommended reforms require
                          many years to be fully implemented, and crises can be caused by factors
                          outside the scope of the reforms.




                          Page 15                                   GAO-03-734 International Financial Crises
Two Key Crisis Prevention   In the wake of the Mexican and Asian financial crises of the 1990s, the
Initiatives Target Four     Fund and the World Bank became increasingly aware of the importance of
                            transparent financial data and policies, stronger financial systems, and
Parties                     better-functioning markets as a complement to member country
                            governments’ sound macroeconomic policies. Fund evaluations
                            acknowledge that the institution failed to collect information that could
                            have enabled it to detect financial and corporate sector vulnerabilities and
                            to provide appropriate policy advice to the affected countries before the
                            crises occurred.

                            In response to this, in the late 1990s, the Fund and the Bank jointly
                            launched two initiatives to prevent the long-term likelihood of financial
                            crises. The first initiative, the Financial Sector Assessment Program
                            (FSAP), consists of in-depth assessments of key elements of member
                            countries’ financial sectors. These elements include the structure of
                            financial markets, financial systems’ response to changes in key variables
                            such as exchange rates, legal arrangements for crisis management, and the
                            quality of financial sector supervision. The second initiative, the Reports on
                            the Observance of Standards and Codes (ROSC), consists of assessments
                            of member countries’ adherence to 12 standards12 related to transparency
                            in government policy making and operations,13 financial sector regulation,
                            and corporate sector practices (see appendix V).14 Building on earlier
                            efforts to assess transparency, in 1999, the IMF and the Bank began
                            conducting joint assessments of observance of standards related to

                            12
                             According to the Fund, 11 of these 12 standards are internationally accepted. The World
                            Bank’s corporate insolvency and creditor rights standard is still in draft form.
                            13
                             In addition to the ROSC, the Fund launched an experimental program in 2000 to help
                            prevent possible misuse of Fund resources. This program assesses weaknesses in central
                            banks’ ability to safeguard Fund resources through internal controls, accounting, reporting,
                            auditing systems, and legal structure. In 2002, the Executive Board made assessments of
                            central banks’ safeguards a permanent policy. See appendix VI for an update on progress in
                            this area.
                            14
                              In February and March 2003, the IMF and Bank published reviews of the FSAP and ROSC
                            initiatives. See International Monetary Fund and World Bank, Financial Sector Assessment
                            Program—Review, Lessons, and Issues Going Forward (Washington, D.C.: 2003);
                            International Monetary Fund and World Bank, International Standards: Strengthening
                            Surveillance, Domestic Institutions, and International Markets (Washington, D.C.: 2003);
                            International Monetary Fund, International Standards: Background Paper on
                            Strengthening Surveillance, Domestic Institutions, and International Markets
                            (Washington, D.C.: 2003); World Bank, Reports on the Observance of Standards and Codes
                            Background Paper on Standards Review: Assessing Progress and Lessons Learned from
                            Bank-Led ROSC Modules (Washington, D.C.: 2003).




                            Page 16                                          GAO-03-734 International Financial Crises
financial sector regulation, covering areas such as banking supervision and
securities regulation. The Bank began performing assessments of
standards related to corporate sector practices, including private sector
accounting rules and corporate governance principles, in 2000. Some
transparency, financial sector, and corporate sector standards may be
assessed under the FSAP. Country participation in both initiatives is
voluntary. The Fund and the Bank initially considered making participation
in the ROSC assessments mandatory for member countries after
determining that the Fund’s Articles of Agreement could allow such a
requirement.

IMF or World Bank staff lead FSAP and ROSC assessment teams, with
participation by experts from national central banks and supervisory
agencies and standard-setting bodies such as the Basel Committee and the
International Organization of Securities Commissions (IOSCO). Before
undertaking an FSAP or ROSC, the Fund and the Bank work with country
governments to choose areas on which to focus. During the assessment
process, FSAP and ROSC teams conduct at least one in-country visit,
allowing team members to work with government officials from the finance
ministry, the central bank, and regulatory bodies to collect information for
the assessment.15 For example, an FSAP team in Russia analyzed financial
information for the largest banks and single largest corporation to
determine how changes in economic variables such as oil prices might
affect the banking system. The South Korean ROSC team interviewed
government officials at financial sector regulatory entities and private
sector representatives to determine how closely regulatory practices
conform to standards and to identify weaknesses that could put the
financial sector at risk.

The two initiatives provide information to assist four parties that play a role
in crisis prevention: the Fund, the World Bank, member country
governments, and private sector investors. In-depth information on
member countries’ financial sectors and adherence to standards of good
practice is intended to help the IMF and the Bank fulfill their missions.
Fund staff identify countries’ vulnerabilities and develop appropriate
advice to redress them; Bank staff identify long-term financial sector
development needs and formulate relevant lending and nonlending
responses. Member countries can use these assessments to help prioritize
reform agendas and win domestic support for difficult policy decisions that


15
     Appendix VII contains a diagram of the FSAP and ROSC assessment process.




Page 17                                          GAO-03-734 International Financial Crises
                                   may make their financial sectors and institutions more resistant to crisis.
                                   The Fund and the Bank often provide technical assistance to help
                                   governments build capacity to implement reforms. The financial crises of
                                   the 1990s also raised awareness of the private sector’s role in crisis
                                   prevention. Thus, the Fund and the Bank expect the assessments to help
                                   private sector participants make sounder investment decisions, thereby
                                   reducing volatility in capital markets.



Parties’ Use of Assessments        The use of FSAP and ROSC assessments in crisis prevention efforts is
Is Mixed, and Significant          mixed, and significant limitations remain. Fund staff use the assessments,
                                   when available, as inputs for the policy advice they provide to member
Challenges Remain
                                   country governments. However, the Fund lacks crucial information about
                                   vulnerabilities to financial crisis because some major emerging market
                                   countries have not participated in the assessments. World Bank staff’s use
                                   of the assessments to inform development assistance priorities is also
                                   affected by these gaps in participation and by borrower countries’
                                   competing development needs. Many member country governments face
                                   limitations in using assessments to make policy decisions because the
                                   reforms recommended in the assessments are often difficult to implement.
                                   Finally, some private sector participants find assessments of limited use
                                   because they are untimely, outdated, and dense.

The Fund Uses Assessments,         The IMF uses FSAP and ROSC assessments, when available, as inputs for
When Available, to Inform Policy   the policy advice it provides member country governments. According to
Advice                             Fund officials, these assessments highlight issues such as weak banking
                                   supervision or high levels of debt held in foreign currency that could make
                                   countries vulnerable to crisis. The assessments also provide
                                   recommendations to address these issues. The findings and
                                   recommendations inform the discussions of policy issues that Fund
                                   officials have with member country authorities during Article IV
                                   consultations.16 For example, when an FSAP was performed in Mexico,
                                   Mexican authorities had begun replacing a system where the government
                                   fully insured all bank deposits with one that covers deposits up to a certain

                                   16
                                    In accordance with Article IV of the IMF’s Articles of Agreement, the IMF holds regular
                                   consultations, normally every year, with each of its members. These consultations focus on
                                   the member’s exchange rate, fiscal, and monetary policies; its balance of payments and
                                   external debt developments; the influence of its policies on the country’s external accounts;
                                   the international and regional implications of those policies; and on the identification of
                                   potential vulnerabilities. (“IMF Surveillance: A Factsheet,” http://www.imf.org, downloaded
                                   4/23/03).




                                   Page 18                                           GAO-03-734 International Financial Crises
limit. The FSAP team was concerned because this reform was undertaken
before Mexico had developed a well-defined framework for closing
unprofitable banks. Without a clear framework for bank closures, the
introduction of limited deposit insurance could damage depositor
confidence in the banking system and precipitate a banking crisis.
According to Fund officials, the FSAP team and Mexican authorities
discussed the need to create such a framework, and a subsequent Article IV
mission reviewed the government’s progress in this area. In Poland, an
FSAP team discovered that Polish households and small businesses had
high levels of debt held in foreign currency. The team was concerned that a
depreciation of Polish currency could raise the cost of these loans and
cause widespread repayment difficulties, which could in turn lead to a
banking crisis. FSAP team members raised this issue with Polish central
bank authorities and followed up again during the next Article IV
consultation. In both countries, government officials followed the Fund’s
advice and implemented reforms. The Mexican government began
developing a framework for closing banks, and Poland’s central bank
established a team to monitor household and small business debt.

Since 1999, FSAP assessments have been conducted in more than 40
member countries and ROSC assessments in about 90 member countries.
However, we found that assessments have not been completed for some
major emerging market countries, limiting the Fund’s awareness of crisis
vulnerabilities in certain countries. Appendix VIII contains a record of
country participation in and publication of FSAP and ROSC assessments.
Fund and Bank staff encourage participation in FSAP and ROSC
assessments by countries whose economies have worldwide or regional
implications or have known vulnerabilities to a financial crisis, but officials
acknowledge that some governments have persistently resisted their
efforts. According to our analysis, between 1999 and 2003,17 45 percent of
33 major emerging market countries participated in an FSAP.18 However,
the Fund has not performed FSAPs in important countries such as China
and Thailand because their authorities have not agreed to participate.
These gaps in participation limit the Fund’s ability to develop policy advice
based on in-depth knowledge of their financial sectors.



17
     Our analysis is based on Fund and Bank data through March 2003.
18
  These 33 countries, a subset of the 87 we analyzed in the previous section, represent the
most significant emerging market participants in the international capital markets, as
identified by J.P. Morgan.




Page 19                                           GAO-03-734 International Financial Crises
According to the Fund, the Mexican and Asian financial crises were caused,
in part, by vulnerabilities in areas covered by the ROSCs. Our analysis
found gaps in participation in assessments of several key standards that the
Fund identifies as contributing factors to past crises (see figure 3). For
example, only one-third of major emerging market countries have
participated in assessments of their adherence to standards for
dissemination of economic and financial data. About half have participated
in the fiscal, monetary and financial policy formulation assessments and
the banking supervision assessment. In addition, Fund documents point to
limited progress in assessing adherence to the four World Bank-led
corporate sector standards19 (accounting, auditing, corporate governance,
and insolvency regimes), which play a key role in the effective operation of
domestic and international financial systems. Less than one-third of the 33
major emerging market countries have participated in one or more
assessments related to accounting and auditing.



Figure 3: Participation of 33 Major Emerging Markets in Key ROSC Assessments,
1999–2003




19
  The Bank performs accounting and auditing assessments simultaneously and publishes
them as a single report. An agreed-upon standard for insolvency regimes has not yet been
finalized. The Bank’s draft standard for insolvency regimes is under review by the Bank and
the Fund.




Page 20                                          GAO-03-734 International Financial Crises
                                 The Fund asserts that its delayed response in preventing or mitigating the
                                 Mexican and Asian crises was partially caused by insufficient information
                                 on these vulnerabilities. For example, according to the IMF, the Mexican
                                 government’s publicly available data was outdated and incomplete in 1993–
                                 94, which contributed to significant delays in responding to the country’s
                                 excessive indebtedness. The Fund also was unaware of some Asian
                                 countries’ unsound corporate accounting practices, which contributed to
                                 the Asian financial crisis. Continued participation gaps in these
                                 assessments suggest that the Fund still lacks crucial information about
                                 some countries’ potential vulnerability to crisis.

The World Bank’s Use of          The World Bank acknowledges the importance of FSAP and ROSC
Assessments in Formulating       assessments in formulating its financial sector development programs, but
Country Development Assistance   limited participation in corporate sector assessments (described earlier)
Programs Is Mixed                affects the Bank’s ability to respond to weaknesses in borrower countries’
                                 financial sectors. According to the Bank, country participation in corporate
                                 sector assessments has been lower than in areas related to transparency
                                 and financial sector regulation because the Bank has experienced delays in
                                 finalizing standards and methodologies for evaluating the corporate sector.
                                 For example, the methodologies for performing assessments of the
                                 accounting and auditing standards were not finalized until October 2000.

                                 Bank officials acknowledge that even when assessments are available,
                                 Bank staff do not always incorporate the issues raised as a key priority in
                                 formulating its country development assistance programs. In justifying
                                 their limited prioritization of FSAPs and ROSCs, Bank officials cited
                                 competing development demands and timing issues. First, Bank officials
                                 stated that they must balance borrower countries’ financial sector reform
                                 needs with other demands for development assistance. Most borrower
                                 country governments have multiple concerns, and Bank staff may
                                 determine that its resources will have more impact in areas other than
                                 financial sector development. Second, Bank officials cited the scheduling
                                 of the FSAP and ROSC assessments as a reason for their limited use since
                                 the timing of many assessments does not coincide with the Bank’s
                                 preparation of Country Assistance Strategies.20




                                 20
                                  The Country Assistance Strategy describes the Bank’s assessment of development
                                 priorities in each borrower country and identifies the level and composition of assistance to
                                 be provided based on this assessment. These strategies are currently prepared every 3 years.




                                 Page 21                                           GAO-03-734 International Financial Crises
Member Country Governments’        Although member country authorities sometimes use FSAP and ROSC
Use of Assessments Is Limited      assessments to inform policy decisions, reforms recommended in the
                                   assessments are often difficult to implement. Some member country
                                   governments have faced obstacles to implementing reforms, including
                                   political opposition, legal constraints, and lack of technical capacity. For
                                   example, IMF officials stated that political opposition has limited the South
                                   Korean government’s progress in eliminating extrabudgetary funds, a key
                                   recommendation of the fiscal transparency ROSC. Extrabudgetary funds
                                   diminish transparency because they are exempt from rules that require
                                   scrutiny and prioritization of expenditures for most of South Korea’s
                                   budget. Fund officials cited Peru as a case where legal constraints delayed
                                   reform efforts. The FSAP and banking supervision ROSC found that
                                   protecting bank supervisors from the political pressures of the powerful
                                   bankers’ lobby would strengthen Peru’s banking supervision. According to
                                   Fund officials, existing legislation precluded awarding supervisors greater
                                   independence, and passage of new legislation was delayed. In Russia,
                                   limited technical capacity interfered with the government’s ability to
                                   implement reform recommendations. For example, the FSAP team
                                   reviewed the government’s proposal to stimulate competition in the
                                   banking sector by introducing a deposit insurance system for household
                                   deposits. However, Fund staff noted that Russia’s bank supervisory agency
                                   lacks the capacity to implement a deposit insurance system for a large
                                   number of banks.

Private Sector Participants Find   The Fund claims that private sector participants increasingly use the
Assessments Difficult to Use       results of FSAPs and ROSCs to inform investment decisions and risk
                                   management. However, representatives of major international investment
                                   firms and ratings agencies we interviewed stated that the reports were
                                   untimely, outdated, and too dense to be useful. For example, several
                                   respondents indicated that delays in publishing ROSC assessments reduced
                                   their usefulness. Some private sector participants stated that ROSC reports
                                   and FSAP summaries, known as Financial System Stability Assessments
                                   (FSSAs), should be published within 6 months of performing the
                                   assessments. However, our analysis of the 58 ROSC reports published for
                                   major emerging market countries found that in one-third of the cases, 9
                                   months or more elapsed between assessment and publication. Several
                                   private sector participants we interviewed stated that outdated ROSC
                                   reports are unreliable for decision making. The Fund acknowledges that
                                   assessments must be current for private sector participants to use them.
                                   According to Fund data, 13 countries have published an update of at least
                                   one ROSC module. However, IMF officials estimate that, of the more than
                                   40 FSAPs performed to date, only 4 have been fully updated. Some private



                                   Page 22                                  GAO-03-734 International Financial Crises
                                sector participants also stated that FSSA and ROSC reports are not clearly
                                written. Representatives of one multinational investment bank stated that
                                the assessments are written in a way that is difficult to understand, which
                                limits the reports’ usefulness for investment decisions. While these
                                interviews were limited in number and may not be representative of all
                                private sector participants, they do provide an indication of the problems
                                these individuals may currently have in using FSAPs and ROSCs.

                                Fund and Bank outreach sessions and a 2002 Fund survey corroborated our
                                findings on private sector participants’ difficulties in using ROSC
                                assessments. The Fund reports that private sector participants place high
                                priority on timely publication and frequent updates of ROSC assessments.
                                For example, several participants observed that ROSC reports for
                                Argentina had not been updated since their publication in 1999. Moreover,
                                respondents to the Fund’s survey commented that ROSC assessments
                                should state more clearly the deficiencies in a country’s adherence to a
                                standard. In a March 2003 review of the standards initiative, the Fund and
                                the Bank concluded that ROSC reports would be more useful if they stated
                                the main findings and their significance clearly and prioritized
                                recommendations more explicitly.



Crisis Prevention Initiatives   The Fund and the World Bank acknowledge that FSAP and ROSC
Cannot Prevent all Crises       assessments cannot prevent all crises because recommended reforms may
                                require many years to be fully implemented and because crises can be
                                caused by factors outside the reforms’ scope. For example, Argentina
                                participated in four ROSC assessments in 1999 to improve economic data
                                dissemination; banking supervision; and transparency in the formulation of
                                fiscal, monetary, and financial policies. According to senior IMF officials,
                                the Argentine government followed many of the recommendations
                                generated by these assessments, but their actions did not address
                                vulnerabilities related to weak fiscal policy and a fixed exchange rate
                                regime that contributed to Argentina’s 2001 crisis. Fund officials cite
                                Turkey as another example of a country that made considerable progress in
                                improving transparency and data provision based on reforms
                                recommended by the fiscal transparency and economic data dissemination
                                ROSC assessments of 2000–2001. However, according to the Fund, these
                                reforms could not have prevented Turkey’s 2001 crisis, which originated
                                with declines in its exchange rate.




                                Page 23                                  GAO-03-734 International Financial Crises
The Fund’s Efforts to        Fund officials assert that the current process for renegotiating the terms of
                             member countries’ loans with external private sector creditors is lengthy
Better Resolve Future        and costly. In 2001, the Fund began considering the SDRM, an international
Financial Crises Face        legal framework that would allow a majority of a country’s external
                             creditors to approve a restructuring of most private sector loans. The Fund
Significant Challenges       is also encouraging members to include CACs in bonds, which would allow
                             a majority of bondholders to renegotiate the terms of that bond. Although
                             some elements of both approaches are acceptable to the private sector and
                             governments, a number of political, legal, and technical challenges stand in
                             the way of implementing the SDRM; it seems unlikely that these issues will
                             be resolved in the immediate future. While private sector officials expect
                             that many restructurings need only involve the private sector and the
                             debtor country, under some circumstances, voluntary debt restructurings
                             will not adequately resolve all financial crises. These officials stated that, in
                             those cases, the Fund should provide short-term loans to eligible countries
                             to help fill their external financing gaps. However, concerns have been
                             raised by some financial experts and government representatives that such
                             Fund loans have the potential to increase the probability of future crises.
                             In response to these concerns, the Fund has clarified and strengthened its
                             policy of lending into crisis situations.



Efforts to Resolve Future    According to the Fund, countries facing severe liquidity problems often go
Sovereign Debt Crises Face   to extraordinary lengths to avoid renegotiating or restructuring the terms
                             of their loans. They do so because, in the past, restructuring damaged the
Significant Challenges
                             economy and the banking system of participating countries. In some cases,
                             even when a voluntary restructuring process is initiated, individual
                             creditors may hold out for the best possible terms or sue in an attempt for
                             better terms. Additionally, countries believe that creditors also may be
                             unwilling to make future loans if they default on their existing debt.

                             The SDRM approach is an attempt to create a more orderly, predictable,
                             and comprehensive restructuring process and to lower the costs of
                             restructuring for both the debtor and creditors. The approach sought to
                             reduce the duration of the restructuring process from years to months and
                             to provide incentives to restructure debt before default to better protect
                             debtor and creditor interests. In the case of debtors, the Fund maintains
                             that an orderly restructuring process could reduce the likelihood of a
                             reduction in future capital flows. For creditors, it could provide more
                             favorable repayment terms from the restructured debt.




                             Page 24                                    GAO-03-734 International Financial Crises
The SDRM is a proposed international legal framework that would allow a
member country to declare its debt unsustainable and invoke a process to
restructure most of its external private sector loans.21 A specified majority
of the country’s external creditors would vote to approve the terms for
restructuring, which would bind all eligible private sector creditors. The
framework is designed to increase the incentives for the Fund’s member
countries and their creditors to reach a rapid and collaborative agreement
on restructuring unsustainable debt.

A number of political, legal, and technical challenges stand in the way of
implementing the SDRM, and it seems unlikely that these issues will be
resolved in the immediate future. According to the Fund, successful
implementation of SDRM will require overcoming certain political
constraints. The SDRM could be put into practice either by countries
adopting a new international treaty or by amending the Fund’s Articles of
Agreement. Both options would be difficult to implement since a number of
countries have indicated opposition to the SDRM. The draft framework
recommended that the SDRM be created through an amendment to the
Fund’s Articles of Agreement because the SDRM is closely related to the
role already assigned to the Fund under the Articles in the resolution of its
members’ external financial obligations. However, the Fund acknowledged
that, given the opposition of some countries, changing the Articles could be
difficult to achieve since it requires acceptance by three-fifths of the
members, having 85 percent of the total voting representation. The United
States, for example, could unilaterally veto any proposed amendment to
the Fund’s Articles given its 17 percent voting representation.




21
 The SDRM proposal has undergone several revisions. Our report discusses the proposal
presented to the International Monetary and Financial Committee in April 2003. See
http://www.imf.org/external/np/omd/2003/040803.htm.




Page 25                                        GAO-03-734 International Financial Crises
                             A key legal challenge to the implementation of SDRM is the need for most
                             countries to change their domestic laws to conform to the requirements of
                             any new Fund articles. Before a member country can vote to accept an
                             amendment, it must take all the necessary steps needed under its own
                             domestic law to ensure that the amendment will be given full force and
                             effect under its domestic law. However, some Fund members have raised
                             concerns over whether the domestic legal systems of some member
                             countries could accommodate a new legal framework that applied to
                             preexisting claims. The proposed SDRM approach also faces technical
                             challenges. For example, the proposed framework does not specify how
                             the claims of official bilateral creditors and some guaranteed domestic
                             debts would be treated. The Fund is consulting with the Paris Club on how
                             the Club’s practices may be modified to better facilitate coordination
                             between official bilateral and the private creditors in a debt restructuring
                             process.22



The Fund Has Encouraged      CACs are terms in individual bonds that permit a specified majority of
the Adoption of Collective   sovereign bondholders to agree on a debt restructuring that would bind all
                             holders of that bond. In June 2002, the Fund’s Executive Board endorsed
Action Clauses               the use of certain CAC provisions in new bonds and agreed to encourage
                             member countries to incorporate CACs into their sovereign bonds in future
                             restructurings. Inclusion of these clauses into new bonds would be
                             voluntary.




                             22
                              The Paris Club is an informal group of official creditors whose role is to find solutions to
                             the payment difficulties experienced by debtor countries. Paris Club creditors agree to
                             provide a country with debt relief through a postponement and, in the case of concessional
                             rescheduling, a reduction in debt service obligations.




                             Page 26                                           GAO-03-734 International Financial Crises
The Fund views CACs and SDRM as complementary instruments in
resolving future financial crises. According to the Fund, the existence of
CACs in certain bond agreements inspired the development of the SDRM
framework. Although the Fund has not created its own CAC framework, it
has endorsed the use of two key features from a G-10 Working Group
Report23 and an Industry Associations draft proposal.24 These features
include the following:

• Majority restructuring provision25 enables a qualified majority of
  bondholders to bind all holders of a particular bond to the financial
  terms of a restructuring, both before and after a default. Although
  majority restructuring provisions have generally been included in bonds
  governed by the laws of the United Kingdom or Japan, they have not
  been included in bonds governed by the laws of the United States or
  Germany.

• Majority enforcement provisions prevent a minority of creditors from
  pursuing disruptive legal action after a default and before reaching a
  restructuring agreement. Many international sovereign bonds governed
  by both U.S. and English law contain these provisions. Specifically, the
  Fund supports the requirement that (1) an affirmative vote of a
  minimum percentage of bondholders is necessary to approve claims
  following a default and (2) a specified majority of bondholders can
  reverse an approval of a claim that has already occurred.

In early 2003, Mexico became the first emerging market country to issue a
public, SEC-registered global bond with CACs under New York law.
Previous issues under New York law by Lebanon, Qatar, and Egypt had
been placed privately to institutional investors and included only a limited


23
 The G-10 Working Group on Contractual Clauses (the Working Group) was formed in June
2002 at the behest of ministers and governors. The mandate of the Working Group was to
consider how sovereign debt contracts could be modified to resolve debt crises in a more
orderly fashion.
24
  The proposal was put forward to Fund members for consideration by the Institute of
International Finance and six other financial industry trade associations. The six financial
industry associations are the Emerging Market Traders Association (EMTA), the
International Primary Market Association (IPMA), the Bond Market Association (BMA), the
Securities Industry Association (SIA), the International Securities Market Association
(ISMA), and the Emerging Markets Creditors Association (EMCA).
25
 These provisions are referred to in the G-10 Working Group Report as “majority
amendment provisions.”




Page 27                                          GAO-03-734 International Financial Crises
                             range of CACs. For example, the bonds issued by Egypt and Qatar
                             included a very limited form of majority enforcement provisions, while
                             Lebanese bonds did not contain them at all. Since Mexico’s successful
                             issue, Brazil, South Africa, and Korea have issued bonds with CACs.
                             Uruguay included CACs in the bonds resulting from its debt exchange. The
                             details of the Brazil, South Africa, Korea, and Uruguay bond provisions
                             were not available at the time we conducted our review.

                             Some countries criticize CACs because they would only apply to new bond
                             offerings and not existing bonds. Accordingly, during a restructuring of a
                             country’s bond obligations, not all creditors would be bound by the CAC
                             provisions. Borrowing countries also contend that inclusion of CACs in
                             bond offerings could suggest to creditors that countries anticipate having
                             difficulty repaying their loans. In response, creditors may charge a higher
                             interest rate. However, a May 2000 academic study26 compared interest
                             rates on bonds issued in the United States (where CACs are not used) with
                             the United Kingdom (where CACs are used) and found that CACs do not
                             contribute to higher rates in United Kingdom bonds.



While Private Sector         To date, officials from the private sector, including lenders, have expressed
Officials Prefer Voluntary   preference for continuing the current voluntary process, which only
                             involves the private sector and borrowing countries, in the efforts to
Debt Restructuring, They     restructure sovereign debts. Many private sector officials we interviewed
Expect Continued Fund        oppose the proposed SDRM approach and the Fund’s attempts to integrate
Loans in Exceptional         CACs into new bond issues, partly because they would interfere with the
Circumstances                normal bargaining process. They maintain that a voluntary approach to the
                             restructurings that took place from 1998 to 2001 in Ecuador, Pakistan,
                             Russia, and Ukraine were successful.27 Private sector officials assert that
                             these and other experiences have worked well enough, and that a
                             substantive change in current market practice is unnecessary. In contrast
                             to the Fund’s assertion that new approaches are needed to make
                             restructurings shorter and less expensive, private sector officials note that


                             26
                              See Eichengreen, Barry and Ashoka Mody. “Would Collective Action Clauses Raise
                             Borrowing Costs?” Working paper, Center for International and Development Economics
                             Research, Berkeley, CA, 2000.
                             27
                              The restructurings in Russia involved domestic debt and Soviet-era foreign debt owed to
                             commercial banks. In 1998, Pakistan froze withdrawals in foreign currency from all
                             nonresident foreign currency deposits and subsequently reached restructuring agreements
                             with these nonresident investors.




                             Page 28                                         GAO-03-734 International Financial Crises
                             most recent voluntary restructurings successfully concluded in 1 year or
                             less and that creditor holdouts or litigation did not significantly delay the
                             restructurings.28

                             While private sector officials expect that many restructurings need only
                             involve the private sector and the debtor country, under some
                             circumstances, voluntary debt restructurings will not adequately resolve all
                             financial crises. In those cases, they said the Fund should provide loans to
                             eligible countries to help fill their external financing gaps. Such loans
                             would assist the restructuring process and facilitate efforts at
                             implementing necessary reforms. However, large Fund loans, such as those
                             given during the Asian financial crises, have received substantial criticism
                             from financial experts and government representatives, including U.S.
                             government officials. One concern is that the possibility of receiving
                             substantial financial assistance provides an incentive for debtor countries
                             to adopt unsustainable economic policies to forestall needed reform.
                             Another concern is that these large loans may encourage private sector
                             creditors to continue providing large capital flows to countries with
                             unsustainable economic policies because these otherwise risky
                             investments have the potential of being “bailed out” by future Fund loans.
                             This condition is referred to as “moral hazard.” According to these critics,
                             efforts to help resolve existing financial crises through large Fund loans
                             may increase the probability of future crises due to these two concerns.
                             The Fund has advocated the SDRM framework and CACs to replace the
                             current voluntary approach, partially in response to concerns over the
                             potential adverse affects of its lending.



The Fund Has Clarified and   To reduce the risk that Fund loans would increase the probability of future
Strengthened its Lending     financial crises, the Fund clarified and strengthened its policy of lending in
                             crisis situations. The Fund has clarified elements of its Lending into
Practices to Address
                             Arrears Policy and strengthened its criteria for requesting large short-term
Concerns Over Exceptional    loans under the Supplemental Reserve Facility (SRF). Since 1997, nine
Lending                      countries have received loans under the two mechanisms.

                             The Fund’s Lending into Arrears Policy permits the IMF to provide
                             resources to countries that are unable to repay their external creditors and
                             are thus considered in default. Conceived in the late 1980s and amended in

                             28
                              For some countries, the negotiating process entailed several years and more than one
                             restructuring agreement.




                             Page 29                                         GAO-03-734 International Financial Crises
             the late 1990s, the policy is designed to protect the value of creditor assets
             while providing creditors with incentives to enter rapidly into restructuring
             negotiations with countries. The Lending into Arrears Policy increases the
             likelihood that a country’s private sector lenders would agree to reduce the
             value of their loans because Fund resources reduce short-term fiscal
             pressures experienced by the country while in default. A country is eligible
             for Fund resources while in default if the Fund determines that the debt
             burden is unsustainable, and the country is making satisfactory progress in
             implementing reforms. Additionally, the country must have demonstrated a
             good faith effort to reach a restructuring agreement with creditors to
             restore its ability to repay its debt. In 2002, the Fund clarified the criteria to
             be used to determine whether the debtor country is making a good faith
             effort. For example, the Fund would consider how quickly the debtor
             engaged in negotiations with its creditors after it defaulted. To date, the
             Fund has lent into arrears on international bonds on four occasions—
             Ukraine, Ecuador, Moldova, and Argentina.

             Introduced in 1997, the SRF provides large short-term loans to members
             experiencing exceptional balance of payments difficulties prior to a
             default. The interest rates on these loans are much higher than standard
             Fund loans. The increased cost of these loans is expected to reduce the
             probability that countries consider Fund resources a viable means for
             underwriting unsustainable economic policies. Higher loan terms also
             increase incentives for early repayment and compensate for additional
             repayment risks to the Fund. Countries are expected to repay SRF loans
             within 2 to 2½ years, but they may request extensions of up to 6 months. All
             SRF loans carry a substantial surcharge of 3–5 percentage points. In 2003,
             the Fund strengthened its criteria for providing large short-term loans
             under the SRF. For example, countries requesting SRF loans must provide a
             more extensive justification for their repayment difficulties. Additionally,
             the member has to demonstrate good prospects of regaining access to
             private capital markets within the time period that Fund resources are
             outstanding to minimize long-term reliance on Fund resources. To date, the
             Fund has provided SRF loans on nine occasions to six countries, including
             Korea, Russia, Brazil, Turkey, Argentina, and Uruguay (see appendix IX).



Conclusion   In accordance with its goal of strengthening the international financial
             system, the Fund has undertaken a number of reforms to better anticipate,
             prevent, and resolve sovereign financial crises. The Fund’s new
             vulnerability assessment process is more comprehensive than its previous
             crisis anticipation efforts, but it is too soon to judge its effectiveness. The



             Page 30                                     GAO-03-734 International Financial Crises
                      Fund’s proposed approaches to better resolve financial crises have met
                      considerable resistance, and it is unclear whether they will ultimately be
                      adopted. The Fund and the Bank have made progress in their crisis
                      prevention efforts by performing assessments of member countries’
                      financial sectors and adherence to standards. However, the effectiveness of
                      these crisis prevention efforts is hindered by (1) private sector participants’
                      limited use of published assessments, which they find untimely, outdated,
                      and too dense to be useful and (2) gaps in crucial information about crisis
                      vulnerabilities in some important emerging market countries due to
                      voluntary participation in the assessments. These limitations prevent
                      multilateral institutions, national policy makers, and private sector
                      participants from making sound decisions, thus reducing the likelihood
                      that these reforms will help prevent crises.



Recommendations for   To help strengthen the Fund’s crisis prevention initiatives we recommend
                      that the Secretary of the Treasury instruct the U.S. Executive Director of
Executive Action      the Fund to work with other Executive Board members to encourage the
                      Fund to

                      • improve the timeliness of publication of Financial System Stability
                        Assessments and Reports on the Observance of Standards and Codes;

                      • expand the coverage, frequency, and publication of reports on member
                        countries’ progress on implementing assessment recommendations;

                      • improve the assessment reports’ readability, for example, by creating a
                        standardized format in which to present executive summaries and key
                        findings; and

                      • pursue strategies for increasing participation in the Financial Sector
                        Assessment Program and all modules of the Reports on the Observance
                        of Standards and Codes, including the possibility of making
                        participation mandatory for all members of the IMF.



Agency Comments and   We received written comments on this report from the Department of the
                      Treasury, the International Monetary Fund, and the World Bank. These
Our Evaluation        comments and GAO’s evaluation of them are reprinted in appendixes X, XI,
                      and XII. The organizations also separately provided technical comments




                      Page 31                                   GAO-03-734 International Financial Crises
that GAO discussed with relevant officials and included in the text of the
report where appropriate.

Treasury agreed with the report’s recommendations. Treasury recognized
that some important countries have not volunteered to participate in the
FSAP and ROSC and that there should be a shorter turnaround between the
completion of an assessment and its public release. Treasury also pointed
out that the acceptance of collective action clauses in some recent bond
offerings serves as an important signal to investors that official financing is
limited and that they cannot expect to be protected from risks.

The IMF broadly agreed with the report’s recommendations. However, the
IMF stated that we mischaracterized the role of the WEO forecasts and
EWS models in IMF crisis anticipation efforts by saying that they have a
greater importance than is warranted. We disagree with this depiction. Our
assessment examined all six components of the IMF’s vulnerability
assessment framework, including the WEO and the EWS. As the only
mature and quantifiable elements of the framework, our analysis focused
more heavily on the track records of the WEO and EWS. The IMF also
stated that its responsibility to maintain financial stability could make its
predictions less accurate so as not to contribute to a crisis. The IMF’s
comment not only validates our finding on the WEO’s weakness but also
raises questions regarding the purpose and credibility of the WEO
forecasts.

The World Bank generally agreed with the report’s recommendations.
However, the Bank expressed concern with the report’s suggestion that
consideration be given to making participation in the FSAP and ROSC
assessments mandatory. While we are not suggesting that the assessments
should be made mandatory, the voluntary nature of the FSAP and ROSC
has posed an obstacle to full participation by important emerging market
countries.


We are sending copies of this report to the Secretary of the Treasury, the
International Monetary Fund, the World Bank, and interested congressional
committees. We also will make copies available to other interested parties
upon request. In addition, the report will be available at no charge on the
GAO Web site at http://www.gao.gov.




Page 32                                   GAO-03-734 International Financial Crises
If you or your staff have any questions regarding this report, please call me
at (202) 512-8979. Other GAO contacts and staff are acknowledged in
appendix XIII.




Joseph A. Christoff, Director
International Affairs and Trade




Page 33                                  GAO-03-734 International Financial Crises
Appendix I

Objectives, Scope, and Methodology                                                              A
                                                                                                A
                                                                                                ppep
                                                                                                   nen
                                                                                                     d
                                                                                                     xIeis




              The Chairman of the House Financial Services and the Vice Chairman of
              the Joint Economic Committee requested that we assess (1) the
              International Monetary Fund’s (IMF’s or the Fund’s) framework for
              anticipating financial crises, (2) the status of key IMF initiatives to prevent
              financial crises, and (3) new IMF proposals to resolve future financial
              crises. They requested that our review cover the period after the Mexican
              financial crisis of 1994–95.

              To assess IMF’s framework for anticipating financial crises, we examined
              prior and new IMF mechanisms for anticipating crises. Our analysis
              focused on the World Economic Outlook (WEO) forecasts and the IMF’s
              Early Warning System (EWS) models that were the IMF’s primary
              forecasting tools prior to the implementation of the new vulnerability
              assessment framework in May 2001. Data on the performance of the other
              four components of the framework were not made available to us because
              these elements are considered part of the staff level deliberative process,
              and are not provided to the Executive Board. We obtained near-term data
              from the WEO forecasts, including real gross domestic product (GDP)
              growth rate and current account balance for 87 emerging market countries
              for the period 1990–2001 (see appendix III). We focused on the 81 middle-
              income countries and an additional 6 low-income countries listed by J.P.
              Morgan as emerging markets. To evaluate the WEO and program forecasts,
              we used standard econometric techniques based on methods commonly
              found in the forecasting literature. The formal methodology of our forecast
              evaluations was based on several expert publications, the replication of our
              summary statistics with another author’s results, and discussions with a
              forecasting expert. To describe the performance of the IMF’s EWS models
              in anticipating crises, we reviewed and summarized the results of IMF
              evaluations. We interviewed IMF staff, including country desk economists
              and staff from several departments, to discuss the IMF’s new framework
              for vulnerability assessment, the EWS models, and the WEO methodology.
              We also interviewed 23 major private sector emerging market participants
              to discuss whether and how they use IMF forecasts in their investment
              decisions.

              To assess the status of key IMF initiatives to prevent financial crises, we
              reviewed Fund and World Bank documents published between 1999 and
              2003 on the creation of the Financial Sector Assessment Program (FSAP)
              and Reports on the Observance of Standards and Codes (ROSC) and
              evaluations of progress in implementing these reforms. We interviewed
              senior Fund officials, including staff from the Monetary Affairs and
              Exchange Department, the Policy Development and Review Department,



              Page 34                                   GAO-03-734 International Financial Crises
Appendix I
Objectives, Scope, and Methodology




and the Fiscal Affairs Department. We also met with senior advisers at the
World Bank (the Bank) who oversee the Bank’s participation in the FSAP
and ROSC initiatives. To gain a better understanding of how the Fund uses
FSAP and ROSC assessments to inform the policy advice it provides to
member country governments and challenges it faces in using these
assessments, we interviewed Fund officials in nine area departments
(Argentina, Brazil, Korea, Mexico, Peru, Poland, Russia, Turkey, and
Uruguay). We also spoke with Fund officials in these area departments
about member country governments’ use of the assessments in shaping
their reform agendas and the obstacles that member country authorities
encounter in implementing the reforms recommended in the assessments.
To assess the extent to which emerging market countries have participated
in and authorized publication of FSAP and ROSC assessments, we
examined Fund and World Bank data on country participation in the FSAP
and 12 ROSC modules and publication of the resulting reports between
May 1999 and March 2003. To obtain views on the private sector’s use of
Fund and World Bank assessments, we conducted structured interviews
with 13 representatives of private sector firms, including ratings agencies,
investment banks, and pension funds. We focused on 33 countries (a subset
of the 87 we analyzed in the previous section) identified as the major
emerging market countries by J.P. Morgan.1

To describe new proposals to resolve future financial crises and their
potential challenges, we obtained the most current Fund documentation
for the two key proposals, the Sovereign Debt Restructuring Mechanism
(SDRM) and Collective Action Clauses (CACs). We examined the purpose,
goals, requirements, and status of implementation. To obtain views on the
private sector’s understanding of the components of the new proposals and
potential implementation challenges, we conducted structured interviews
with 22 representatives of private sector firms, including ratings agencies,
investment banks, and pension funds. We also met with government,
private sector emerging market participants, and nongovernmental
officials at several conferences. We also interviewed Department of the
Treasury officials and experts in international finance and law. The IMF did
not meet with us on these proposals because they were still under
negotiation at the time of our review.




1
 These 33 countries represent the most significant emerging market participants in the
international capital markets, as identified by J.P. Morgan.




Page 35                                          GAO-03-734 International Financial Crises
Appendix I
Objectives, Scope, and Methodology




We conducted fieldwork in Washington, D.C., and in New York. We
performed our work from May 2002 to May 2003 in accordance with
generally accepted government auditing standards.




Page 36                              GAO-03-734 International Financial Crises
Appendix II

Private Sector Structured Interview
Participants                                                                                                     AppenIx
                                                                                                                       di




                                                      Perspectives obtained on IMF efforts in crisis
Company                                               Anticipation            Prevention               Resolution
Citicorp                                                        x                        x                      x
Cleary Gottlieb & Hamilton                                      x                                               x
Credit Suisse                                                   x                        x                      x
Darby’s Overseas Investments                                    x                        x                      x
Elliot Associates                                               x                                               x
Emerging Market Traders Association                             x                                               x
eStandards Forum                                                x                        x                      x
Eurasia Group                                                   x                        x                      x
Fitch Ratings                                                   x                        x                      x
HBK                                                             x                                               x
HSBC                                                            x                        x                      x
International Primary Markets Association                       x                                               x
Institute for International Finance                             x                                               x
Japan Center for International Finance                          x                                               x
J.P. Morgan                                                     x                                               x
Lehman Brothers                                                 x                        x                      x
Mass Mutual                                                     x                                               x
Moody’s Investor Service                                        x                        x                      x
Securities Industry Association                                 x                       x                       x
Societe Generale                                                x                        x                      x
Standard & Poor’s                                               x                        x                      x
Straus & Boies                                                  x                                               x
Wilshire Associates                                             x                        x
Source: GAO.




                                            Page 37                      GAO-03-734 International Financial Crises
Appendix III

Assessment of IMF Forecasting                                                                                 Appen
                                                                                                                  Ix
                                                                                                                   di




                       Congress expressed concerns regarding the accuracy of the International
                       Monetary Fund’s (IMF’s) growth rate projections and asked us to examine
                       them. In response, we analyzed the quality of the forecasts produced by the
                       World Economic Outlook (WEO), the Fund’s primary forecasting tool.
                       Using econometric techniques common to forecast evaluation studies, we
                       examined the basic measures of forecast accuracy, bias, and efficiency.
                       This assessment supplements our finding on WEO’s efforts to anticipate
                       crises reported earlier. We found that WEO forecasts of gross domestic
                       product (GDP) growth and inflation perform somewhat better than an
                       assumption that next year’s rate will be the same as this year’s (called a
                       “naive” forecast). However, there is evidence of an optimistic bias in the
                       forecasts of GDP growth and inflation. In addition, we found that the naive
                       forecast of the current account generally performed better than the WEO
                       forecast. Moreover, WEO forecasts for the major industrialized countries
                       were superior to emerging market forecasts, and forecasts for emerging
                       market countries that had been on an IMF program were better than for
                       those countries that were not. The shortcomings we observed in WEO
                       forecasting are similar to those encountered by other private sector and
                       official forecasters.



GAO Analysis Used      We evaluated IMF forecasts for 87 emerging market countries. Our analysis
                       focused on WEO forecasts of GDP, inflation, and the current account. Our
Standard Econometric   measures of forecast quality relied on generally accepted econometric
Techniques for         measures of accuracy, bias, and efficiency.
Forecast Evaluation

Overall Approach       To evaluate the quality of IMF forecasts, we analyzed the near-term and
                       year-ahead WEO forecasts for 87 middle-income emerging market
                       countries.1 Appendix IV lists the 87 emerging market countries used in the
                       analysis. Our analysis focused on three WEO forecast variables: (1) the
                       growth rate of real GDP, (2) consumer price index (CPI) inflation (average
                       over period), and (3) current account balance in billions of U.S. dollars. Our
                       evaluation methodology is based on standard econometric techniques
                       commonly found in the forecast literature, including the work of



                       1
                        Our analysis focused on 81 middle-income countries and an additional 6 low-income
                       countries listed by J.P. Morgan as emerging markets.




                       Page 38                                        GAO-03-734 International Financial Crises
                                   Appendix III
                                   Assessment of IMF Forecasting




                                   forecasting experts such as Stekler (1991), Artis (1996), and Loungani
                                   (2001).2

                                   We also compared the quality of WEO forecasts of emerging market
                                   countries with WEO forecasts of the G-7 countries and we compared
                                   borrowers of IMF resources with those that were not.3 Our comparison
                                   with the G-7 countries4 allowed us to informally assess whether income
                                   level or data quality mattered in forecast quality. Our analysis of forecasts
                                   for program countries permitted us to assess whether WEO forecasts differ
                                   from forecasts contained within program documents, which are produced
                                   under conditions of greater staff scrutiny.5 We also reviewed a number of
                                   forecast evaluation studies to see how our results compared to previous
                                   reviews and to contrast IMF forecast quality with other forecasting efforts.

Basic Definitions and Methods of   Our analysis focused on the WEO’s near-term and year-ahead forecasts.
Evaluation                         Near-term forecasts originate from the May WEO for each year, and they
                                   project for the remainder of the existing year (approximately 6 months
                                   ahead). The year-ahead forecasts come from the October WEO of the
                                   preceding year. Thus, a near-term forecast for 2000 would come from the
                                   May 2000 WEO, and a year-ahead forecast for 2000 would come from the
                                   October 1999 WEO. We compared these WEO forecasts to the “first settled
                                   estimate,” which comes from the October WEO of the following year for
                                   which the forecast is made. Thus, we compared both the near-term and




                                   2
                                    Herman Stekler, “Macroeconomic Forecast Evaluation Techniques,” International Journal
                                   of Forecasting 7 (1991): 375-384; Michael J. Artis, “How Accurate are the IMF’s Short-Term
                                   Forecasts? Another Examination of the World Economic Outlook,” WP/96/89 (Washington,
                                   D.C.: International Monetary Fund [IMF], August 1996); Prakash Loungani, “How Accurate
                                   are Private Sector Forecasts? Cross-Country Evidence from Consensus Forecasts of Output
                                   Growth,” International Journal of Forecasting 17 (2001): 419-432.
                                   3
                                    We include the following General Resources Account-funded IMF programs: Stand-By
                                   Arrangements (SBA), Extended Fund Facility (EFF), Systemic Transformation Facility
                                   (STF), Compensatory Contingency Finance Facility (CCFF). Under the CCFF, countries can
                                   borrow resources on a stand-alone basis, i.e., outside of an IMF program. Macedonia and
                                   South Africa borrowed funds under the CCFF.
                                   4
                                   The G-7 include Canada, France, Germany, Italy, Japan, United Kingdom, and United States.
                                   5
                                    Of the 87 emerging market countries analyzed, 57 were on an IMF program for some part of
                                   the 1990–2001 period.




                                   Page 39                                         GAO-03-734 International Financial Crises
           Appendix III
           Assessment of IMF Forecasting




           year-ahead forecasts for 2000 with the “first settled estimate” from the
           October 2001 WEO.6

           Most of the econometric tools we used to assess the quality of WEO
           forecasts analyze the errors deriving from the forecasts. Our econometric
           tools examined these errors for certain qualities and patterns. We defined
           the forecast error, et , as the difference between the forecasted, f t , and
           realized, a t , value of an indicator. Hence, we have




           Our examination of the errors in the Fund’s forecasts focused on three
           measures of “goodness”: accuracy, bias, and efficiency. We performed
           these tests separately for the 87 countries over the 11-year forecasting
           period.

Accuracy   The credibility of a forecast is established by its accuracy. The ultimate test
           of any forecast is whether it can predict future events accurately. Accuracy
           assesses whether forecasts tend, by some standard, to be close to actual
           outcomes. Although there is no objective standard of accuracy,
           comparisons to alternative forecasting methods, such as a naive model that
           uses historical trend data, is one way to judge relative accuracy. The
           accuracy measure we used is Theil’s U-statistic (U1), based on the naive
           model that assumes this year’s growth rate will be the same as last year’s.7

           The Theil U-statistic is based on an examination of the forecast’s root mean
           square error (RMSE). To compute RMSE, the forecast errors are squared
           and averaged over the sample to get the mean square error (MSE). RMSE is
           the square root of MSE.




           6
            A persistent issue in the forecasting literature concerns which “actual” value to use to
           evaluate the accuracy of the forecasts: the first available estimate (available in May of the
           following year), the first settled estimate (available in October of the following year), or a
           later revision. We have taken the middle ground, as suggested by Loungani (2001), on the
           basis that forecasters are not attempting to predict later revisions, which incorporate
           information such as revisions of weights and changes in methods of construction that
           forecasters would not have been aware of at the time of the forecast.
           7
           That is, the naive model assumes there is no change in the growth rate between t-1 and t.




           Page 40                                             GAO-03-734 International Financial Crises
       Appendix III
       Assessment of IMF Forecasting




       The Theil U-statistic allows us to scale the RMSE by the variability of the
       underlying data. The standard Theil statistic, commonly denoted as U 1 ,
       compares the RMSE of forecast series to the RMSE of the actual series,




       where ∆at = a t - a t −1 . A U 1 statistic greater than one denotes that the
       naive model performed better than the model being tested.

Bias   Bias determines whether forecast errors in one direction tend to be larger
       and/or more numerous than errors in the opposite direction. Forecast
       errors can be divided into two parts. One part is the “random error,” which
       varies unsystematically, or randomly, from one forecast to the next. The
       other part is the “bias error,” which remains constant for any particular
       forecasting procedure. Bias happens when factors other than random
       events influence the forecast results, resulting in an upward or downward
       tendency. An unbiased forecast means that forecast errors are
       approximately zero on average over time. However, an unbiased forecast
       does not guarantee that a forecast will be accurate enough to be useful if
       the errors are large.

       To assess the extent of bias in the forecasts, we regress the forecast error
       on a constant term and then carry out a t-test for the coefficient. For a time
       series of forecasts (1990–2001 in our analysis), we have the set of forecast
       errors {e1 , e2 ,K, eN }. The regression test for bias involves estimating the
       intercept coefficient, β 0 , for the simple regression




       Page 41                                    GAO-03-734 International Financial Crises
             Appendix III
             Assessment of IMF Forecasting




             This reflects a partition of the forecast error into an estimate of the
             systematic component and a random component ( ε t ). A t-test with N − 1
             degrees of freedom is then performed to test H 0 : β 0 = 0 versus the two-
             sided alternative H A : β 0 ≠ 0. We perform a t-test to determine whether
             the average bias is significantly different from zero. If the p-value is less
             than .05, we reject the hypothesis that there is no bias at the 95 percent
             level of significance. This means that there is less than a 5 percent chance
             that we are making a false rejection, that is saying the forecast is biased
             when it is not. A determination that a forecast is unbiased is a necessary,
             although not sufficient, condition for concluding that a forecast is efficient.

Efficiency   Efficiency examines whether a forecast has taken into account all available
             information. Establishing that a forecast is efficient means that no other
             model or readily available information would be able to improve the
             forecast, and there is no way to predict the direction or size of the errors. A
             test of efficiency makes use of the simple linear model where we regress
             the actual outcome on the forecast




             If the forecast is efficient in predicting the actual outcome, then the
             intercept, β 0 , should be equal to 0 and the slope, β , should be equal to 1.8
                                                                    1
             Using the regression model defined above, we perform a joint hypothesis
             test to check whether these conditions hold simultaneously.




             8
              A zero intercept implies that the errors are randomly distributed; they vary
             unsystematically (unbiased). However, a forecast can be unbiased and not efficient. A slope
             of 1 implies a straight line through the zero intercept, denoting efficiency. A slope of 1
             indicates that the forecast and actual value essentially coincide.




             Page 42                                          GAO-03-734 International Financial Crises
                                                    Appendix III
                                                    Assessment of IMF Forecasting




                                                    The algebraically simplified version of the test statistic for this hypothesis
                                                    is shown below. 9




                                                    The reference distribution for this statistic is an F-distribution with 2 and
                                                     N − 2 degrees of freedom. If the p-value for this statistic is less than .05,
                                                    then we reject the hypothesis at the 95 percent level of significance that
                                                    both the intercept is zero and the slope is one. This means that there is only
                                                    a 5 percent chance that we are making a false rejection, that is saying the
                                                    forecast is not efficient when it is.



WEO Forecasts                                       Our analysis of the WEO forecasts for 87 emerging market countries shows
Demonstrate Some                                    that WEO forecasts perform somewhat better than a naive model for GDP
                                                    growth and inflation, but not for current account (see table 2).
Accuracy, but Also
Optimistic Bias


Table 2: Forecast Quality for 87 Emerging Market Countries, 1990–2001

                                     Accuracya                      Biasb                                                                    Efficiencyd
                                     (percent)                    (percent)                        Direction of biasc                         (percent)
                                     Year-       Near-              Year-       Near-                  Year-                  Near-          Year-          Near-
Forecast variable                   ahead         term             ahead         term                 ahead                    term         ahead            term
GDP growth rate                        62          74                   21          15               Upward                 Upward               76              85
Inflation                              55          70                   21          13           Downward              Downwarde                 72              67
                                                                                                             e
Current account                        24          56                    8            8             Upward          Indeterminate                67              79
(billions of U.S. $)
Source: GAO analysis of IMF data.
                                                    a
                                                     The percentage of countries in which the WEO forecast does a better job than the naive forecast
                                                    (based on the Theil statistic).
                                                    b
                                                        The percentage of countries in which there is statistically significant bias (at the 5 percent level).
                                                    c
                                                     When bias occurs in 15 percent or more countries and the forecast errors tend to vary predominantly
                                                    in one direction (more than 70 percent), then we indicate the direction of this bias.


                                                    9
                                                        RSS is the residual sum of squares.




                                                    Page 43                                                      GAO-03-734 International Financial Crises
                               Appendix III
                               Assessment of IMF Forecasting




                               d
                                Percentage of countries where we fail to reject the joint hypothesis that the intercept is zero and the
                               slope is one; i.e., the evidence is not strong enough to reject the assumption of efficiency.
                               e
                               Italics indicate that there are too few countries to consider bias a systematic trend.


                               We found the year-ahead WEO forecast does a better job than the naive
                               forecast in more than 60 percent of the countries with GDP, in more than
                               half the countries with inflation, and in about one-quarter of the countries
                               with the current account. However, even for GDP, nearly 40 percent of the
                               country forecasts were no better than an assumption that next year’s value
                               is the same as this year’s. For all three variables, the shorter the forecast
                               period, the more accurate the forecast. When the forecast time horizon
                               shortens from 1 year to 6 months, the percentage of cases in which the
                               WEO does a better job forecasting than the naive model increases for all
                               three variables, exceeding 50 percent for the current account.

                               WEO forecasts for GDP and inflation demonstrated bias in about 20
                               percent of the country cases. The direction of the bias was upward for GDP
                               and downward for inflation, indicating an optimistic tendency within the
                               WEO forecasting process.10 Although the bias was upward for the current
                               account, also consistent with optimism, it occurred in only 8 percent of
                               country forecasts.11 For all three variables, we could not reject the
                               hypothesis that WEO forecasts were efficient for at least two-thirds of the
                               country’s forecasts. However, in about one-fourth of the country cases, the
                               WEO forecast could have been improved through the use of a different
                               model or the addition of new information.



Forecasts for Major            WEO forecasts of the most developed countries are superior to its
Industrialized Countries Are   forecasts of emerging market countries when compared to the naive model
                               forecasts. The improved forecast quality is likely due to better available
Better than Emerging           data and greater stability of the wealthiest economies. Similarly, WEO
Market Forecasts and           forecasts for countries that borrow from the IMF are superior to those that
Program Forecasts Are          do not. The increased scrutiny of borrowing countries by IMF staff likely
Better than Nonprogram         contributes to the improved forecasts.
Forecasts


                               10
                                Our finding of an optimistic (upward) bias for GDP forecasts is consistent with, and helps
                               explain, our finding in the main report of the WEO’s difficulty in forecasting recessions.
                               11
                                An optimistic bias in the current account means that the forecast was for a smaller current
                               deficit or a larger surplus than occurred.




                               Page 44                                                  GAO-03-734 International Financial Crises
                                                      Appendix III
                                                      Assessment of IMF Forecasting




WEO Forecasts for Major                               WEO forecasts of GDP and inflation for the G-7 countries are considerably
Industrialized Countries Are                          better than its forecasts for the emerging market countries when compared
Better than Those for Emerging                        to the naive model forecasts. (see table 3).
Markets




Table 3: Forecast Quality for G-7 Countries, 1990-2001

                                      Accuracya                        Biasb
                                     (number of                     (number of                       Direction of bias                    Efficiencyc
                                      countries)                     countries)                   (number of countries)               (number of countries)
                                     Year-         Near-               Year-        Near- Year-                  Near-                       Year-             Near-
Forecast variable                   ahead           term              ahead          term ahead                  term                       ahead               term
GDP growth rate                         6                 7                  2           0 Upwardd               N/A                               5              7
Inflation                               6                 7                  0           0 N/A                   N/A                               7              7
Current account                         1                 5                  1           0 Downwardd             N/A                               5              7
(billions of U.S. $)
                                                      Legend
                                                      N/A = not applicable
Source: GAO analysis of IMF data.

                                                      Note: Number of countries, and not percentages, are reported given the small number of cases.
                                                      a
                                                       The number of countries for which the WEO forecast does a better job than the naive forecast (based
                                                      on the Theil statistic).
                                                      b
                                                          The number of countries in which there is statistically significant bias (at the 5 percent level).
                                                      c
                                                       The number of countries where we fail to reject the joint hypothesis that the intercept is zero and the
                                                      slope is one, i.e., the evidence is not strong enough to reject the assumption of efficiency.
                                                      d
                                                          Italics indicate that there are too few countries to consider bias a systematic trend.


                                                      This improvement is evident across the full range of analyses. For example,
                                                      in six out of seven countries, the year-ahead WEO forecasts of GDP and
                                                      inflation for the G-7 countries were found to be accurate, and the near-term
                                                      forecasts for GDP and inflation were accurate for all of the G-7 countries.
                                                      These results are considerably better than WEO forecasts for emerging
                                                      markets. In the year-ahead forecasts, bias and efficiency were a concern in
                                                      two forecasts of GDP, and one current account forecast, but they were not
                                                      a concern in the inflation forecasts. Although the year-ahead forecast for
                                                      current account was inaccurate for six of seven countries, the near-term
                                                      forecast was accurate for five of the G-7 country cases. The improved
                                                      quality of WEO forecasts for the G- 7 countries is likely due to better
                                                      available data and greater stability of these economies, compared to
                                                      emerging market countries.



                                                      Page 45                                                      GAO-03-734 International Financial Crises
                                                       Appendix III
                                                       Assessment of IMF Forecasting




WEO Forecasts of Countries on                          We found that WEO forecasts for 57 countries that were on an IMF
an IMF Program Are Superior to                         program12 (or that borrowed Fund resources under the CCFF) for any part
Nonprogram Country Forecasts                           of the forecast period tend to be more accurate than WEO forecasts for the
                                                       30 countries that were never on an IMF program during this period (see
                                                       table 4).



Table 4: Comparison of Year-Ahead WEO Forecasts for Program and Nonprogram Countries, 1990–2001

                                         Accuracy                                  Bias
                                      Theil U-statistica               5-percent significance levelb                           Direction of bias
                                    57 program 30 Nonprogram                  57 program 30 Nonprogram                      57 program         30 Nonprogram
Forecast variable                     countries     countries                   countries     countries                       countries             countries
GDP growth rate                           .827               .941                      Yes                  No                  Upward                       N/A
Inflation                                 .776               .918                      Yes                  No                Downward                        N/A
Current account                          2.399              1.129                       No                  Yes                       N/A             Downward
(billions of U.S. $)
                                                       Legend
                                                       N/A = not applicable
Source: GAO analysis of IMF data.
                                                       a
                                                        For pooled countries, if the U statistic is greater than 1, the naive forecast does a better job than the
                                                       forecast under evaluation.
                                                       b
                                                       Bias at the 5 percent level of significance means that there is less than a 5 percent chance that we are
                                                       making a false rejection—i.e., saying the forecast is biased when it is not.


                                                       Countries that borrow from the IMF are likely to be under greater scrutiny
                                                       from Fund staff than those that do not borrow, which could contribute to
                                                       an improved forecast. For this analysis, we compared the Theil statistics
                                                       for GDP growth, inflation, and current account for the two pooled
                                                       forecasts. We found that the WEO forecasts of GDP and inflation for
                                                       program countries are more accurate than those for nonprogram countries.
                                                       That is, when compared to the naive model, the program countries have a
                                                       lower Theil statistic than nonprogram countries. For both groups, the
                                                       forecast of current account is inferior to the naive forecast (a Theil statistic
                                                       greater than 1). The WEO program countries forecasts for GDP and
                                                       inflation are biased, whereas the forecasts for the nonprogram countries
                                                       were not. This indicates that by assuming implementation of the policies


                                                       12
                                                        We include countries that were on the following General Resources Account-funded IMF
                                                       programs and facilities: SBA, EFF, STF, CCFF. Some countries were on more than one
                                                       program during the forecast period.




                                                       Page 46                                                    GAO-03-734 International Financial Crises
                                                     Appendix III
                                                     Assessment of IMF Forecasting




                                                     contained within the program, the Fund expects that GDP and inflation will
                                                     perform better than they actually do.

                                                     In addition to the publicly available WEO forecasts for all countries, the
                                                     IMF also produces a set of program forecasts for countries in the years they
                                                     borrow from the Fund.13 According to the Fund, these two forecasts should
                                                     be very similar since they are prepared by the same staff in the same
                                                     manner. Our comparison of program14 and WEO forecasts for the initial
                                                     year that each country was on program confirmed that the accuracy of the
                                                     two forecasts for GDP and inflation were nearly identical (see table 5).



Table 5: Comparison of Program and WEO Year-Ahead Forecasts, for 52 Countries in the Initial Years on Program, 1990–2001a

                                        Accuracy                                 Bias
                                     Theil U-statisticb              5 percent significance levelc                           Direction of bias
                                     Program             WEO Program                      WEO                   Program                 WEO
Forecast variable                   forecasts        forecasts forecasts                  forecasts             forecasts               forecasts
GDP growth rate                          .689             .676 Yes                        Yes                   Upward                  Upward
Inflation                                .683             .694 Nod                        Nod                   N/A                     N/A
Current account                          .524             2.292 No                        No                    N/A                     N/A
(billions of U.S. $)
                                                     Legend
                                                     N/A = not applicable
Source: GAO analysis of IMF data.

                                                     Note: This analysis is based on 96 country/year observations, which is an average across three
                                                     variables.
                                                     a
                                                      We were able to obtain program projections for only 52 of the 57 program countries.
                                                     b
                                                      If the U statistic is greater than 1, the naive forecast does a better job than the forecast under
                                                     evaluation.
                                                     c
                                                       Bias at the 5 percent level of significance means that there is less than a 5 percent chance that we are
                                                     making a false rejection—i.e., saying the forecast is biased when it is not.
                                                     d
                                                      The forecast is biased at the 10 percent level of significance means that there is less than a 10
                                                     percent chance that we are making a false rejection—i.e., saying the forecast is biased when it is not.




                                                     13
                                                      For most of the years studied, these forecasts were not made public. In recent years, when
                                                     countries agree, these forecasts are made public.
                                                     14
                                                      We used the original program numbers, that is, the projections established at the outset
                                                     when the IMF’s Executive Board first approves an arrangement.




                                                     Page 47                                                   GAO-03-734 International Financial Crises
                               Appendix III
                               Assessment of IMF Forecasting




                               However, program forecasts of current account are substantially better
                               than those reported in the WEO. In addition, in all cases the program
                               forecasts were substantially more accurate than the naive model. This is
                               further evidence that the greater scrutiny experienced by these countries
                               while under a program probably contributes to an improved forecast.



Other Forecasting Efforts      Our review of other forecast evaluations found that the shortcomings we
Have Difficulties Similar to   observed in WEO forecasting are similar to difficulties encountered by
                               other forecasters.15 These studies examined forecasts16 produced by the
the Fund                       private sector (for example, consensus forecasts), governments, and
                               multinational agencies including the IMF and Organization for Economic
                               Cooperation and Development. These studies, similar to our observation in
                               this report, found a general inability to predict recessions. In addition,
                               consistent with our results, these studies found that (1) the shorter the time
                               horizon, the more accurate the forecasts; (2) current account forecasts are


                               15
                                 Harjit K. Arora & David J. Smyth, “Forecasting the Developing World: An Accuracy
                               Analysis of the IMF’s Forecasts,” International Journal of Forecasting 6 (1990): 393-400;
                               Michael J. Artis, “How Accurate are the IMF’s Short-Term Forecasts? Another Examination
                               of the World Economic Outlook,” WP/96/89 (Washington, D.C.: International Monetary Fund
                               [IMF], August 1996); José M. Barrionuevo, “How Accurate Are the World Economic Outlook
                               Projections?” in World Economic Outlook, ch. 2, Staff Studies for World Economic Outlook
                               (Washington, D.C.: IMF, 1993); Roy Batchelor, “The IMF and OECD versus Consensus
                               Forecasts” (London, England: City University Business School, August 2000); William W.
                               Beach, Aaron B. Schavey, & Isabel M. Isidro, “How Reliable Are IMF Economic Forecasts?”
                               CDA 99-05 (Washington, D.C.: The Heritage Foundation, The Heritage Center for Data
                               Analysis, August 27, 1999); David Fintzen & H.O. Stekler, “Why Did Forecasters Fail to
                               Predict the 1990 Recession?” International Journal of Forecasting 15 (1999): 309-323; IMF,
                               “The Accuracy of World Economic Outlook Growth Forecasts: 1991–2000,” in World
                               Economic Outlook, Box 3.1 (Washington, D.C.: IMF, December 2001): 37-39, and IMF, “How
                               Well Do Forecasters Predict Turning Points?” in World Economic Outlook, Box 1.1
                               (Washington, D.C.: IMF, May 2002): 6-8; Grace Juhn & Prakash Loungani, “Further Cross-
                               Country Evidence on the Accuracy of the Private Sector’s Output Forecasts,” IMF Staff
                               Papers 49, no. 1 (2002); Prakash Loungani, “How Accurate are Private Sector Forecasts?
                               Cross-Country Evidence from Consensus Forecasts of Output Growth,” International
                               Journal of Forecasting 17 (2001): 419-432; Albert Musso & Steven Phillips, “Comparing
                               Projections and Outcomes of IMF-Supported Programs,” IMF Staff Papers 49, no. 1 (2002);
                               Scott Shuh, “An Evaluation of Recent Macroeconomic Forecast Errors,” New England
                               Economic Review (January/February 2001); Marten Blix et al., “How Good is the
                               Forecasting Performance of Major Institutions?” Economic Review (Stockholm, Sweden:
                               Sveriges Riksbank Monetary Policy Department, March 2001): 38-68; and Victor Zarnowitz,
                               “The Record and Improvability of Economic Forecasting,” NBER Working Paper No. 2099
                               (December 1986).
                               16
                                Forecasters evaluated the accuracy of many variables, including output (GDP), inflation,
                               current account, exports, and imports, among others.




                               Page 48                                         GAO-03-734 International Financial Crises
Appendix III
Assessment of IMF Forecasting




markedly weaker than the forecasts for GDP and inflation; (3) when bias is
found, forecasts tend to overestimate GDP and underestimate inflation;
and (4) GDP and inflation forecasts for the industrial countries tend to be
more accurate and less biased than forecasts for developing countries.
While several studies found that WEO forecasts for developing countries
were inferior to those generated by the naive model, one study found that
WEO forecasts for developing countries did notably better than a naive
forecast. A number of studies compared the quality of WEO forecasts with
those produced by the private sector. Although some studies found the
relative quality of the forecasts to be generally the same, a few studies
found WEO forecasts to be less accurate than those of the private sector.




Page 49                                 GAO-03-734 International Financial Crises
Appendix IV

Eighty-seven Emerging Market Countries                                                            Appen
                                                                                                      V
                                                                                                      Id
                                                                                                       xi




    Country                     Regiona                   Income groupb
1   Albania                     Countries in transition   Lower middle income
2   Algeria                     Africa                    Lower middle income
3   Antigua and Barbuda         Western Hemisphere        Upper middle income
4   Argentina                   Western Hemisphere        Upper middle income
5   Bahrain                     Middle East and Turkey    Upper middle income
6   Belarus                     Countries in transition   Lower middle income
7   Belize                      Western Hemisphere        Lower middle income
8   Bolivia                     Western Hemisphere        Lower middle income
9   Botswana                    Africa                    Upper middle income
10 Brazil                       Western Hemisphere        Upper middle income
11 Bulgaria                     Countries in transition   Lower middle income
12 Cape Verde                   Africa                    Lower middle income
13 Chile                        Western Hemisphere        Lower middle income
14 China                        Developing Asia           Lower middle income
15 Colombia                     Western Hemisphere        Lower middle income
16 Costa Rica                   Western Hemisphere        Upper middle income
17 Cote d’Ivoire                Africa                    Low income
18 Croatia                      Countries in transition   Upper middle income
19 Czech Republic               Countries in transition   Upper middle income
20 Djibouti                     Africa                    Lower middle income
21 Dominica                     Western Hemisphere        Upper middle income
22 Dominican Republic           Western Hemisphere        Lower middle income
23 Ecuador                      Western Hemisphere        Lower middle income
24 Egypt                        Middle East and Turkey    Lower middle income
25 El Salvador                  Western Hemisphere        Lower middle income
26 Equatorial Guinea            Africa                    Lower middle income
27 Estonia                      Countries in transition   Upper middle income
28 Fiji                         Developing Asia           Lower middle income
29 Gabon                        Africa                    Upper middle income
30 Grenada                      Western Hemisphere        Upper middle income
31 Guatemala                    Western Hemisphere        Lower middle income
32 Guyana                       Western Hemisphere        Lower middle income
33 Honduras                     Western Hemisphere        Lower middle income
34 Hungary                      Countries in transition   Upper middle income
35 India                        Developing Asia           Low income




                          Page 50                         GAO-03-734 International Financial Crises
                                        Appendix IV
                                        Eighty-seven Emerging Market Countries




(Continued From Previous Page)
    Country                                   Regiona                            Income groupb
36 Indonesia                                  Developing Asia                    Low income
37 Iran, Islamic Rep. of                      Middle East and Turkey             Lower middle income
38 Iraq                                       Middle East and Turkey             Lower middle income
39 Jamaica                                    Western Hemisphere                 Lower middle income
40 Jordan                                     Middle East and Turkey             Lower middle income
41 Kazakhstan                                 Countries in transition            Lower middle income
42 Kiribati                                   Developing Asia                    Lower middle income
43 Korea                                      Developing Asia                    Upper middle income
44 Latvia                                     Countries in transition            Lower middle income
45 Lebanon                                    Middle East and Turkey             Upper middle income
46 Libya                                      Middle East and Turkey             Upper middle income
47 Lithuania                                  Countries in transition            Lower middle income
48 Macedonia, former Yugoslav Rep. of         Countries in transition            Lower middle income
49 Malaysia                                   Developing Asia                    Upper middle income
50 Maldives                                   Developing Asia                    Lower middle income
51 Mauritius                                  Africa                             Upper middle income
52 Mexico                                     Western Hemisphere                 Upper middle income
53 Morocco                                    Africa                             Lower middle income
54 Namibia                                    Africa                             Lower middle income
55 Nigeria                                    Africa                             Low income
56 Oman                                       Middle East and Turkey             Upper middle income
57 Pakistan                                   Developing Asia                    Low income
58 Panama                                     Western Hemisphere                 Upper middle income
59 Papua New Guinea                           Developing Asia                    Lower middle income
60 Paraguay                                   Western Hemisphere                 Lower middle income
61 Peru                                       Western Hemisphere                 Lower middle income
62 Philippines                                Developing Asia                    Lower middle income
63 Poland                                     Countries in transition            Upper middle income
64 Romania                                    Countries in transition            Lower middle income
65 Russia                                     Countries in transition            Lower middle income
66 Samoa                                      Developing Asia                    Lower middle income
67 Saudi Arabia                               Middle East and Turkey             Upper middle income
68 Seychelles                                 Africa                             Upper middle income
69 Slovak Republic                            Countries in transition            Upper middle income
70 South Africa                               Africa                             Upper middle income
71 Sri Lanka                                  Developing Asia                    Lower middle income
72 St. Kitts and Nevis                        Western Hemisphere                 Upper middle income




                                        Page 51                                  GAO-03-734 International Financial Crises
                                             Appendix IV
                                             Eighty-seven Emerging Market Countries




(Continued From Previous Page)
      Country                                      Regiona                                        Income groupb
73 St. Lucia                                       Western Hemisphere                             Upper middle income
74 St. Vincent and the Grenadines                  Western Hemisphere                             Lower middle income
75 Suriname                                        Western Hemisphere                             Lower middle income
76 Swaziland                                       Africa                                         Lower middle income
77 Syrian Arab Rep.                                Middle East and Turkey                         Lower middle income
78 Thailand                                        Developing Asia                                Lower middle income
79 Tonga                                           Developing Asia                                Lower middle income
80 Trinidad and Tobago                             Western Hemisphere                             Upper middle income
81 Tunisia                                         Africa                                         Lower middle income
82 Turkey                                          Middle East and Turkey                         Upper middle income
83 Turkmenistan                                    Countries in transition                        Lower middle income
84 Ukraine                                         Countries in transition                        Low income
85 Uruguay                                         Western Hemisphere                             Upper middle income
86 Vanuatu                                         Developing Asia                                Lower middle income
87 Venezuela                                       Western Hemisphere                             Upper middle income
Sources: IMF, World Bank, and J.P. Morgan.
                                             a
                                             Based on WEO regional groupings, IMF 2002.
                                             b
                                             World Bank analytical classification by income, July 2001.




                                             Page 52                                               GAO-03-734 International Financial Crises
Appendix V

Standards, Codes, and Principles Assessed
under IMF and World Bank Reports on the
Observance of Standards and Codes                                                                                                         Appen
                                                                                                                                              V
                                                                                                                                              di
                                                                                                                                              x




Assessment start
date                     Responsible institution                     Standard or code and rationale for adoption
Transparency standards: The standards on transparency in government operations and policy making are considered within the Fund’s
direct operational focus.
1996–97                  IMF                                         IMF Special Data Dissemination Standard (SDDS) and General
                                                                     Data Dissemination Standard (GDDS).

                                                                     The purpose of the IMF’s SDDS is to guide member country
                                                                     governments that have, or might seek, access to international
                                                                     capital markets in publishing comprehensive, timely, accessible,
                                                                     and reliable economic and financial statistics. The purpose of the
                                                                     GDDS is to help any member country government provide more
                                                                     reliable economic data. The SDDS and GDDS were created in
                                                                     1996 and 1997, respectively.
1998                     IMF                                         IMF Code of Good Practices on Fiscal Transparency.
(modified 2001)
                                                                     This Code is intended to help member country governments
                                                                     improve the disclosure of information about the design and results
                                                                     of fiscal policy, making governments more accountable for policy
                                                                     implementation and strengthening credibility and public
                                                                     understanding of macroeconomic policies and choices.
1999                     IMF                                         IMF Code of Good Practices on Transparency in Monetary and
                                                                     Financial Policies.

                                                                     This Code is designed to increase the effectiveness of monetary
                                                                     and financial policies by raising public awareness of the
                                                                     government’s policy goals and instruments and making
                                                                     governments (especially independent central banks and financial
                                                                     agencies) more accountable.
Financial sector standards: The financial sector standards are considered within the direct operational focus of both the Fund and the
World Bank and are generally assessed under the joint Fund-Bank FSAP.
1999                    Joint IMF-Bank                               Basel Committee’s Core Principles for Effective Banking
                                                                     Supervision (BCP).

                                                                     The BCP is intended to guide the development of an effective
                                                                     system for supervising banks, a large sector of most economies.
                                                                     The IMF and World Bank began assessments of countries’
                                                                     compliance with the BCP standard in conjunction with the Financial
                                                                     Sector Assessment Program (FSAP) launched in 1999.
1999                    Joint IMF-Bank                               International Organization of Securities Commissions’ (IOSCO)
                                                                     Objectives and Principles for Securities Regulation.

                                                                     The IOSCO Objectives and Principles are designed to help
                                                                     governments establish effective systems to regulate securities
                                                                     which contribute strongly to investor confidence. The IMF and
                                                                     World Bank began using them to assess securities regulation in
                                                                     conjunction with the FSAP, launched in 1999.




                                               Page 53                                          GAO-03-734 International Financial Crises
                                              Appendix V
                                              Standards, Codes, and Principles Assessed
                                              under IMF and World Bank Reports on the
                                              Observance of Standards and Codes




(Continued From Previous Page)
Assessment start
date                    Responsible institution                     Standard or code and rationale for adoption
1999                    Joint IMF-Bank                              International Association of Insurance Supervisors’ (IAIS)
(modified 2000)                                                     Insurance Core Principles.

                                                                    The IAIS Core Principles are designed to contribute to effective
                                                                    insurance supervision that supports financial stability. The IMF and
                                                                    World Bank began assessing member countries’ regulatory
                                                                    practices in this area in conjunction with the FSAP, launched in
                                                                    1999.
1999                    Joint IMF-Bank                              Committee on Payments and Settlements Systems’ (CPSS) Core
                                                                    Principles for Systemically Important Payment Systems.

                                                                    The CPSS Core Principles are intended to strengthen payments
                                                                    systems, which provide the channels through which funds are
                                                                    transferred among banks and other institutions. The IMF and the
                                                                    World Bank began assessing member countries’ observance of
                                                                    this standard in conjunction with the FSAP, launched in 1999.
2002                    Joint IMF-Bank                              Financial Action Task Force (FATF) 40 Recommendations on Anti-
                                                                    Money Laundering and 8 Special Recommendations on Terrorism
                                                                    Financing.

                                                                    The FATF’s 40 Recommendations and 8 Special
                                                                    Recommendations are intended to promote policies that combat
                                                                    money laundering and terrorist financing, which threaten financial
                                                                    system integrity and may undermine the sound functioning of
                                                                    financial systems. In 2002, the IMF and World Bank agreed to
                                                                    perform anti-money laundering and terrorist financing
                                                                    assessments as a 12-month pilot program, generally in conjunction
                                                                    with the FSAP.
Corporate sector standards: The corporate sector standards are considered important for the effective operation of domestic and
international financial systems and are assessed by the World Bank.
2000                    World Bank                                  Organization for Economic Cooperation and Development’s
                                                                    (OECD) Principles of Corporate Governance.

                                                                    The OECD developed its corporate governance principles to help
                                                                    governments evaluate and improve their legal, institutional, and
                                                                    regulatory frameworks for corporate governance. The World Bank
                                                                    developed a template for assessing adherence to corporate
                                                                    governance principles based on the OECD’s Principles established
                                                                    in 1999.




                                              Page 54                                          GAO-03-734 International Financial Crises
                                                   Appendix V
                                                   Standards, Codes, and Principles Assessed
                                                   under IMF and World Bank Reports on the
                                                   Observance of Standards and Codes




(Continued From Previous Page)
Assessment start
date                           Responsible institution                   Standard or code and rationale for adoption
2000                           World Bank                                International Accounting Standards Board’s (IASB) International
                                                                         Accounting Standards.

                                                                         The ROSC’s accounting module is intended to compare member
                                                                         countries’ corporate accounting practices with international
                                                                         accounting standards and to analyze actual accounting practice to
                                                                         determine the extent of compliance with applicable standards.
                                                                         There is special focus on the strengths and weaknesses of the
                                                                         institutional framework for supporting high quality accounting and
                                                                         financial reporting. In 2000, the World Bank developed a template
                                                                         for assessing adherence to accounting standards based on the
                                                                         IASB Standards.
2000                           World Bank                                International Federation of Accountants’ (IFAC) International
                                                                         Standards on Auditing.

                                                                         The ROSC’s auditing module compares member countries’
                                                                         auditing standards and auditors’ professional code of ethics with
                                                                         the standards and codes issued by IFAC. Also, the quality of actual
                                                                         auditing practices is evaluated. There is special focus on the
                                                                         strengths and weaknesses of the institutional framework for
                                                                         supporting high quality audit. In 2000, the World Bank developed a
                                                                         template for assessing adherence to auditing standards based on
                                                                         the IFAC’s Standards.
2001 (draft)                   World Bank                                World Bank Principles and Guidelines for Effective Insolvency and
                                                                         Creditor Rights Systems.

                                                                         In 2001, the World Bank developed draft Principles and Guidelines
                                                                         intended to help countries develop effective insolvency and creditor
                                                                         rights’ systems, two important components of financial system
                                                                         stability. The World Bank has conducted several assessments
                                                                         based on its draft Principles and Guidelines. The United Nations
                                                                         Commission on International Trade Law (UNCITRAL) is completing
                                                                         a draft Legislative Guide on Insolvency Law. UNCITRAL, Bank,
                                                                         and IMF staff are working toward a single standard.
Sources: IMF and World Bank.




                                                   Page 55                                          GAO-03-734 International Financial Crises
Appendix VI

Update on the International Monetary Fund’s
Safeguards Assessments                                                                                        AppenV
                                                                                                                   d
                                                                                                                   xiI




                        In response to allegations of misreporting and misuse of International
                        Monetary Fund (IMF or the Fund) disbursements in the late 1990s, the
                        Fund increased its efforts to protect its resources by introducing
                        safeguards assessments, a process for evaluating the controls employed by
                        the central banks of borrowing member countries and for recommending
                        measures to address inadequacies. Safeguards assessments have detected
                        numerous inadequacies that could lead to misuse of Fund resources and
                        have recommended measures to remedy them.



The Fund Introduces     In 2000, the Fund introduced safeguards assessments, a process for
                        identifying inadequacies in central banks’ ability to ensure the integrity of
Safeguards              their operations, especially the use of Fund resources. Safeguards
Assessments for         assessments evaluate central banks’ internal and external audit
                        mechanisms, legal structure and independence, financial reporting
Member Countries        procedures, and systems of internal controls.
That Currently Borrow
                        In April 2002, the Fund’s Executive Board made safeguards assessments a
                        permanent policy. Safeguards assessments apply to all member countries
                        with current or anticipated borrowing arrangements with the Fund.
                        Countries with borrowing arrangements approved after June 30, 2000, are
                        subject to a full safeguards assessment covering the five areas listed above.
                        Countries with arrangements in effect before June 30, 2000, that have not
                        yet repaid all Fund resources, were subject to a partial assessment
                        covering only the external audit mechanism. Countries that do not have
                        borrowing arrangements or have already repaid all Fund resources are not
                        subject to safeguards assessments. According to Fund officials, since 2000
                        the IMF has not provided financial resources to countries that did not meet
                        its safeguards requirements.

                        As of December 2002, the Fund had performed 37 full safeguards
                        assessments and 27 partial assessments, with 23 assessments under way.1
                        The completed assessments detected a number of serious vulnerabilities
                        that could lead to misuse of central bank resources, including those
                        borrowed from the Fund. Of the full safeguards assessments, the Fund
                        found the following:




                        1
                         International Monetary Fund, Safeguards Assessments – Semi-Annual Update, SM/03/88
                        (Washington, D.C.: Mar. 6, 2003).




                        Page 56                                      GAO-03-734 International Financial Crises
                        Appendix VI
                        Update on the International Monetary Fund’s
                        Safeguards Assessments




                        • Inadequate accounting standards in 82 percent of the central banks,
                          which interfere with the accurate recording of central bank operations.
                          For example, some central banks did not adhere to a financial reporting
                          framework such as the International Accounting Standards (IAS), which
                          would help prevent misreporting of transactions.

                        • Deficient internal audit in 79 percent of central banks, which reduces
                          their ability to address risks of misuse and misreporting of Fund
                          resources. For example, some internal audit departments did not audit
                          high-risk areas such as foreign reserves management.

                        • Poor controls over foreign reserves and data reporting to the Fund in 49
                          percent of the central banks, increasing the possibility of misreporting
                          and misuse of Fund resources. For example, safeguards assessments
                          identified improper techniques for valuing foreign reserves and failure
                          to reconcile data reported to the Fund for program monitoring purposes
                          with underlying accounting records.

                        According to Fund officials, when IMF staff detect significant weaknesses
                        in the controls of assessed central banks, they recommend that the
                        government take corrective actions. For actions that IMF staff consider
                        essential, they may incorporate the recommendations into the list of
                        preconditions that the Fund requires borrower countries meet before
                        receiving IMF resources, or they may suggest that the government include
                        the recommended actions in its official economic program. The Fund
                        reports that of the 275 recommendations that were expected to be
                        implemented on or before December 31, 2002, 23 percent were
                        incorporated as conditions for IMF resources or included in official
                        economic program statements.



Implementation of the   Fund staff monitor central banks’ implementation of recommendations by
                        performing in-depth reviews of their audited annual financial statements
Fund’s Assessment       and other documents every 12 to 18 months until the borrower country
Recommendations         government has repaid all Fund resources. The Fund monitors on a
                        continuous basis, central banks’ implementation of all other safeguards
                        measures and of developments within the central banks that may lead to
                        new vulnerabilities.




                        Page 57                                       GAO-03-734 International Financial Crises
Appendix VI
Update on the International Monetary Fund’s
Safeguards Assessments




Recently, the Fund reported that central banks have implemented 90
percent of recommendations that IMF staff included as a precondition for
receiving IMF resources. According to Fund officials, the IMF stopped
disbursing resources in the few cases where governments failed to
implement these essential recommendations. Similarly, the Fund reported
that central banks have implemented 84 percent of measures included in
governments’ official economic program statements.

On the other hand, the Fund reported that some recommendations made by
the safeguards assessments have not been implemented as intended,
although Fund officials state that these delays typically occurred in
nonpriority areas. When central bank authorities fail to implement the
recommendations, Fund staff increase pressure to comply, often proposing
the measures’ inclusion as a precondition for the next disbursement.
However, the Fund reports that staff can only adopt this approach in
countries where the Fund is actively disbursing funds. For countries that
are not currently receiving Fund disbursements, implementation of
recommendations from the safeguards assessments tends to be more
problematic because the Fund cannot exert pressure through a borrowing
arrangement.




Page 58                                       GAO-03-734 International Financial Crises
Appendix VII

Fund/World Bank FSAP and ROSC Process                              AppenV
                                                                        d
                                                                        xiI




               Page 59     GAO-03-734 International Financial Crises
Appendix VIII

Country Participation in and Publication of
FSAPs and ROSCs                                                                                 AppenV
                                                                                                     diI
                                                                                                     x




                Figure 4 lists all countries that have participated in Financial Sector
                Assessment Program (FSAP) or Reports on the Observance of Standards
                and Codes (ROSC) assessments and whether or not these assessments
                were published. The figure describes participation and publication by the
                33 major emerging market countries. Countries highlighted in bold have
                not participated in any assessments. Figure 5 describes participation and
                publication by other countries (industrial, developing, and smaller
                emerging markets).




                Page 60                                 GAO-03-734 International Financial Crises
                                         Appendix VIII
                                         Country Participation in and Publication of
                                         FSAPs and ROSCs




Figure 4: FSAP and ROSC Participation and Publication by Major Emerging Market Countries




                                         a
                                          These countries participated in the FSAP under a pilot program. The reports were not intended for
                                         publication.




                                         Page 61                                               GAO-03-734 International Financial Crises
                                         Appendix VIII
                                         Country Participation in and Publication of
                                         FSAPs and ROSCs




Figure 5: FSAP and ROSC Participation and Publication by Other Countries




                                         Page 62                                       GAO-03-734 International Financial Crises
Appendix VIII
Country Participation in and Publication of
FSAPs and ROSCs




a
 These countries participated in the FSAP under a pilot program. The reports were not intended for
publication.




Page 63                                               GAO-03-734 International Financial Crises
Appendix IX

Fund Resources Provided under the
Supplemental Reserve Facility                                                                    Appen
                                                                                                     X
                                                                                                     Id
                                                                                                      xi




              In recent financial crises, the International Monetary Fund (IMF or the
              Fund) provided large short-term loans under the Supplemental Reserve
              Facility (SRF) with high interest rates to member countries experiencing
              exceptional balance of payments problems. These problems resulted from
              a sharp decline of investor confidence and significant outflows of capital.
              These loans generally were provided when the countries had exceeded
              their financing limit under other loan mechanisms, including the Stand-By
              Arrangement (SBA). In some circumstances, such as Argentina and
              Uruguay, the Fund provided a mix of SRF and SBA loans. Table 6 lists Fund
              members receiving SRF loans from 1997 to 2002.



              Table 6: IMF Supplemental Reserve Facility Loans, 1997–2002


                                                                      SRF repaid as of May
              Country            Loan approval date Loan mix          2003
              Korea              12/18/97            SRF              Yes
              Russia             7/20/98             SRF              Yes
              Brazil             12/2/98             SRF              Yes
              Turkey             12/21/00            SRF              Yes
                                                     SRF (40%)
              Argentina          1/12/01             SBA (60%)        No
                                                     SRF (63%)
              Argentina          9/7/01              SBA (37%)        No
              Brazil             9/14/01             SRF              No
                                                     SRF (33%)
              Uruguay            6/25/02             SBA (67%)        No
                                                     SRF (33%)
              Brazil             9/6/02              SBA (67%)        No
              Source: IMF.




              Page 64                                    GAO-03-734 International Financial Crises
Appendix X

Comments from the Department of the
Treasury                                                             Appen
                                                                         X
                                                                         di
                                                                         x




              Page 65        GAO-03-734 International Financial Crises
Appendix X
Comments from the Department of the
Treasury




Page 66                               GAO-03-734 International Financial Crises
Appendix XI

Comments from the International Monetary
Fund                                                                       AppenX
                                                                                d
                                                                                xiI




Note: GAO comments
supplementing those in
the report text appear
at the end of this
appendix.




See comment 1.




See comment 2.




                         Page 67   GAO-03-734 International Financial Crises
                 Appendix XI
                 Comments from the International Monetary
                 Fund




See comment 3.




See comment 4.




See comment 5.




See comment 6.




                 Page 68                                    GAO-03-734 International Financial Crises
Appendix XI
Comments from the International Monetary
Fund




Page 69                                    GAO-03-734 International Financial Crises
               Appendix XI
               Comments from the International Monetary
               Fund




               The following are GAO’s comments on the letter from the International
               Monetary Fund, dated June 2, 2003.



GAO Comments   1. Our assessment of IMF’s efforts to anticipate financial crises did not
                  focus exclusively on the World Economic Outlook (WEO) and the Early
                  Warning System (EWS) models. We examined all six components of
                  the IMF’s vulnerability assessment framework. However, as the only
                  mature and quantifiable elements of the framework, our analysis
                  focused more heavily on the track records of the WEO and EWS. As we
                  reported, these elements have not performed well at anticipating prior
                  crises. While we acknowledge that the new framework is more
                  comprehensive than past efforts at anticipating crises, it is too early to
                  assess whether this framework will be successful in anticipating future
                  crises.

               2. The IMF’s comment supports our finding. The IMF states that its
                  responsibility to maintain financial stability could make its predictions
                  appear less accurate since an accurate prediction of crises within WEO
                  forecasts would be irresponsible and could lead to a crisis. In effect,
                  the IMF acknowledges that their forecasts are overly optimistic and
                  validates our finding on the weakness of the WEO component of the
                  vulnerability assessment framework. This raises questions regarding
                  the purpose and credibility of the WEO forecasts.

               3. We state in the report that the IMF’s new vulnerability assessment
                  framework, which includes the examination of external reserve
                  adequacy and the strengths and weaknesses of banking systems, is
                  more comprehensive than its previous efforts to identify countries at
                  risk of crisis. However, it is too early to assess whether this framework
                  will be successful in anticipating future crises.

               4. We recognize that the EWS models are just one of six components of
                  the IMF’s vulnerability assessment framework. However, the IMF’s
                  own internal review of the EWS models concluded that the false alarm
                  rate was too high.

               5. The report recognizes that the FSAP and ROSC assessments constitute
                  a valuable source of information about vulnerabilities in member
                  countries and states that IMF staff use these assessments to formulate
                  policy recommendations. The report also recognizes that the IMF often
                  provides technical assistance to help member countries build their



               Page 70                                    GAO-03-734 International Financial Crises
Appendix XI
Comments from the International Monetary
Fund




    capacity to implement FSAP and ROSC recommendations. However,
    the report points out several factors that limit the usefulness of FSAP
    and ROSC assessments. Our recommendation, with which the IMF
    agrees, is designed to improve the timeliness and coverage of these
    assessments.

6. We based our description of IMF safeguards assessments on the IMF’s
   reviews of this program. We consider this topic to be within the scope
   of this evaluation because the framework for conducting safeguards
   assessments is derived from the IMF’s Code of Good Practices on
   Transparency in Monetary and Financial Policies. Safeguards
   assessments are thus related to the standards initiative, which
   constitutes a central element of this report.




Page 71                                    GAO-03-734 International Financial Crises
Appendix XII

Comments from The World Bank                                               AppenX
                                                                                d
                                                                                xiI




Note: GAO comments
supplementing those in
the report text appear
at the end of this
appendix.




See comment 1.




See comment 2.




                         Page 72   GAO-03-734 International Financial Crises
Appendix XII
Comments from The World Bank




Page 73                        GAO-03-734 International Financial Crises
               Appendix XII
               Comments from The World Bank




               The following are GAO’s comments on the letter from the World Bank,
               dated June 2, 2003.



GAO Comments   1. The report states unambiguously that crises can stem from a number of
                  factors, some of which are outside the scope of the FSAP and ROSC
                  assessments. However, there is broad agreement that the roots of the
                  Mexican and Asian financial crises lay in weaknesses in financial
                  systems and other institutions. The IMF and the World Bank based
                  their decision to launch the FSAP and ROSC initiatives on the premise
                  that timely identification of financial sector and institutional
                  vulnerabilities can contribute to crisis prevention. The IMF and the
                  World Bank have also acknowledged that FSAP and ROSC assessments
                  can contribute to crisis prevention efforts by helping private sector
                  participants make better informed investment decisions.

               2. Our recommendation to pursue strategies to increase participation in
                  the FSAP and ROSC assessments, including the possibility of making
                  these assessments mandatory, stems from the IMF’s and the World
                  Bank’s recognition of the need to prioritize participation by important
                  emerging market countries. Although many of these countries have
                  volunteered to participate in these assessments, others have not. While
                  we are not suggesting that the assessments should be made mandatory,
                  it is evident that the voluntary nature of the FSAP and ROSC has posed
                  an obstacle to full participation by important emerging market
                  countries.




               Page 74                                 GAO-03-734 International Financial Crises
Appendix XIII

GAO Contacts and Staff Acknowledgments                                                           AppenX
                                                                                                      diI
                                                                                                      x




GAO Contacts      Tom Melito (202) 512-9601
                  Zina Merritt (202) 512-5257



Staff             In addition to those individuals named above, Eric Clemons, Suzanne Dove,
                  Bruce Kutnick, Jonathan Rose, R.G. Steinman, Ian Ferguson, Mary
Acknowledgments   Moutsos, Lynn Cothern, Carl Barden, David Dornisch, and Martin De
                  Alteriis made key contributions to this report.




(320114)          Page 75                                GAO-03-734 International Financial Crises
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO
                         Mailing Lists” under “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested