oversight

Nonproliferation and Disarmament Fund: State Should Better Assure the Effective Use of Program Authorities

Published by the Government Accountability Office on 2012-11-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States Government Accountability Office

GAO             Report to the Ranking Member,
                Committee on Foreign Relations, U.S.
                Senate



                NONPROLIFERATION
November 2012



                AND DISARMAMENT
                FUND

                State Should Better
                Assure the Effective
                Use of Program
                Authorities




GAO-13-83
                                              November 2012


                                              NONPROLIFERATION AND DISARMAMENT
                                              FUND
                                              State Should Better Assure the Effective Use of
Highlights of GAO-13-83, a report to the      Program Authorities
Ranking Member, Committee on Foreign
Relations, U.S. Senate




Why GAO Did This Study                        What GAO Found
The proliferation of weapons of mass          The Department of State’s (State) Nonproliferation and Disarmament Fund
destruction and advanced conventional         (NDF) has several key authorities that provide it significant operational flexibility;
weapons poses significant threats to          however, it has not determined its needed carryover balances and it has taken
U.S. and international security. State’s      years to close out many of its projects in the absence of guidance for closing
NDF began operating in 1994 to help           them. Annual appropriations bills have consistently provided NDF with three key
combat such threats by funding a              authorities that it has used to carry out its activities. First, NDF has the authority
variety of nonproliferation and               to undertake projects notwithstanding any other provision of law. NDF has used
disarmament projects. NDF’s legal             this authority to fund projects in countries, such as North Korea, where U.S.
authorities provide it significant
                                              assistance is prohibited by U.S. sanctions and other legal restrictions. Second,
flexibility to perform its work and it has
                                              NDF has the authority to undertake projects globally. NDF has used this
initiated high-profile projects in
locations that are significant to U.S.
                                              authority to fund projects in numerous regions around the world, in contrast with
interests. Nonetheless, questions have        other U.S. nonproliferation programs, which have historically focused on
been raised about how NDF has used            countries in the former Soviet Union. Third, NDF’s appropriations do not expire
its authorities, including its authority to   within a particular time period, enabling NDF to carry over balances from year to
carry over balances into future fiscal        year not designated for specific projects. However, NDF has not determined
years, and the extent to which NDF is         appropriate levels for these balances, which increased significantly in the past
effectively implementing its activities.      few years. Additionally, NDF has sometimes taken many years to close projects,
This report examines (1) State’s use of       including those where work was never started or was suspended, and has not
NDF authorities in developing and             established criteria to determine when inactive projects should be closed and
implementing NDF projects and (2) the         unexpended resources made available for other projects. As a result, NDF funds
extent to which State has conducted a         may be tied up for years in inactive projects, precluding the funds’ use for other
program evaluation of NDF and used            projects.
this information to improve program
performance. To conduct this review,          State has never conducted a program evaluation of NDF. In February 2012,
GAO analyzed NDF program and                  State developed a policy requiring bureaus to evaluate programs, projects, and
project data and documentation,               activities, and outlined the requirements for these evaluations. As part of this
analyzed a sample of NDF project              policy, State required bureaus to submit an evaluation plan for fiscal years 2012
close-out documents, and interviewed          through 2014 that identified the programs and projects they plan to evaluate.
NDF and other U.S. officials.                 However, the Bureau of International Security and Nonproliferation (ISN), which
                                              oversees NDF, did not include NDF in its fiscal years 2012 through 2014
What GAO Recommends                           evaluation plan. State currently lacks information that could be used to conduct a
                                              program evaluation and to improve NDF’s management of the program. Project
GAO recommends that State (1)                 close-out reports are critical to the process of closing out a project and identifying
develop a methodology for determining         lessons learned, but NDF project close-out reports do not contain information
the amount of carryover reserves              that could enable NDF to better manage its program. For example, not all close-
needed to meet program requirements,
                                              out reports address the results of the project. NDF uses e-mails and face-to-face
(2) develop guidance for determining
                                              meetings to communicate lessons learned without documenting them.
when inactive NDF projects should be
closed out, (3) conduct periodic              Established standards suggest that these should be transferred to a database of
program evaluations of NDF, and (4)           lessons learned for use in future projects and activities, an action State officials
establish requirements for the types of       said they are considering taking. NDF has also produced a project management
information to be included in project         guide to encourage project managers to use standard procedures and write
close-out reports. State agreed with          close-out reports, but does not require the use of this guide. In addition, the guide
the recommendations.                          does not detail a format for project managers to use in preparing their close-out
                                              reports or list the information that project managers must address. NDF officials
                                              said they plan to develop standard operating procedures to address these
View GAO-13-83. For more information,         issues, but had not done so as of November 2012.
contact Thomas Melito at (202) 512-9601 or
melitot@gao.gov.



                                                                                        United States Government Accountability Office
Contents


Letter                                                                                     1
               Background                                                                  3
               NDF’s Authorities Provide It Significant Operational Flexibility, but
                 under Its No-Year Budget Authority, NDF Has Not Determined
                 Needed Carryover Balances                                               10
               State Has Not Conducted a Program Evaluation of NDF and Lacks
                 Information for Conducting an Evaluation to Inform NDF’s
                 Program Management                                                      22
               Conclusion                                                                29
               Recommendations for Executive Action                                      30
               Agency Comments and Our Evaluation                                        31

Appendix I     Scope and Methodology                                                     32



Appendix II    Comments from the Department of State                                     36



Appendix III   GAO Contact and Staff Acknowledgments                                     39



Table
               Table 1: Authorized NDF Activities under the FREEDOM
                        Support Act                                                        4


Figures
               Figure 1: NDF-Funded Dismantling of a Scud Missile in Ukraine               5
               Figure 2: NDF Project Review and Approval Process                           6
               Figure 3: Congressionally-Notified NDF Projects, Fiscal Years 2007
                        through 2012                                                       8
               Figure 4: NDF Funding by Project Category, Fiscal Years 2007
                        through 2012                                                       9
               Figure 5: Locations of NDF Projects both Before and After Fiscal
                        Year 2007                                                        15
               Figure 6: Overview of NDF Funding Categories and Accumulation
                        of Carryover Balances                                            17
               Figure 7: NDF Funds Available for Notification, Fiscal Years 1994
                        through 2013                                                     18


               Page i                                             GAO-13-83 State NDF Program
Abbreviations
CTR         Cooperative Threat Reduction Program
DOD         Department of Defense
DOE         Department of Energy
FAR         Federal Acquisition Regulation
FIMS        Financial and Information Management System
FSU         former Soviet Union
GTR         Global Threat Reduction Program
ISN         Bureau of International Security and Nonproliferation
NDF         Nonproliferation and Disarmament Fund
OMB         Office of Management and Budget
State       Department of State


This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                                      GAO-13-83 State NDF Program
United States Government Accountability Office
Washington, DC 20548




                                   November 30, 2012

                                   The Honorable Richard G. Lugar
                                   Ranking Member
                                   Committee on Foreign Relations
                                   United States Senate

                                   Dear Senator Lugar:

                                   The proliferation of weapons of mass destruction and their delivery
                                   systems, as well as the spread of advanced conventional weapons,
                                   poses significant threats to U.S. and international security. Congress
                                   created the Nonproliferation and Disarmament Fund (NDF) in 1992 to
                                   support U.S. government efforts to combat such threats. 1 Since then, the
                                   Department of State (State), which manages the program, has used NDF
                                   to fund a variety of nonproliferation and disarmament projects. These
                                   projects have included the shutdown of chemical weapons facilities, the
                                   transfer of highly enriched uranium to secure locations, the construction
                                   of border security facilities, and the destruction of stockpiles of ballistic
                                   missiles. NDF has funded projects in a range of countries around the
                                   globe and has also supported work by international organizations, such
                                   as the International Atomic Energy Agency. Congress has provided NDF
                                   with various legal authorities to increase its ability to carry out
                                   nonproliferation and disarmament projects, as opportunities arise. State
                                   officials have noted that these authorities provide NDF a level of flexibility
                                   in carrying out its mission that is uncommon among U.S. government
                                   programs. According to State, NDF has made it possible for the U.S.
                                   government to respond rapidly to unanticipated or unusually difficult, high-
                                   priority nonproliferation and disarmament opportunities. While NDF has
                                   taken on high-profile projects in locations significant to U.S. interests,
                                   such as North Korea and Libya, questions have been raised about how
                                   NDF has used its legal authorities, including its authority to carry over
                                   funds into future fiscal years, and the extent to which it is effectively
                                   implementing its activities.




                                   1
                                    While the legislation creating NDF was passed in 1992, NDF did not become operational
                                   and begin funding projects until 1994.




                                   Page 1                                                    GAO-13-83 State NDF Program
This report examines (1) State’s use of NDF authorities in developing and
implementing NDF projects and (2) the extent to which State has
conducted a program evaluation of NDF and used this information to
improve program performance. To address these objectives, we analyzed
NDF data on program appropriations, commitments, obligations, and
carryover balances for fiscal years 1994 through 2012. We also analyzed
NDF project data on funding amounts, locations, objectives, and time
frames for fiscal years 1994 through 2012. We assessed both the program
and project data and found them sufficiently reliable for our purposes.
Additionally, we reviewed NDF project documentation, such as project
proposal summaries, approval memos, and congressional notifications to
obtain additional information on NDF projects. To identify NDF’s key legal
authorities, we reviewed relevant laws and regulations. To assess the
extent to which State has evaluated NDF, we met with State officials from
the Bureaus of Budget and Planning and International Security and
Nonproliferation (ISN), and reviewed project close-out documentation for a
judgmental sample of 23 project close-out reports—14 of which we
selected, and 9 of which State selected. In selecting our sample, we
considered only projects that NDF had closed out since the beginning of
fiscal year 2007. Our selection criteria included project cost, location, and
type. State selected its projects using similar criteria; however, State did
not limit itself to projects that were closed out. For example, State selected
some projects for which work was completed, but the project was not yet
officially closed out. For all 23 project close-out reports, we assessed the
types of information contained in these reports, and the degree to which
they could be used to provide needed information for evaluations and
improve NDF’s management of the program. In reviewing the
documentation for the projects State selected, we determined that the
inclusion of these projects in our analysis did not alter our overall findings
and thus did not compromise the independence of our work. We also met
with State officials in the NDF program office and officials from the
Departments of Defense (DOD) and Energy (DOE) who were responsible
for proposing or implementing NDF projects. Additionally, we met with
officials from the Office of Management and Budget (OMB) to better
understand NDF’s budget planning process. Additional details about our
scope and methodology can be found in appendix I.

We conducted this performance audit from March 2012 through
November 2012 in accordance with generally accepted government
auditing standards. These standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We



Page 2                                              GAO-13-83 State NDF Program
             believe that the evidence obtained provides a reasonable basis for our
             findings and conclusions based on our audit objectives.


             Congress authorized the President to establish NDF in 1992 under
Background   section 504 of the FREEDOM Support Act. 2 The legislation authorized the
             President to use NDF to promote a variety of bilateral and multilateral
             nonproliferation and disarmament activities. In 1994, the President
             delegated authority for the program to the Secretary of State, who
             subsequently delegated authority for the program to the Under Secretary
             of State for Arms Control and International Security. 3 The NDF office,
             within ISN, is responsible for day-to-day management of the program.
             The NDF Director leads the office, which has a staff of 16 people,
             including both State officials and contract employees.

             Congress funds NDF annually through the Nonproliferation, Anti-
             terrorism, Demining, and Related Programs appropriations account,
             within the Foreign Operations, Export Financing, and Related Programs
             Appropriations Acts. NDF received $10 million in initial funding for fiscal
             year 1994. Since fiscal year 1994, NDF has received $597 million in total
             appropriations. From fiscal years 2007 through 2012, NDF appropriations
             ranged from a high of $118 million in fiscal year 2009 to a low of $30
             million in fiscal year 2012. 4 According to State, NDF is unusual among
             U.S. foreign assistance programs in that it does not request funding for
             specific activities as part of its annual Congressional Budget Justification.
             The NDF Director stated that this helps ensure that NDF has the flexibility
             to respond to nonproliferation and disarmament opportunities as they
             arise, rather than tying NDF funds to particular projects or locations in
             advance.




             2
              Pub. L. No. 102-511, § 504.
             3
              At the time, the Secretary of State delegated the authority to the Under Secretary of State
             for International Security Affairs, but this position was subsequently renamed the Under
             Secretary of State for Arms Control and International Security.
             4
               The fiscal year 2009 amount included $41 million in the original appropriation for the
             fiscal year and an additional $77 million provided in a supplemental appropriation.




             Page 3                                                        GAO-13-83 State NDF Program
NDF Authorities                                The FREEDOM Support Act provided NDF with a broad mission to fund
                                               bilateral and multilateral nonproliferation and disarmament activities and
                                               annual appropriations bills have consistently granted NDF other key
                                               authorities. NDF has used its authorities under the FREEDOM Support Act
                                               to fund a diverse set of projects. Table 1 outlines NDF activities authorized
                                               by the FREEDOM Support Act and provides examples of the types of
                                               activities NDF has funded. State officials and NDF program documents
                                               have characterized NDF’s mission as focused on funding unanticipated or
                                               unusually difficult projects of high priority to the U.S. government.

Table 1: Authorized NDF Activities under the FREEDOM Support Act

Activities authorized by the FREEDOM Support Act                                  Example of each activity type
1   Support the dismantlement and destruction of nuclear,                         Identified and eliminated conventional weapons, such as
    biological, and chemical weapons; their delivery systems; and                 man-portable air defense systems and certain rocket-
    conventional weapons                                                          propelled grenades, in Libya after the fall of the Ghadafi
                                                                                  regime.
2   Support bilateral and multilateral efforts to halt the proliferation          Negotiated and executed the removal of highly enriched
    of nuclear, biological, and chemical weapons; their delivery                  uranium from Serbia’s closed Vinca Institute to a secure
    systems; related technologies; and other weapons                              facility in Russia where it was converted into standard
                                                                                  nuclear fuel.
3   Establish programs for safeguarding against the proliferation                 Funded the purchase of radiation detection equipment and
    of nuclear, biological, chemical, and other weapons                           training for 10 countries to assist them in locating, securing,
                                                                                  and disposing of high-risk radioactive materials.
4   Establish programs for preventing diversion of weapons-                       Established and equipped a biosecurity and biosafety
    related scientific and technical expertise to terrorist groups or             training center in Jordan to train scientists from the Middle
    to third countries                                                            East and Central Asia on how to safely and securely handle
                                                                                  dangerous pathogens to prevent their proliferation.
5   Establish science and technology centers for the purpose of                   Funded the Iraqi International Center for Science and
    engaging weapons scientists and engineers (in particular                      Industry to support the civilian employment of Iraqi scientists,
    those who were previously involved in the design and                          engineers, and other technicians formerly working in Iraq’s
    production of nuclear, biological, and chemical weapons) in                   weapons of mass destruction programs.
    productive, nonmilitary undertakings
6   Establish programs for facilitating the conversion of military                Removed and destroyed biological and chemical production
    technologies and capabilities and defense industries into                     capabilities at a facility in Russia and reconfigured the facility
    civilian activities                                                           to support non-weapons, civilian-use production.
                                               Source: FREEDOM Support Act and GAO analysis of NDF project documentation.


                                               Figure 1 illustrates the dismantling of a Scud missile as part of an NDF-
                                               funded project in Ukraine. 5




                                               5
                                                The Scud is a short-range, liquid-fueled missile first built by the Soviet Union.




                                               Page 4                                                                       GAO-13-83 State NDF Program
Figure 1: NDF-Funded Dismantling of a Scud Missile in Ukraine




Page 5                                                GAO-13-83 State NDF Program
                                       In addition to the authorities granted to NDF in the FREEDOM Support
                                       Act, annual appropriations bills have also consistently provided NDF with
                                       three key authorities that are designed to increase NDF’s flexibility in
                                       carrying out nonproliferation and disarmament activities around the globe,
                                       as opportunities arise. These include the authority to (1) undertake
                                       projects notwithstanding other provisions of law (notwithstanding
                                       authority); (2) implement projects anywhere in the world or through
                                       international organizations when it is in the national security interest of the
                                       United States to do so, notwithstanding provisions of the FREEDOM
                                       Support Act that limited certain NDF activities to the independent states of
                                       the former Soviet Union (FSU) (geographic authority); and (3) use funding
                                       without restriction to fiscal year (no-year budget authority).


NDF Project Review and                 State uses a multistep process to review NDF project proposals and
Approval Process                       determine which projects to fund, as shown in figure 2.

Figure 2: NDF Project Review and Approval Process




                                       Page 6                                              GAO-13-83 State NDF Program
According to NDF officials, NDF does not typically develop its own project
proposals. Rather, other agencies, such as DOD and DOE, and other
State offices, such as the Office of Export Control Cooperation, submit
project proposals. NDF’s Review Panel, which is chaired by the Assistant
Secretary of State from ISN, reviews these proposals. Two ISN Deputy
Assistant Secretaries of State and the Assistant Secretaries of State from
the Bureau of Political-Military Affairs and the Bureau of Arms Control,
Verification, and Compliance serve as the other voting members on the
panel. Officials from other U.S. agencies, including DOD, DOE, OMB, the
Department of Commerce, and the Department of Homeland Security, as
well as representatives from the National Security Council and U.S.
intelligence community, also attend panel meetings. After reviewing the
project proposals, the voting members of the NDF Review Panel make
recommendations to State’s Under Secretary for Arms Control and
International Security to approve, deny, or defer projects. In the Review
Panel meetings, members can also propose modifications, such as
increasing or decreasing the amount of funding for a project. The Under
Secretary has the final authority to approve a project. NDF officials stated
that in certain cases—for example, if a project is particularly urgent—the
NDF Review Panel may not formally meet to review a proposal before it is
submitted to the Under Secretary. 6 In those cases, NDF instead may
discuss the proposal with other Review Panel agencies in a different
venue, such as at a National Security Council meeting. 7

After the Under Secretary approves a project, but before work begins,
State provides a 15-day advance notification to Congress to inform it of
State’s intent to begin work on the project. 8 As part of the notification,
NDF informs Congress of its intent to obligate a specified amount of funds
on the project. NDF then considers these funds designated for that
project and not available for use on other projects, unless a subsequent
notification is made.


6
 According to State officials, even if the Review Panel does not meet, its voting members
still provide their formal recommendations to the Under Secretary.
7
 The National Security Council is the President’s principal forum for considering national
security and foreign policy matters with his senior national security advisors and cabinet
officials.
8
 Congress, through a provision in annual appropriations acts for Foreign Operations,
requires such notification for projects not previously justified to Congressional
Appropriations Committees. For example, Pub. L. No. 112-74, 125 Stat. 1201 and Pub. L.
No. 107-115, 115 Stat. 2142.




Page 7                                                       GAO-13-83 State NDF Program
NDF Projects   From fiscal years 1994 through 2012, NDF notified Congress of its intent
               to initiate work on 179 projects. NDF subsequently cancelled 19 of these
               projects after their notification and put an additional project on hold
               because of congressional concerns. As of the end of fiscal year 2012,
               NDF had 33 active projects. NDF also reported that, as of the end of
               fiscal year 2012, it had an additional 42 projects for which work was
               completed or cancelled, and the financial review of the projects was
               finished. In accordance with NDF close-out procedures, NDF is in the
               process of seeking approval from the Under Secretary for Arms Control
               and International Security before officially closing them.

               Since the beginning of fiscal year 2007, NDF has notified Congress of its
               intent to initiate work on a total of 24 projects, with a high of 15 in fiscal
               year 2010 and a low of zero in fiscal year 2011. Figure 3 shows the
               number of congressionally-notified projects from fiscal years 2007
               through 2012.

               Figure 3: Congressionally-Notified NDF Projects, Fiscal Years 2007 through 2012




               Note: NDF did not initiate work on one of the projects notified in fiscal year 2010 because of
               congressional concerns.


               NDF funding amounts for projects vary significantly. NDF has notified
               Congress of its intent to spend as much as $50 million to as little as
               $179,000 on individual projects initiated since the beginning of fiscal year



               Page 8                                                               GAO-13-83 State NDF Program
2007. The lengths of projects also vary. For example, since the beginning
of fiscal year 2007, NDF has closed out projects that were completed in
as little as a few months to more than 9 years. In addition, some projects
are follow-up projects that build on projects initiated in earlier fiscal years.
For example, beginning in fiscal year 1998, NDF has undertaken five
separate projects—the most recent of which was initiated in fiscal year
2010—to assist the government of Kazakhstan in shutting down a nuclear
reactor in Aktau.

NDF divides its projects into four categories: (1) destruction and
conversion, (2) safeguards and verification, (3) enforcement and
interdiction, and (4) education and training. Since the beginning of fiscal
year 2007, State has committed the most resources to projects in the
destruction/conversion category. In fiscal years 2007 through 2012, 39
percent of NDF funding for new projects went to projects in this category.
Figure 4 shows a breakdown of funding for NDF among the four project
categories, as well as administrative expenses, for fiscal years 2007
through 2012.

Figure 4: NDF Funding by Project Category, Fiscal Years 2007 through 2012




Note: NDF did not fund any projects in the education/training category in fiscal years 2007 through
2012.




Page 9                                                              GAO-13-83 State NDF Program
                             NDF has several key authorities that provide it significant operational
NDF’s Authorities            flexibility; however, it has not determined its needed carryover balances
Provide It Significant       and it has taken years to close out many of its projects in the absence of
                             guidance for closing them. Annual appropriations bills have consistently
Operational                  provided NDF with three key authorities that it has used to carry out its
Flexibility, but under       activities. First, NDF has used its notwithstanding authority to fund
                             projects in countries where other U.S. programs are barred from
Its No-Year Budget           operating by U.S. sanctions or other legal restrictions. 9 Second, NDF has
Authority, NDF Has           used its geographic authority to fund projects in a range of countries
                             around the globe. 10 Third, NDF has used its no-year budget authority to
Not Determined               carry over balances not designated for specific projects from one year to
Needed Carryover             the next. 11 However, NDF has not determined appropriate levels for these
Balances                     balances, which have increased significantly in the past several years.
                             Additionally, NDF has taken many years to close some projects where
                             work was never started, or was suspended, and has not established
                             guidance for determining when inactive projects should be closed out and
                             unexpended no-year funds made available for other projects.


NDF Used Its                 Annual appropriations acts have consistently granted NDF
Notwithstanding Authority    notwithstanding authority, which allows NDF to undertake projects
to Implement Several         “notwithstanding any other provision of law.” As a result, NDF has the
                             ability to fund projects in countries where other U.S. programs are
Projects Where Laws and      generally barred from operating by U.S. legal restrictions. For example,
Regulations Otherwise        when North Korea agreed to the disablement of its Yongbyon nuclear
Restricted U.S. Assistance   reactor in 2007 after progress in diplomatic talks, NDF was able to fund
                             the project because of its notwithstanding authority, while other U.S.
                             agencies, such as DOD and DOE, could not because various U.S. legal
                             restrictions limited the assistance they could provide the country.



                             9
                              Notwithstanding authority allows NDF to expend funds, “notwithstanding any other
                             provision of law.” This authority allows NDF to fund projects when other forms of U.S.
                             assistance may be prohibited by U.S. sanctions or other legal restrictions.
                             10
                               The geographic authority granted to NDF in annual appropriations measures provides it
                             the ability to fund projects anywhere in the world, when it is deemed to be in the national
                             security interest of the United States to do so, notwithstanding provisions of the
                             FREEDOM Support Act that limit certain NDF activities to the independent states of the
                             FSU.
                             11
                               Funds appropriated with no-year budget authority remain available for obligation for an
                             indefinite period of time. A no-year appropriation is usually identified by language
                             indicating that the appropriation is “to remain available until expended.”




                             Page 10                                                      GAO-13-83 State NDF Program
According to State officials, NDF’s broad notwithstanding authority is
uncommon among U.S. government programs. For example, the 2010
National Defense Authorization Act provided DOD’s Cooperative Threat
Reduction (CTR) program notwithstanding authority for the first time in
the program’s history and granted only limited use of the authority. 12 DOD
cannot use its notwithstanding authority for more than 10 percent of
CTR’s appropriation for a given fiscal year and must meet other
requirements before exercising the authority, such as obtaining
concurrence from the Secretaries of State and Energy.

When seeking to use its notwithstanding authority, NDF requests
approval from the Under Secretary for Arms Control and International
Security. According to the NDF Director, when NDF was established,
State decided that NDF’s notwithstanding authority should, as a matter of
policy, be approved at the Under Secretary level, rather than at a lower
level. 13 State officials said that NDF typically informs Congress of its
intent to use the authority as part of the 15-day congressional notification
process. Although U.S. law does not require that State inform Congress
of NDF’s use of its notwithstanding authority, the conference report
accompanying the fiscal year 2012 Consolidated Appropriations Act
directed the Secretary of State to notify the Committees on Appropriations
in writing, within 5 days of exercising NDF’s notwithstanding authority.
The conference report also directed that the notification include a
justification for the use of the authority.

Since the beginning of fiscal year 2007, NDF has requested and received
approval from the Under Secretary to use its notwithstanding authority for
11 of the 24 projects it initiated. In four cases, NDF identified specific laws
or regulations it needed to overcome using its notwithstanding authority.
In another case, which involved funding for a multinational exercise, NDF
stated that it might need to use its notwithstanding authority depending on
the countries participating in the exercise. In three additional cases, NDF
stated that it had not identified specific legal restrictions that would
necessitate using the authority, but made a general request to the Under



12
 Pub. L. No. 111-84, §1305. CTR, which is implemented by DOD’s Defense Threat
Reduction Agency, is designed to assist countries in securing and eliminating their
weapons of mass destruction and preventing their proliferation.
13
  State noted that legally, notwithstanding authority applies to NDF funds by the terms of
the legislation and does not require a formal determination to rely upon this authority.




Page 11                                                      GAO-13-83 State NDF Program
Secretary for approval given the sensitive nature of the projects. In the
final three cases, NDF requested the use of its notwithstanding authority
for classified projects whose details cannot be publicly reported.

In those cases where NDF requested the use of its notwithstanding
authority to overcome specific laws or regulations, it identified several
different legal restrictions it needed to overcome. For example:

•     NDF requested the use of its notwithstanding authority to initiate work
      on a project in Libya in fiscal year 2012. Among other things, the
      authority was required to overcome restrictions on U.S. security
      assistance to countries that engage in a consistent pattern of gross
      violations of human rights. 14

•     In the case of two projects at the Yongbyon site in North Korea, NDF
      requested the use of its notwithstanding authority to, among other
      things, overcome “Glenn Amendment” restrictions within the Arms
      Export Control Act. The Glenn Amendment triggers U.S. sanctions if
      the President determines that a nonnuclear country (as defined by the
      Nuclear Nonproliferation Treaty) has detonated a nuclear explosive
      device. 15

•     NDF also requested the use of its notwithstanding authority to
      overcome a restriction that the Foreign Assistance Act would have
      imposed on a project in Iraq. The Act restricts U.S. assistance to
      countries that have severed diplomatic relations with the United
      States and which have not entered into a new bilateral assistance
      agreement once diplomatic relations have resumed. 16 At the time of
      the project, there were concerns regarding the status of the United
      States’ bilateral agreement with Iraq.

In addition to using its notwithstanding authority to bypass restrictions on
U.S. assistance to particular countries, NDF has in some cases also used
its notwithstanding authority to overcome laws and regulations pertaining
to contracting and acquisitions. For example, NDF used its
notwithstanding authority on some contracts to overcome Federal


14
    22 U.S.C. 2304.
15
    22 U.S.C. 2799aa-1.
16
    22 U.S.C. 2370(t).




Page 12                                             GAO-13-83 State NDF Program
                           Acquisition Regulation (FAR) competition requirements, according to a
                           2004 State Inspector General report. 17 Additionally, a 2009 National
                           Academies of Science report examining options for strengthening and
                           expanding DOD’s CTR program noted that, because of its
                           notwithstanding authority, NDF is not subject to contracting requirements,
                           including the FAR, which CTR must follow. The report noted that this
                           ability may allow NDF to undertake certain projects more quickly and at
                           less expense than CTR. However, according to State officials, NDF has
                           not typically used the program’s notwithstanding authority to bypass
                           federal contracting laws and regulations. State officials said that, while
                           NDF has almost always relied on sole-source bids, rather than a
                           competitive bidding process, it primarily selected contractors to implement
                           projects using existing flexibilities in the law and regulations available to
                           all agencies. For example, State officials stated that NDF has relied on
                           provisions in the FAR that permit sole-source contracts in situations
                           where there is an urgent and compelling need.

                           In addition to competition requirements, some NDF officials stated that
                           NDF may use its notwithstanding authority to bypass other types of
                           acquisition requirements, such as “Buy America” provisions. For example,
                           one NDF official stated that to expedite work on NDF’s project at the
                           Yongbyon reactor in North Korea, NDF purchased some of the equipment
                           used from China.


NDF’s Geographic           Since 1994, annual appropriations acts have provided NDF with broad
Authority Has Allowed It   geographic authority to fund projects worldwide as nonproliferation and
to Fund Projects around    disarmament opportunities arise. NDF’s geographic authority allows it to
                           fund projects outside the states of the FSU if the Under Secretary for
the World                  Arms Control and International Security makes a determination that it is in
                           the national security interest of the United States to do so. NDF’s
                           authority to fund projects globally since the program’s start in 1994 is in
                           contrast to the authorities of some other U.S. nonproliferation programs.
                           For example, DOD’s CTR program was not authorized to fund any
                           projects outside the FSU until the passage of the Fiscal Year 2004
                           Defense Authorization Bill and continued to face various restrictions on
                           conducting work outside the FSU until 2007. In addition, as we reported in


                           17
                             The United States Department of State and the Broadcasting Board of Governors Office
                           of the Inspector General, Report of Inspection: Bureau of Nonproliferation, ISP-I-05-50
                           (Washington, D.C.: Dec. 2004).




                           Page 13                                                   GAO-13-83 State NDF Program
December 2011, many of DOE’s Defense Nuclear Nonproliferation
programs, which originated in the early 1990s following the dissolution of
the Soviet Union, have focused primarily on improving nuclear security in
Russia. 18 The NDF Director stated that, while NDF has been used to
supplement projects in the FSU or to fill emergency gaps, its primary
emphasis has always been on other parts of the world.

Since 1994, NDF has used its geographic authority to fund projects in
Central and South America, North and Sub-Saharan Africa, Eastern and
Western Europe, the Balkans, the Middle East, and Asia. As shown in
figure 5, NDF has funded projects in several different countries since the
beginning of fiscal year 2007, including Afghanistan, Egypt, Kazakhstan,
North Korea, and Ukraine, among others. It has also funded a limited
number of projects in the United States, including the construction of
training facilities at DOE’s Hazardous Materials Management and
Emergency Response site in Washington State for the purpose of training
foreign nationals.




18
  GAO, Nuclear Nonproliferation: Action Needed to Address NNSA’s Program
Management and Coordination Challenges, GAO-12-71 (Washington, D.C: Dec. 14,
2011).




Page 14                                               GAO-13-83 State NDF Program
Figure 5: Locations of NDF Projects both Before and After Fiscal Year 2007




                                         Note: Some projects were not associated with a specific location and are not shown on the map. In
                                         addition, the locations of some projects are classified and not shown. Finally, in some cases, NDF did
                                         not actually undertake work in a country, but instead provided the country funding to participate in an
                                         activity, such as a multilateral exercise, in a different location.




                                         Page 15                                                              GAO-13-83 State NDF Program
Under Its No-Year Budget     Over the life of the program, Congress has consistently granted NDF no-
Authority, NDF’s Carryover   year budget authority in annual appropriations bills. This authority makes
Balances Have Reached        NDF appropriations available for obligation until expended, rather than
                             requiring them to be obligated within a particular time period, such as a
Historically High Levels     fiscal year. This authority has allowed NDF to carry over balances across
and Are Likely               multiple fiscal years that it has not designated for specific projects. NDF
Understated Because of       considers money to be designated for a specific project and no longer
Delays in Closing Projects   available for use on other projects at the point when it notifies Congress
                             of its intent to fund the project, unless NDF renotifies the funds for the
                             purpose of another project. 19 NDF’s carryover balances have increased
                             over time and are at historically high levels. Figure 6 provides an
                             overview of NDF’s various categories of funding and how NDF
                             accumulates carryover balances. In addition, NDF’s no-year budget
                             authority allows it to close projects and apply the unexpended funds to
                             future projects; however, NDF has sometimes delayed in closing out
                             some projects for many years, including projects where no work ever
                             occurred or was suspended. Until projects are closed, any unexpended
                             project funds are not reported as part of NDF’s carryover balances. As a
                             result, NDF’s carryover balance is likely understated.




                             19
                               For the purposes of this report, we use the term “notified” to indicate funds that have
                             been designated for specific projects. While NDF considers notified funds to be internally
                             committed to a project, notified funds may not necessarily have been obligated for that
                             project through an act creating a legal liability of the U.S. government for payments for
                             goods or services.




                             Page 16                                                      GAO-13-83 State NDF Program
Figure 6: Overview of NDF Funding Categories and Accumulation of Carryover Balances




Carryover Balances Have                NDF’s carryover balances have grown significantly in the past few years
Increased over Time to                 to historically high levels. NDF’s carryover balance peaked at the end of
Historically High Levels, but          fiscal year 2009 at $122 million in unnotified funds, which were carried
NDF Has Not Formally                   over into fiscal year 2010 as shown in figure 7. Unnotified funds include
Determined Needed Reserve              funds never designated for a project, as well as any unobligated and
                                       unexpended project funds that once again become available as unnotified
Amounts


                                       Page 17                                         GAO-13-83 State NDF Program
                                         funds when a project is closed. Before the end of fiscal year 2009, NDF’s
                                         balance carried over into the next fiscal year had been $10 million or
                                         higher three times since the program began in fiscal year 1994 and had
                                         never been higher than $22 million. NDF’s carryover balance was $86
                                         million at the end of fiscal year 2012.

Figure 7: NDF Funds Available for Notification, Fiscal Years 1994 through 2013




                                         Note: NDF funds available for notification are those that have not been designated for a particular
                                         project. New budget authority represents the NDF appropriation amount for the fiscal year plus or
                                         minus any appropriations adjustments, such as supplemental appropriations.
                                         a
                                         The new budget authority amount for fiscal year 2013 reflects NDF’s budget request of $30 million as
                                         NDF has not received its fiscal year 2013 appropriation.



                                         NDF has not established a formal means of determining the amount of
                                         money it needs to carry over from year to year to respond to




                                         Page 18                                                              GAO-13-83 State NDF Program
unanticipated nonproliferation and disarmament opportunities. 20
According to the Assistant Secretary of State for International Security
and Nonproliferation, State management is aware of the growth in NDF’s
carryover balances and is committed to spending them down as
opportunities consistent with the mission of NDF arise.

In the past several years, increases in NDF’s annual appropriations from
levels in earlier fiscal years, as well as the initiation of work on a smaller
number of projects, have contributed to NDF’s increased carryover
balances. As shown in figure 7 above, NDF’s appropriation was never
more than $30 million before fiscal year 2005 and was only higher than
$20 million in one fiscal year. However, from fiscal years 2005 through
2012, NDF’s appropriation has been $30 million or more every year. 21
NDF’s appropriation reached a high of $118 million in fiscal year 2009,
which included $77 million in a supplemental appropriation. NDF also
initiated a limited number of projects in the past 2 years. For example, it
initiated only one new project in fiscal year 2012 and no projects in fiscal
year 2011. In total, NDF notified Congress of 24 projects from fiscal years
2007 through 2012, compared with 63 projects from fiscal years 2001
through 2006. In part, NDF officials stated that the decline in the number
of projects initiated was caused by the creation of other U.S. government
programs that are now able to fund various activities from their own
budgets that might have previously required NDF funding. For example,
NDF officials noted that NDF previously funded certain types of export
control assistance activities that State’s Export Control and Related
Border Security Assistance program is now able to fund and implement.
However, NDF officials noted that while NDF has initiated a smaller
number of projects in fiscal years 2007 through 2012, many of the
projects it has initiated have involved significantly larger amounts of
notified funds than in fiscal years 2001 through 2006. For example, NDF
had only one project with over $10 million in notified funds in fiscal years
2001 through 2006, while it had 11 projects with over $10 million in
notified funds in fiscal years 2007 through 2012.



20
  According to NDF officials, the office seeks to keep about $30 million in unnotified
balances to use as a reserve in the event that unanticipated nonproliferation or
disarmament opportunities arise. NDF officials noted that this amount was an increase
from earlier years of the program, when NDF sought to keep approximately $10 million in
reserve. However, the officials noted that this amount is not a formally established target
and that the office did not use any particular process to develop it.
21
  NDF’s budget request for fiscal year 2013 was $30 million.




Page 19                                                        GAO-13-83 State NDF Program
                                Other U.S. government programs also receive no-year money and have
                                the ability to carry over balances from year to year. We have previously
                                reported on efforts by some of these programs to determine appropriate
                                carryover balance amounts. For example, in contrast with NDF, DOE’s
                                National Nuclear Security Administration has established thresholds for
                                the carryover balances of its Defense Nuclear Nonproliferation
                                programs. 22 These threshold amounts are based upon specified
                                percentages of the total funds available to each of the Defense Nuclear
                                Nonproliferation programs in a given fiscal year. As we reported in
                                December 2011, if programs’ carryover balances exceed these
                                thresholds, they will trigger additional scrutiny by the National Nuclear
                                Security Administration as to whether the carryover balances are
                                appropriate to meet program requirements. 23

NDF Has Taken Years to Close    NDF also maintains a significant amount of funds that it notified to
Some Projects, Delaying         Congress in the past for projects, but has not yet obligated. Of the 32
Unexpended Funds’               active NDF projects initiated in fiscal year 2010 or earlier, 25 percent of
Availability for New Projects   the total notified funds have not yet been obligated. This represents more
                                than $66 million in notified but unobligated funds. As some NDF projects
                                take many years to complete, NDF does not necessarily obligate all funds
                                early in their implementation. However, of the 32 active NDF projects
                                initiated in fiscal year 2010 or earlier, we identified 5 projects for which
                                less than 25 percent of the notified funds had been obligated.

                                Because NDF’s funding is no-year money, NDF can close projects for
                                which it has never started work, or has suspended work, and apply the
                                unexpended funds to future projects. However, NDF has not established
                                guidance for determining when it should close out inactive projects. As a
                                result, NDF funds may be tied up for years in projects where no work is
                                occurring, precluding the funds’ use for other projects. For example, NDF
                                maintains over $24 million in unobligated funds from a $25 million project
                                in North Korea that it notified to Congress in fiscal year 2008. NDF has
                                not obligated any of the funds for this project since North Korea expelled
                                International Atomic Energy Agency inspectors and U.S. monitors from



                                22
                                  The National Nuclear Security Administration implements more than 20 nonproliferation
                                programs worldwide that work to secure nuclear warheads; protect, consolidate, and
                                dispose of weapon-useable nuclear materials; and transition weapons of mass destruction
                                expertise and infrastructure in partner countries to peaceful purposes, among other things.
                                23
                                 GAO-12-71.




                                Page 20                                                      GAO-13-83 State NDF Program
the country in April 2009 and work on the project was abruptly halted.
Additionally, NDF has not yet obligated any of the $750,000 notified in
fiscal year 2005 for a project to support Proliferation Security Initiative
interdiction activities. According to NDF officials, no funds have been
obligated to date because they have not identified any Proliferation
Security Initiative activities that warranted the use of the funds.

NDF has not developed guidance that establishes time frames for closing
cancelled or completed projects to ensure that they are closed out in a
timely manner. NDF data show that in the past, NDF has taken years to
cancel and close some projects where little or no work ended up
occurring. Of the 61 projects NDF has closed out since the beginning of
fiscal year 2007, 16 were cancelled projects for which less than 20
percent of the notified funds were ever obligated and expended. 24 For six
of these cancelled projects, NDF took more than 10 years to close them
out from the date they were initially notified to Congress, and for an
additional 3 projects NDF took more than 5 years to close them out from
the date they were notified to Congress. In total, these 9 projects had
over $8.3 million in notified funds that were never expended.

In addition to cancelled projects, NDF has taken years to close some
completed projects. For example, of the 61 projects NDF has closed out
since the beginning of fiscal year 2007, we identified 13 that NDF closed
out more than 10 years after work on the project was completed and an
additional 18 that NDF closed out more than 5 years after work on the
project was completed. These 31 projects had over $3.5 million in notified
but unexpended funds. The unexpended funds for these cancelled and
completed projects were eventually made available for use on future
projects. However, it can take years from the time projects are cancelled
or completed to the time they are closed out, which can result in an
understatement of the amount of money NDF has available.

NDF officials noted that prior to 2005, NDF took years to close completed
and cancelled projects because it lacked the needed staff. However, NDF
officials stated that since then, the office has hired additional staff and
developed procedures to help ensure that projects are closed out more
quickly. Additionally, NDF officials noted that the office has eliminated its
backlog of projects needing to be closed. However, NDF still has 42



24
 For all but three of these projects, NDF expended none of the notified funds.




Page 21                                                     GAO-13-83 State NDF Program
                          projects for which it has completed all financial close-out activities, but is
                          in the process of seeking approval from the Under Secretary for Arms
                          Control and International Security before closing them and returning the
                          unexpended funds to the NDF account. 25 These 42 projects have a total
                          of over $19 million in unexpended funds that will be added to NDF’s
                          unnotified balances, once they are closed.


                          State has not conducted a program evaluation of NDF and lacks
State Has Not             information that would be useful in doing so. A program evaluation is a
Conducted a Program       systematic study to assess how well a program is working and that can
                          identify lessons learned for future projects. State has developed a new
Evaluation of NDF         policy requiring bureaus to evaluate programs, projects, and activities. To
and Lacks                 comply with this policy, State issued guidance requiring bureaus to submit
                          an evaluation plan for fiscal years 2012 through 2014, identifying the
Information for           programs and projects they plan to evaluate. However, ISN, which
Conducting an             oversees NDF, did not include NDF in its fiscal years 2012 through 2014
                          evaluation plan. Moreover, State currently lacks information, such as the
Evaluation to Inform      results of some projects and lessons learned, that could be used to
NDF’s Program             conduct a program evaluation of NDF and that would help inform the
                          management of the program.
Management
State Has Not Conducted   Since NDF became operational in 1994, State has not conducted a
an NDF Program            program evaluation of NDF, according to ISN and NDF officials. 26
Evaluation                Although NDF reported to Congress in its fiscal year 2013 budget
                          submission that all of its projects are evaluated in-house, these
                          documents are project close-out monitoring reports and not evaluations.
                          As State and other organizations have noted, monitoring and evaluations
                          are conceptually and operationally different. GAO defines evaluation as
                          individual, systematic studies that are conducted periodically or on an as-
                          required basis to assess how well a program is working, while project



                          25
                            NDF informs the Under Secretary by memorandum of the completion or proposed
                          cancellation of a project. The Under Secretary then approves the closure of the project
                          and the return to the NDF account of any unexpended funds that were notified for the
                          project.
                          26
                            While State has never conducted a program evaluation of NDF, NDF has contracted for
                          two evaluations of NDF’s financial management system. The first was completed in May
                          2009. The second is planned for completion in December 2012.




                          Page 22                                                     GAO-13-83 State NDF Program
                             close-out reports consist of formal documentation that indicates
                             completion of the project or phase of the project. 27

                             ISN and NDF officials explained that NDF and its projects have never
                             been subject to a program evaluation because of the unique nature of
                             each project. For example, according to NDF officials, to get one country
                             to agree to dismantle its Scud missiles, NDF agreed to pay for that
                             country’s armed forces to use a labor-intensive method to dismantle the
                             missiles. However, NDF officials also noted that there are common
                             features to many projects that can serve as the basis for lessons learned.
                             Our analysis of NDF’s project database shows that since NDF’s first
                             project in 1994, NDF has implemented a number of similar projects that
                             could have been evaluated to determine the lessons learned for use in
                             present and future projects. For example, NDF has implemented 11
                             destruction and conversion projects involving missiles and rockets, 5 of
                             which involved the destruction of Scud missiles. The first of these missile
                             destruction and conversion projects took place in 1994 and the latest
                             began in 2010. In addition, as noted earlier in this report, NDF has
                             implemented at least five projects involving the shutdown of a nuclear
                             reactor in Kazakhstan.


State Has Developed a        The Government Performance and Results Modernization Act of 2010 28
New Evaluation Policy        strengthened the mandate to evaluate programs, requiring agencies to
Along with Tools to          include a discussion of evaluations in their strategic plans and
                             performance reports. In part to comply with the requirements of this Act,
Facilitate Evaluations and   State established a policy in February 2012 to evaluate programs and
Lessons Learned              projects. In addition, as we reported in May 2012, according to officials
                             from State’s Bureau of International Narcotics and Law Enforcement, the
                             policy was established to comply with a June 2009 directive from the
                             Secretary of State for systematic evaluation and to promote a culture
                             change among program offices. 29 According to State officials, the
                             February 2012 policy superseded an evaluation policy dating from
                             September 2010 that did not fully comply with a recommendation later


                             27
                               For additional information about designing and conducting evaluations, see GAO:
                             Designing Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C: Jan. 2012).
                             28
                              Pub. L. No. 111-352.
                             29
                              See GAO, Foreign Police Assistance: Defined Roles and Improved Information Sharing
                             Could Enhance Interagency Collaboration, GAO-12-534 (Washington, D.C.: May 9, 2012).




                             Page 23                                                  GAO-13-83 State NDF Program
detailed in State’s December 2010 Quadrennial Diplomacy and
Development Review that State adopt an evaluation framework
consistent with that of the U.S. Agency for International Development. 30

State’s 2012 evaluation policy outlines requirements and provides a
framework and justification for evaluations of all State programs, including
both diplomatic and development programs, projects, and activities. For
example, the policy notes that a robust, coordinated, and targeted
evaluation policy is essential to State’s ability to measure and monitor
program performance, document program impact, and identify best
practices and lessons learned. It also states that such a policy can help
assess return on investment and provide input for policy, planning, and
budget decisions. 31

State’s evaluation policy assigns a key role to the bureaus and requires
them to evaluate two to four programs, projects, or activities over a 24-
month period starting in fiscal year 2012 and all large programs, projects,
and activities at least once in their lifetime or every 5 years, whichever is
less. 32 It also

•    requires the bureaus to appoint a coordinator to ensure that the
     bureaus meet the new policy’s requirements;

•    requires bureaus to develop and submit a bureau evaluation plan as
     an annex to their multiyear strategic plans, but gives bureaus flexibility
     in determining the specific programs to evaluate, as well as the timing
     and manner of evaluations they will perform; and



30
  Department of State, Leading Through Civilian Power: The First Quadrennial Diplomacy
and Development Review (Washington, D.C.: 2010). The 2010 Quadrennial Diplomacy
and Development Review was intended as a sweeping review of State’s and the U.S.
Agency for International Development’s core missions. While DOD has conducted such
periodic reviews, this review was the first of its kind undertaken by State.
31
  The terminology that is used to define the types of program evaluations that can be
conducted differs among agencies. For example, State’s policy refers to “performance,”
“impact,” “summative/ex-post,” and “global/regional” evaluations, while GAO-12-208G
refers to “process,” “outcome,” and “net impact” evaluations.
32
  State’s policy defines a “large” program, project, or activity in two ways: one whose
dollar value equals or exceeds the median program, project, or activity size in that bureau,
or one for which the number of full-time staff exceeds the median number of staff
associated with similar individual programs, projects, and activities in that bureau.




Page 24                                                      GAO-13-83 State NDF Program
•   notes that bureaus should integrate evaluation findings into decision
    making about strategies, program priorities, and project design, as
    well as into the planning and budget formulation process.

State’s evaluation policy also draws a clear distinction between
monitoring and evaluation. State defines monitoring as a continual
process designed to assess the progress of a program, project, or
activity. By comparison, evaluations go beyond monitoring to identify the
underlying factors and forces that affect the implementation process, as
well as the efficiency, sustainability, and effectiveness of the intervention
and its outcomes. As our previous work, State, and other organizations
have noted, evaluations also require a measure of independence.
According to State, this can be promoted in several ways, including
entrusting the evaluation to an outside research and evaluation
organization or fostering a professional culture that emphasizes the need
for rigorous and independent evaluations.

To complement the new evaluation policy and provide further direction,
State issued new guidance in March 2012 that describes several types of
evaluations that bureaus can conduct and outlines data collection
methods. The March 2012 guidance also defines the information that
must be included in each bureau evaluation plan. For example, bureaus
must include in the first plan a list of evaluations to be initiated or
completed between fiscal years 2012 and 2014. Bureaus are expected to
update these plans annually, according to State officials.

In addition to the guidance, State has developed or is in the process of
developing other resources and tools to complement and support the new
evaluation policy. These include an internal website containing resources
to assist bureaus with their evaluation responsibilities and the
establishment of a community of practice where officials can share their
expertise and discuss evaluation issues. 33




33
  State has also taken other steps, such as holding an evaluation conference. According
to State officials, the agency is also working to establish two weeklong courses on
evaluations that will be open to all State officials.




Page 25                                                    GAO-13-83 State NDF Program
ISN Did Not Include NDF    ISN submitted its first bureau evaluation plan in April 2012, but the plan
in Its Fiscal Years 2012   did not include any NDF projects. According to ISN officials, the bureau
through 2014 Bureau        had a short amount of time in which to submit its bureau evaluation plan
                           and for that reason the plan focused on programs that already had
Evaluation Plan            projects scheduled for evaluation. After the State evaluation guidance
                           was finalized in late March 2012, the bureaus only had 1 month to submit
                           their bureau evaluation plans for fiscal years 2012 through 2014,
                           according to ISN officials. In canvassing ISN’s five program offices, ISN
                           determined that some offices were already planning evaluations for
                           certain projects within their programs, according to ISN officials and
                           documents. For example, according to the ISN bureau evaluation plan,
                           State’s Global Threat Reduction (GTR) Program plans to contract for four
                           evaluations during the fiscal years 2012 through 2015 period. 34 GTR has
                           in the past contracted for evaluations of its projects in Iraq, Ukraine, and
                           Russia.


State Currently Lacks      State currently lacks information that would be useful in conducting a
Information Useful in      program evaluation of NDF and in improving the management of its
Conducting an NDF          program. NDF uses project close-out reports to document its final
                           monitoring of a project. 35 State’s March 2012 evaluation guidance notes
Program Evaluation and     the importance of preparing good monitoring reports since these both
Managing NDF’s Program     complement evaluations and can provide valuable information for use in
                           preparing evaluations. They can also be a key source of information that
                           can be used to improve the management of a program, such as the



                           34
                             GTR focuses on front-line states like Iraq, Pakistan, and Yemen. It is intended to reduce
                           the threat posed by terrorists or countries of concern seeking to acquire weapons of mass
                           destruction. State’s GTR program is distinct from DOE’s Global Threat Reduction
                           Initiative, which is implemented by the National Nuclear Security Administration. Other
                           program evaluations that ISN has included in its bureau evaluation plan include certain
                           projects managed by the Preventing Nuclear Smuggling Program and the Global Initiative
                           to Combat Nuclear Terrorism. The Preventing Nuclear Smuggling Program assists
                           vulnerable countries in strengthening their capacity to prevent, detect, and respond to
                           nuclear and radioactive smuggling incidents. The Global Initiative to Combat Nuclear
                           Terrorism is a voluntary group of 85 countries that have committed to strengthening their
                           capacity to prevent, detect, and respond to nuclear terrorism. Funding for this initiative
                           supports workshops and training activities focusing primarily on nuclear forensics and
                           detection.
                           35
                             NDF officials have described the project close-out reports as evaluations. However,
                           based on various criteria described in appendix I, we determined that the reports better fit
                           the standard of a project monitoring report. See appendix I for a fuller explanation of our
                           determination.




                           Page 26                                                       GAO-13-83 State NDF Program
                             results of a project and lessons learned. However, NDF’s project close-
                             out reports did not document information that could be useful to NDF and
                             the NDF Review Panel. The reports also varied in content and format.

Project Close-Out Reports    Project management standards note the importance of documenting
Omitted Project Results,     results in project close-out documents, but not all of the project close-out
Lessons Learned, and Other   reports that we examined discussed the results of the project. 36 Of the 23
Information                  project close-out reports that we examined, 2 did not address project
                             results at all. In addition, for the other 21, we found some instances where
                             the discussions of results were fairly minimal and other instances where
                             the reports did not state whether all intended outcomes or goals had been
                             achieved. According to the Project Management Body of Knowledge
                             Guide, a recognized standard for project managers, project close-out
                             documents or reports should include formal documentation that indicates
                             completion of a project, including results. 37 Moreover, according to NDF
                             officials, NDF and the NDF Review Panel consider potential results in
                             determining whether to fund future projects.

                             Project management standards note the importance of documenting
                             project results and entering this information into a database of lessons
                             learned. However, 13 of the 23 project close-out reports that we
                             examined did not discuss lessons learned. Moreover, NDF officials stated
                             that they did not have a database of lessons learned. To document and
                             share lessons learned, NDF officials said that they primarily used informal
                             mechanisms such as e-mails or face-to-face meetings. The Project
                             Management Body of Knowledge Guide notes the importance of
                             documenting lessons learned and entering this information into a lessons-
                             learned database for use in future projects. Some agencies that
                             implement projects or with an interest in communicating lessons learned
                             have formal databases that they use to enter lessons learned and
                             communicate this information to project implementers. For example, the
                             U.S. Agency for International Development and the U.S. Army Center for



                             36
                               To conduct our analysis, we reviewed each project close-out report to determine the
                             presence of a list of key terms developed based on our analysis of project management
                             standards and NDF’s 1994 project suitability guidelines, as well as interviews with NDF
                             officials. Appendix I contains a fuller discussion of the methodology we used to produce
                             our analysis of NDF project close-out reports.
                             37
                              Project Management Institute, A Guide to the Project Management Body of Knowledge,
                             Fourth Edition (2008).




                             Page 27                                                     GAO-13-83 State NDF Program
                                 Lessons Learned have both established lessons-learned databases. 38
                                 State Bureau of Budgeting and Planning officials told us that as part of its
                                 effort to implement the new evaluation policy, State is considering the
                                 establishment of a lessons-learned database that could include
                                 information from NDF.

                                 In addition, the close-out reports often did not address other criteria that
                                 the NDF Review Panel considers in assessing future projects for NDF
                                 funding. For example, 11 of the 23 project close-out reports that we
                                 examined did not discuss cost, and 17 of the 23 did not discuss the
                                 timeliness of the project. In one instance, the final cost of the project was
                                 approximately 66 percent under the amount notified to Congress, but the
                                 close-out report did not provide a reason why this had occurred. Of the 23
                                 reports we examined, 19 did not discuss the appropriateness of using
                                 NDF funding for the project and none discussed the project’s return on
                                 investment. 39 According to the guidelines promulgated by State when
                                 NDF was established in 1994, NDF criteria used to assess a project’s
                                 suitability for NDF funding include the cost and the appropriateness of
                                 using NDF as a source of funding. In addition, according to NDF officials,
                                 the NDF Review Panel also considers the project’s return on investment
                                 and timeliness as part of its criteria. Moreover, according to NDF officials,
                                 the NDF Review Panel has sometimes modified its initial assessment of a
                                 project’s cost based on past experience.

Project Close-Out Reports Also   Although the 14 project close-out reports written by NDF officials were
Varied in Content and Format     more similar in content and format, the other 9 differed considerably. For
                                 example, 10 of the 14 reports written by NDF officials had a section
                                 labeled “summary” or “overview” and 11 of the 14 were in the form of a
                                 memorandum. The reports written by contractors and others varied
                                 considerably. One consisted of a two-page letter from the project
                                 implementer that stated that the report funded by the project had been
                                 completed, but little else. Another consisted of a PowerPoint presentation
                                 written by a foreign government that did not provide many details about
                                 the implementation of the project. NDF officials stated that requiring the



                                 38
                                   The U.S. Agency for International Development implements and manages U.S. foreign
                                 assistance programs. The U.S. Army’s Center for Lessons Learned collects and analyzes
                                 data from current and historical sources and produces and disseminates lessons learned
                                 for commanders and staff.
                                 39
                                  Return on investment is a measure of the benefits gained by implementing a project.




                                 Page 28                                                   GAO-13-83 State NDF Program
             use of a standard format in project close-out reports might not always be
             appropriate or useful given the wide variety of projects that NDF funds
             and undertakes. However, it may be difficult to obtain information useful
             to future evaluations from reports that vary so significantly in content and
             format.

             Recognizing the need for NDF project managers to prepare a close-out
             report to ensure that information is consistently documented, in
             December 2010, NDF established the expectation that NDF project
             managers produce a project close-out report. NDF also produced a
             project management guide designed to encourage project managers to
             standardize their procedures. The NDF project management guide, which
             according to NDF officials is based on the Project Management Body of
             Knowledge Guide, among other things lists the preparation of a project
             close-out report as one of the steps for closing out a project. 40 However,
             NDF officials stated in July 2012 that while project managers are
             expected to write project close-out reports, they are not required to do so.
             In addition, NDF officials stated that NDF encourages but does not
             require the use of the project management guide and the guide does not
             detail the information that project managers need to include in their
             reports or specify the report format. Partly in response to our work, NDF
             officials stated that they plan to develop standard operating procedures to
             address the issues we identified in the project close-out reports, which will
             also include a requirement for project managers to identify lessons
             learned. However, as of November 2012, they had not made any
             changes to their procedures.


             Over its lifetime, NDF has responded to pressing nonproliferation and
Conclusion   disarmament needs, helping to address significant threats to international
             security. To support NDF in accomplishing its mission, U.S. law has
             provided NDF with an unusual degree of flexibility in how it manages its
             resources and conducts its work. While the critical nature of NDF’s
             mission provides a strong rationale for such flexibility, it also increases
             the need for State to effectively manage its program resources to ensure
             that NDF is achieving its intended results. However, State has not taken


             40
               While NDF’s project management guide is based on the Project Management Body of
             Knowledge Guide, according to NDF officials, State as a whole has not adopted the use of
             this guide and the guide is not used as a reference in any of State’s project management
             courses.




             Page 29                                                    GAO-13-83 State NDF Program
                      the necessary steps to do so. For example, unlike some programs, NDF
                      lacks a formal process for determining how much carryover balance it
                      needs to maintain in reserve to meet unanticipated program
                      requirements. Without such a process, NDF cannot know to what extent
                      its carryover balances, which have increased in the past few years to
                      historically high levels, may be exceeding its unanticipated funding needs.
                      In addition, NDF has taken years to close some projects, delaying the
                      availability of unexpended funds for other projects and likely understating
                      NDF’s carryover balances. A methodical process for determining NDF’s
                      needed carryover balances and for closing projects could help ensure
                      that NDF’s budget requests accurately reflect program needs.
                      Additionally, NDF lacks a process to identify and incorporate lessons
                      learned into future projects. State has never performed a program
                      evaluation of NDF in its 18-year history to determine lessons learned for
                      better designing projects that contribute to U.S. nonproliferation goals.
                      State has implemented a new evaluation policy that could encourage the
                      bureaus to more rigorously rationalize and prioritize their resources over
                      time and identify and incorporate lessons learned. Nonetheless, State is
                      not including NDF among the programs to be evaluated during fiscal
                      years 2012 through 2014. Finally, NDF’s project close-out reports could
                      provide useful information to inform future program evaluations’
                      identification of lessons learned that could be systematically incorporated
                      into future projects.


                      To more effectively manage NDF’s resources, increase program
Recommendations for   accountability, and ensure that NDF has the information necessary to
Executive Action      improve program performance, we recommend that the Secretary of
                      State take the following four actions:

                      •   direct NDF to develop a methodology for determining the amount of
                          reserves that it should carry over annually to meet program
                          requirements to address unanticipated nonproliferation and
                          disarmament opportunities;

                      •   direct NDF to develop guidance for determining when inactive NDF
                          projects should be closed and the remaining, unexpended funds
                          made available for use on other projects;

                      •   direct ISN and NDF to periodically and systematically conduct and
                          document program evaluations of NDF;




                      Page 30                                           GAO-13-83 State NDF Program
                     •   direct NDF to revise its project management guide to establish
                         requirements for project managers’ close-out reports to include
                         information useful for improving the management of NDF projects.


                     We provided a draft of our report to DOD, DOE, OMB, and State for their
Agency Comments      review and comment. DOD and OMB did not provide comments. State
and Our Evaluation   provided written comments, which we have reprinted in appendix II. State
                     concurred with all four of our recommendations and identified several
                     actions it intends to take in response to the recommendations. For
                     example, State said that it will direct NDF to develop a methodology that
                     the NDF Review Panel can then use to make an annual recommendation
                     on the appropriate level of carryover balances for the next fiscal year to
                     the Under Secretary for Arms Control and International Security. State
                     also said that NDF has begun implementing the recommendation to
                     revise its project management guidance to establish requirements for
                     close-out reports, by creating a standard operating procedure for these
                     reports. State and DOE provided technical comments, which we
                     incorporated in the report, as appropriate.


                     We are sending copies of this report to interested congressional
                     committees, the secretaries and agency heads of the departments
                     addressed in this report, and other interested parties. In addition, the
                     report is available at no charge on the GAO website at
                     http://www.gao.gov.

                     If you or your staff have any questions about this report, please contact
                     me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices
                     of Congressional Relations and Public Affairs may be found on the last
                     page of this report. GAO staff who made key contributions to this report
                     are listed in appendix III.

                     Sincerely yours,




                     Thomas Melito
                     Director
                     International Affairs and Trade


                     Page 31                                             GAO-13-83 State NDF Program
Appendix I: Scope and Methodology
             Appendix I: Scope and Methodology




             This report examines (1) the Department of State’s (State) use of
             Nonproliferation and Disarmament Fund (NDF) authorities in developing
             and implementing NDF projects and (2) the extent to which State has
             conducted a program evaluation of NDF and used this information to
             improve program performance.

             To assess how State has used NDF’s authorities in developing and
             implementing NDF projects, we obtained program-wide and project-level
             data from NDF’s Financial and Information Management System (FIMS)
             for fiscal years 1994 through 2012. To assess the reliability of data in
             FIMS, we reviewed NDF documentation on the system, reviewed
             previous audits that assessed the reliability of FIMS data, compared FIMS
             data to data from other sources to confirm FIMS data’s accuracy, and
             interviewed cognizant State officials. To gain additional information on the
             reliability of data in FIMS, we met with a private contractor conducting a
             review for NDF under the supervision of the State’s Office of the Inspector
             General. The scope of the contractor’s work included a review of the
             reliability of FIMS data. On the basis of the information we obtained, we
             determined that the FIMS data were sufficiently reliable for our purposes.
             We analyzed NDF program-wide data to determine program
             appropriations, commitments, obligations, and carryover balances for
             fiscal years 1994 through 2012. We analyzed NDF project data to
             determine project funding amounts, locations, objectives, and time frames
             for fiscal years 1994 and 2012. Additionally, we reviewed NDF project
             documentation including project proposals, approval memos, and
             congressional notifications, for all NDF projects initiated since the
             beginning of fiscal year 2007 to assess the types of projects NDF has
             funded and how it used its authorities in developing and implementing
             these projects. To gain additional information on NDF projects, we also
             reviewed State press releases, speeches by State officials, and fact
             sheets describing NDF activities. To identify NDF’s key legal authorities,
             we reviewed relevant laws and regulations, including the FREEDOM
             Support Act and NDF appropriations legislation for fiscal years 1994
             through 2012. Additionally, we examined congressional committee and
             conference reports from 1999 through 2012 to identify relevant
             congressional guidance regarding NDF. We also reviewed key NDF
             documents discussing the program’s authorities, including the 1994
             memorandum pursuant to the FREEDOM Support Act establishing the
             program and the accompanying NDF Guidelines. To gather additional
             information on NDF’s authorities and how it develops and implements
             projects, we conducted a series of interviews with NDF officials and also
             met with officials from other agencies that proposed or implemented NDF
             projects, including the Departments of Defense and Energy. We also


             Page 32                                           GAO-13-83 State NDF Program
Appendix I: Scope and Methodology




interviewed officials from the Office of Management and Budget to gain
additional information on NDF’s budget planning process. Finally, we
reviewed previous GAO reports, as well as reports by the State Inspector
General, the Congressional Research Service, and the National
Academies of Science, to identify relevant findings regarding NDF and
related U.S. nonproliferation and disarmament programs.

To assess the extent to which State has evaluated NDF and used this
information to improve program performance, we interviewed State
officials with the Bureaus of International Security and Nonproliferation
(ISN) and Budgeting and Planning. We also obtained copies of State’s
February 2012 evaluation policy and March 2012 evaluation guidance, as
well as a copy of ISN’s April 2012 bureau evaluation plan. NDF officials
described their project close-out reports as evaluations, but based on our
discussion with State ISN and Budgeting and Planning officials, our
review of GAO reports discussing evaluations, and State’s February 2012
evaluation policy, we determined that NDF’s project close-out reports fit
more closely the standard of a monitoring report. GAO defines
evaluations as individual, systematic studies that are conducted
periodically or on an as-required basis to assess how well a program is
working. State’s evaluation policy notes that in addition to assessing the
progress of a program, project, or activity, evaluations go beyond
monitoring to identify the underlying factors and forces that affect the
implementation process, as well as the efficiency, sustainability, and
effectiveness of the program or project and its outcomes. As such, State’s
policy draws a clear distinction between evaluation and monitoring. As
previous GAO reports, State, and other organizations have noted,
evaluations require a measure of independence, which can be promoted
in several ways, including entrusting the evaluation to an outside research
and evaluation organization or fostering a professional culture that
emphasizes the need for rigorous and independent evaluations. By
comparison, State defines monitoring as a continual process designed to
assess the progress of a program, project, or activity. The Project
Management Body of Knowledge Guide 1 notes that project close-out
documentation consists of formal documentation indicating the
completion of a project or phase of a project. For all these reasons, on the
basis of our analysis of NDF’s project close-out reports, we made the


1
 The Project Management Body of Knowledge Guide is a product of the Project
Management Institute and is a recognized standard for the project management
profession.




Page 33                                                  GAO-13-83 State NDF Program
Appendix I: Scope and Methodology




determination that NDF’s project close-out reports better fit the standard
of a monitoring report than an evaluation. While project close-out reports
serve a different purpose from evaluations, based on our review of the
Project Management Body of Knowledge Guide and NDF’s Project
Management Guide, we determined that we could assess the project
close-out reports to determine their usefulness in enabling NDF to
improve its management of the program. For this purpose, we obtained a
judgmental sample of 23 project close-out reports—14 of which we
selected and 9 of which State selected. In selecting our sample, we chose
only to consider projects that NDF had closed out since the beginning of
fiscal year 2007—of which there were 61—in order to ensure that all
close-out documentation was completed for the projects. Our selection
criteria for our sample included project cost, location, and type. For
example, we selected a variety of projects from all four categories of
projects that NDF funds—destruction and conversion, safeguards and
verification, enforcement and interdiction, and education and training.
State selected its projects using similar criteria; however, State did not
limit itself to projects that were closed out. In some cases, State selected
projects where work was completed, but the project was not yet officially
closed out. In reviewing the documentation for the projects State
selected, we determined that these projects were broadly similar to the
ones that we selected and the inclusion of these projects in our analysis
did not alter our overall findings or compromise the independence of our
work. To conduct our analysis of the close-out reports, we developed a
list of key terms, such as “results,” “completion,” and “lessons learned.”
Our inclusion of these terms was based on our analysis of project
management standards, which note the importance of the project close-
out process in the project management cycle and the importance of
obtaining information about the results of the project and lessons learned.
We also included other terms such as “cost,” “timeliness,” “on time,”
“return on investment,” and “appropriateness of using NDF funding.” We
included these terms because NDF officials told us that NDF and the NDF
Review Panel include these criteria in determining a project’s suitability
for NDF funding. Because NDF does not have any requirement to use a
standard terminology in its reports, we used a dictionary to obtain other
synonyms of these terms as well. We examined each of the project close-
out reports to determine the presence of these key terms. We also
examined each of the project close-out reports to determine the author,
content, and format. We did this on the basis of discussions with NDF
officials, who told us that they had established an expectation that NDF
project managers complete a project close-out report and had developed
a project manager’s guide that contained a checklist. While NDF does not
have a requirement for project reports to be written in a standard format,


Page 34                                           GAO-13-83 State NDF Program
Appendix I: Scope and Methodology




we determined that the close-out reports that we had examined varied
widely in their content and format and concluded that such variety could
make it more difficult for evaluators to extract key information from these
reports. After completing our initial review, the lead analyst submitted the
results of his work and the methodology used to two additional levels of
review. These reviewers were asked to validate the methodology and
results. The sample of 23 project close-out reports cannot be generalized
to the entire population of NDF project reports for the period in our review.
We also reviewed NDF’s Project Management Guide to determine the
extent to which NDF has established specific requirements or guidance
regarding how project close-out reporting should be conducted. To obtain
the list of 11 similar missile destruction and conversion related projects,
we conducted a word search of NDF’s projects using the key terms
“missiles” and “rockets.”

We conducted this performance audit from March 2012 through
November 2012 in accordance with generally accepted government
auditing standards. These standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.




Page 35                                            GAO-13-83 State NDF Program
Appendix II: Comments from the Department
             Appendix II: Comments from the Department
             of State



of State




             Page 36                                     GAO-13-83 State NDF Program
Appendix II: Comments from the Department
of State




Page 37                                     GAO-13-83 State NDF Program
Appendix II: Comments from the Department
of State




Page 38                                     GAO-13-83 State NDF Program
Appendix III: GAO Contact and Staff
                  Appendix III: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Thomas Melito, (202) 512-9601 or melitot@gao.gov
GAO Contact
                  In addition to the contact named above, the following staff made key
Staff             contributions to this report: Jeff Phillips, Assistant Director; Lynn Cothern;
Acknowledgments   Martin De Alteriis; Mark Dowling; José M. Peña, III; and Ryan Vaughan.
                  Etana Finkler and Jeremy Sebest provided graphics support and Debbie
                  Chung provided editorial assistance. Julie Hirshen and Julia Jebo Grant
                  also provided additional technical assistance.




(320899)
                  Page 39                                             GAO-13-83 State NDF Program
                      The Government Accountability Office, the audit, evaluation, and
GAO’s Mission         investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
                      Website: http://www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in   E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.