oversight

Modernizing the Nuclear Security Enterprise: NNSA's Reviews of Budget Estimates and Decisions on Resource Trade-offs Need Strengthening

Published by the Government Accountability Office on 2012-07-31.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States Government Accountability Office

GAO          Report to the Subcommittee on Energy
             and Water Development, Committee on
             Appropriations, U.S. Senate


July 2012
             MODERNIZING THE
             NUCLEAR SECURITY
             ENTERPRISE
             NNSA’s Reviews of
             Budget Estimates and
             Decisions on
             Resource Trade-offs
             Need Strengthening




GAO-12-806
                                              July 2012

                                              MODERNIZING THE NUCLEAR SECURITY
                                              ENTERPRISE
                                              NNSA’s Reviews of Budget Estimates and Decisions
Highlights of GAO-12-806, a report to the
                                              on Resource Trade-offs Need Strengthening
Subcommittee on Energy and Water
Development, Committee on Appropriations,
U.S. Senate



Why GAO Did This Study                        What GAO Found
NNSA, a semiautonomous agency                 The National Nuclear Security Administration’s (NNSA) planning, programming,
within DOE, is responsible for the            budgeting, and evaluation (PPBE) process provides a framework for the agency
nation’s nuclear weapons,                     to plan, prioritize, fund, and evaluate its program activities. Formal policies guide
nonproliferation, and naval reactors          NNSA and management and operating (M&O) contractors through each of four
programs. Since its inception in 2000,        phases of the agency’s PPBE cycle—planning, programming, budgeting, and
the agency has faced challenges in its        evaluation. These phases appear to be sequential, but the process is continuous
ability to accurately identify the costs of   and concurrent because of the amount of time required to develop priorities and
major projects. In addition, both the         review resource requirements, with at least two phases ongoing at any time.
DOE Inspector General, in 2003, and
GAO, in 2007, reported concerns with          NNSA does not thoroughly review budget estimates before it incorporates them
NNSA’s PPBE process, specifically in          into its proposed annual budget. Instead, NNSA relies on informal,
how NNSA validates budget estimates           undocumented reviews of such estimates and its own budget validation review
and decides on resource allocations or        process—the formal process for assessing budget estimates. Neither of these
trade-offs.                                   processes adheres to Department of Energy (DOE) Order 130.1, which defines
GAO was asked to review how NNSA              departmental provisions for the thoroughness, timing, and documentation of
manages programming and budgeting             budget reviews. NNSA officials said the agency does not follow the order
through its PPBE process. GAO                 because it expired in 2003. Nevertheless, the order is listed as current on DOE’s
examined (1) the current structure of         website, and a senior DOE budget official confirmed that it remains in effect,
NNSA’s PPBE process, (2) the extent           although it is outdated in terminology and organizational structure. Additionally,
to which NNSA reviews its budget              according to NNSA officials, the agency’s trust in its contractors minimizes the
estimates, and (3) how NNSA decides           need for formal review of its budget estimates. GAO identified three key
on resource trade-offs in its PPBE            problems in NNSA’s budget validation review process. First, this process does
process. To carry out its work, GAO           not inform NNSA, DOE, Office of Management and Budget, or congressional
reviewed NNSA policies, instructions,         budget development decisions because it occurs too late in the budget cycle—
guidance, and internal reports                after the submission of the President’s budget to Congress. Second, this process
documenting the agency’s PPBE                 is not sufficiently thorough to ensure the credibility and reliability of NNSA’s
process and interviewed NNSA, DOE,            budget because it is limited to assessing the processes used to develop budget
and M&O contractor officials.                 estimates rather than the accuracy of the resulting estimates and is conducted
What GAO Recommends                           for a small portion of NNSA’s budget—approximately 1.5 percent of which
                                              received such review in 2011. Third, other weaknesses in this process, such as
GAO recommends that, among other              no formal evaluative mechanism to determine if corrective actions were taken in
things, DOE update the departmental           response to previous findings, limit the process’s effectiveness in assessing
order for budget reviews, improve the         NNSA’s budget estimates.
formal process for reviewing budget
estimates, and reinstitute an                 NNSA uses a variety of management tools to decide on resource trade-offs
independent analytical capability. The        during the programming phase of the PPBE process. One of these tools,
agency agreed in principle with six           integrated priority lists—which rank program activities according to their
recommendations but not with one to           importance for meeting mission requirements—is to provide senior managers
consolidate various integrated priority       with an understanding of how various funding scenarios would affect program
lists. GAO continues to believe this          activities. However, NNSA has weakened its ability to gauge the effects of
recommendation has merit as                   resource trade-offs. For example, in 2010, NNSA disbanded its Office of
discussed in the report.                      Integration and Assessments, created in response to DOE Inspector General and
                                              GAO recommendations that NNSA establish an independent analysis unit to
                                              perform such functions as reviewing proposals for program activities and
                                              verifying cost estimates. NNSA agreed with these recommendations and, in
                                              2009, instituted the office to identify, analyze, and assess options for deciding on
View GAO-12-806. For more information,
contact Gene Aloise at (202) 512-3841 or      resource trade-offs. Without an independent analytical capability, NNSA may
aloisee@gao.gov.                              have difficulty making the best decisions about what activities to fund and
                                              whether they are affordable.
                                                                                       United States Government Accountability Office
Contents


Letter                                                                                        1
               Background                                                                     5
               NNSA Has Established a Formal PPBE Process That Includes Four
                 Defined Phases                                                               6
               NNSA Does Not Thoroughly Review Budget Estimates When
                 Developing Its Annual Budget                                               10
               NNSA Has Implemented Some Tools to Support Decision Making
                 on Resource Trade-offs but Has Stopped Using or Developing
                 Other Capabilities                                                         17
               Conclusions                                                                  23
               Recommendations for Executive Action                                         25
               Agency Comments and Our Evaluation                                           26

Appendix I     Objectives, Scope, and Methodology                                           29



Appendix II    Comments from the U.S. Department of Energy                                  31



Appendix III   GAO Contact and Staff Acknowledgments                                        37



Figure
               Figure 1: NNSA PPBE Schedule                                                   7




               Page i                     GAO-12-806 Modernizing the Nuclear Security Enterprise
Abbreviations

AFR               Administrator’s Final Recommendation
DOD               Department of Defense
DOE               Department of Energy
M&O               management and operating
NNSA              National Nuclear Security Administration
OMB               Office of Management and Budget
PPBE              planning, programming, budgeting, and evaluation

This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                            GAO-12-806 Modernizing the Nuclear Security Enterprise
United States Government Accountability Office
Washington, DC 20548




                                   July 31, 2012

                                   The Honorable Dianne Feinstein
                                   Chairman
                                   The Honorable Lamar Alexander
                                   Ranking Member
                                   Subcommittee on Energy and Water Development
                                   Committee on Appropriations
                                   United States Senate

                                   The National Nuclear Security Administration (NNSA)—a
                                   semiautonomous agency within the Department of Energy (DOE)—
                                   manages the nation’s nuclear security programs, the primary goals of
                                   which are to maintain the safety, security, and effectiveness of the U.S.
                                   nuclear weapons stockpile; reduce the global threat posed by nuclear
                                   proliferation and terrorism; and provide nuclear propulsion systems for the
                                   U.S. Navy. To manage these programs, NNSA oversees eight separate
                                   sites—collectively known as the nuclear security enterprise—that are
                                   managed and operated by private contractors, known as management
                                   and operating (M&O) contractors. 1 Among other things, these contractors
                                   operate and maintain the government-owned facilities and infrastructure
                                   deemed necessary to support the nuclear security enterprise and to
                                   support the capabilities to conduct scientific, technical, engineering, and
                                   production activities. To fund these activities, the President requested
                                   more than $11.5 billion for NNSA in his fiscal year 2013 budget
                                   submission to Congress. Sound management practices, such as linking
                                   program activities to defined objectives and ensuring the reliability of
                                   budget estimates, can help NNSA use agency resources wisely. It is
                                   particularly important for NNSA to embrace this approach because, as
                                   our past work has shown, it has faced numerous challenges in accurately
                                   estimating the costs of major projects since its inception in 2000. For
                                   example, we reported in March of this year that the estimated cost to
                                   construct the Chemistry and Metallurgy Research Replacement Nuclear
                                   Facility at NNSA’s Los Alamos National Laboratory rose from between
                                   $745 million and $975 million in 2005 to between $3.7 billion and $5.8




                                   1
                                       These sites include three national laboratories, four production plants, and one test site.




                                   Page 1                                 GAO-12-806 Modernizing the Nuclear Security Enterprise
billion in 2010—a near sixfold increase. 2 In addition, in February of this
year we testified that the estimated cost to construct the modern Uranium
Processing Facility at NNSA’s Y-12 National Security Complex jumped
from between $600 million and $1.1 billion in 2004 to between $4.2 billion
and $6.5 billion in 2011—a near sevenfold increase. 3 Furthermore, in
2000 and 2009, respectively, we reported that NNSA’s efforts to extend
the operational lives of nuclear weapons experienced a $300 million cost
increase for the refurbishment of one warhead and a $70 million increase
for another. 4 Given NNSA’s record of weak management of major
projects, we believe improved federal oversight of NNSA’s modernization
of the nuclear security enterprise is critical to ensuring that resources are
spent in as an effective and efficient manner as possible.

Under section 3252 of the National Defense Authorization Act for Fiscal
Year 2000—the NNSA Act—NNSA was required to develop a planning,
programming, and budgeting process that operated under sound financial
and fiscal management principles. 5 The NNSA Act also requires NNSA to
annually submit to Congress a Future Years Nuclear Security Program
plan that details NNSA’s planned expenditures for the next 5 years. The
value of this plan depends, in part, on the accuracy of the cost estimates
supporting each program. A realistic cost estimate provides a basis for
accurate budgeting and effective resource allocation, which increases the
probability of a program’s success in meeting its goals. NNSA’s first
Administrator indicated that he intended to comply with the NNSA Act by
instituting a planning, programming, budgeting, and evaluation (PPBE)
process—which uses short- and long-term planning to define program
priorities and match them to available budgetary resources—that is
similar to the Department of Defense’s (DOD) budgetary process. As we


2
 GAO, Modernizing the Nuclear Security Enterprise: New Plutonium Research Facility at
Los Alamos May Not Meet All Mission Needs, GAO-12-337 (Washington, D.C.: Mar. 26,
2012).
3
  GAO, National Nuclear Security Administration: Observations on NNSA’s Management
and Oversight of the Nuclear Security Enterprise, GAO-12-473T (Washington, D.C.: Feb.
16, 2012).
4
  GAO, Nuclear Weapons: Improved Management Needed to Implement Stockpile
Stewardship Program Effectively; GAO-01-48 (Washington, D.C.: Dec. 14, 2000), and
GAO, Nuclear Weapons: NNSA and DOD Need to More Effectively Manage the Stockpile
Life Extension Program, GAO-09-385 (Washington, D.C.: Mar. 2, 2009).
5
  NNSA was created by the National Defense Authorization Act for Fiscal Year 2000 (Pub.
L. No. 106-65, § 3201 et seq. [1999]).




Page 2                            GAO-12-806 Modernizing the Nuclear Security Enterprise
have reported, DOD’s PPBE process logically attempts to plan programs
and develop budgets based on high-level strategic goals, but at times it
has been poorly implemented. 6 For example, DOD decision makers often
commit to more programs than the department can reasonably support.
Nonetheless, DOD’s process has the potential to provide comparisons of
the cost-effectiveness of program activities and to help managers develop
the kinds of detailed program and budget plans called for in the NNSA
Act. However, both the DOE Inspector General and GAO have reported
on concerns with the development and implementation of NNSA’s PPBE
process. 7 Specifically, these reports identified deficiencies in how NNSA
ensures the validity of its budget estimates and how it decides to allocate
its resources (resource trade-offs). In our January 2007 report, we
recommended, among other things, that NNSA complete all of its PPBE
policy guidance. In response, NNSA issued PPBE policy in the form of
policy letters that identify the responsibilities of NNSA management,
program and site offices, and contractors throughout the agency’s PPBE
cycle.

In this context, you asked us to determine how NNSA manages its
programming and budgeting activities through its PPBE process. For this
report, our objectives were to examine (1) the current structure of NNSA’s
PPBE process, (2) the extent to which NNSA reviews its budget
estimates, and (3) how NNSA decides on resource trade-offs in its PPBE
process.

To address these objectives, we analyzed NNSA documentation—
including policies, instructions, guidance, and internal reports—related to
the implementation of NNSA’s PPBE policy across NNSA, as well as
program-specific documentation from the Offices of Defense Programs,
Defense Nuclear Nonproliferation, and Naval Reactors; these three
offices correspond to NNSA’s primary missions and collectively account
for approximately 85 percent of the President’s fiscal year 2013 NNSA



6
 GAO, Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve
Major Weapon System Program Outcomes, GAO-08-619 (Washington, D.C.: July 2,
2008), and GAO, DOD Weapon Systems: Missed Trade-off Opportunities During
Requirements Reviews, GAO-11-502 (Washington, D.C.: June 16, 2011).
7
 GAO, National Nuclear Security Administration: Additional Actions Needed to Improve
Management of the Nation’s Nuclear Programs, GAO-07-36 (Washington, D.C.: Jan. 19,
2007); and National Nuclear Security Administration’s Planning, Programming, Budgeting,
and Evaluation Process, DOE/IG-0614, August 2003.




Page 3                            GAO-12-806 Modernizing the Nuclear Security Enterprise
budget submission to Congress. 8 To examine the current structure of
NNSA’s PPBE process, we reviewed NNSA policy documents, guidance,
and instructions. To determine the extent to which NNSA reviews its
budget estimates, we reviewed NNSA’s formal budget validation review
process and associated internal NNSA reports and met with officials
involved in the development, oversight, or execution of NNSA budget
estimate reviews. To identify the tools NNSA uses to decide on resource
trade-offs, we analyzed how NNSA determines which program activities it
considers necessary to achieve NNSA goals. Additionally, to address all
these objectives, we met with officials from the NNSA Offices of the
Administrator, Management and Budget, Defense Programs, Defense
Nuclear Nonproliferation, Naval Reactors, and Acquisition and Project
Management; the site offices for Los Alamos, Sandia, and the Y-12
National Security Complex; and the Naval Reactors Laboratory Field
Office; DOE officials from the Office of Budget; and M&O contractor
officials from Los Alamos and Sandia National Laboratories, the Y-12
National Security Complex, and Bettis Atomic Power Laboratory. 9
Because NNSA’s Office of Naval Reactors is organized as a separate
entity within NNSA reporting both to NNSA and the U.S. Navy, we also
met with Navy officials from its offices of Financial Management and
Budgeting, and Cost Engineering and Industrial Analysis. Appendix I
contains a detailed description of our objectives, scope and methodology.

We conducted this performance audit from July 2011 to July 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




8
  Six other NNSA offices are responsible for carrying out additional program activities:
Management and Budget, Emergency Operations, Defense Nuclear Security, Chief
Information Officer, Counterterrorism and Counterproliferation, and Acquisition and Project
Management.
9
  We selected a nonprobability sample of NNSA sites based on the levels of funding they
receive from NNSA for the programs included in the scope of our work. Because we used
a nonprobability sample, the results we obtained from these sites are not generalizable
across the nuclear security enterprise; however, they did provide us with specific
examples of how NNSA conducts PPBE activities at the site level.




Page 4                             GAO-12-806 Modernizing the Nuclear Security Enterprise
             NNSA oversees programs to carry out its missions for nuclear weapons,
Background   nonproliferation, and naval nuclear propulsion, for which the President’s
             budget submission to Congress requested more than $11.5 billion for
             fiscal year 2013—about 42 percent of DOE’s total budget. NNSA has
             primary mission responsibilities in three areas. First, it is responsible for
             providing the United States with safe, secure, and reliable nuclear
             weapons in the absence of underground nuclear testing and maintains
             core competencies in nuclear weapons science, technology, and
             engineering. Second, NNSA implements key U.S. government nuclear
             security, nonproliferation, and arms control activities, including securing
             vulnerable nuclear and radiological material at facilities throughout the
             world; removing plutonium and highly enriched uranium from partner
             countries; eliminating U.S. nuclear material declared surplus to defense
             needs; negotiating and providing the technical capability to verify arms
             control treaties and agreements; strengthening other countries’ capacities
             to implement nonproliferation obligations; and enhancing other nations’
             capabilities to deter and detect illicit movement of nuclear and radiological
             materials. Third, NNSA provides the research, development, design, and
             operational support for militarily effective naval nuclear propulsion plants,
             as well as enriched uranium for fabrication into fuel for the Navy’s
             propulsion reactors. NNSA receives four congressional appropriations to
             fund its activities, three of which align with its primary missions—
             Weapons Activities (for Defense Programs), Defense Nuclear
             Nonproliferation, Naval Reactors—and one that funds its management
             activities—Office of the Administrator. 10

             Since the Manhattan Project produced the first atomic bomb during World
             War II, NNSA, DOE, and predecessor agencies have depended on the
             expertise of private firms, universities, and others to carry out research
             and development work and efficiently operate the government-owned,
             contractor-operated facilities necessary for the nation’s nuclear defense.
             NNSA conducts its activities at research and development laboratories,
             production plants, and other facilities (collectively referred to as the
             nuclear security enterprise). Specifically, NNSA operates three national
             laboratories that design and ensure the reliability of nuclear weapons—
             Lawrence Livermore National Laboratory, California; Los Alamos National
             Laboratory, New Mexico; and the Sandia National Laboratories, New


             10
               The Weapons Activities appropriation also funds the Emergency Operations, Defense
             Nuclear Security, Chief Information Officer, Counterterrorism and Counterproliferation,
             and Acquisition and Project Management program offices.




             Page 5                             GAO-12-806 Modernizing the Nuclear Security Enterprise
                        Mexico and California; and four nuclear weapons production sites—the
                        Pantex Plant, Texas; the Y-12 National Security Complex, Tennessee;
                        the Kansas City Plant, Missouri; and the Savannah River Site, South
                        Carolina; as well as the Nevada National Security Site.

                        NNSA’s relationship with its contractors has been formally established
                        over the years through its M&O contracts—contracting strategies that
                        give these contractors responsibility to carry out major portions of NNSA’s
                        missions and apply their scientific, technical, and management
                        expertise. 11 M&O contractors at NNSA sites operate under NNSA’s
                        direction and oversight but largely independently of one another. Various
                        headquarters organizations within NNSA develop policies and NNSA site
                        offices, colocated with NNSA’s sites, conduct day-to-day oversight of the
                        M&O contractors, and evaluate the M&O contractors’ performance in
                        carrying out the sites’ missions.


                        According to NNSA’s policy, the NNSA PPBE process is composed of
NNSA Has                four phases—planning, programming, budgeting, and evaluation—and
Established a Formal    their associated activities. The different phases of PPBE appear
                        sequential, but because of the amount of time required to develop and
PPBE Process That       review resource requirements, the process is continuous and concurrent
Includes Four Defined   with at least two phases ongoing at any given time, including phases for
Phases                  different fiscal years. Figure 1 shows the four phases and the months
                        during which each phase is scheduled to occur, according to NNSA
                        policies and guidance.




                        11
                           Management and operating contracts are agreements under which the government
                        contracts for the operation, maintenance, or support, on its behalf, of a government-owned
                        or -controlled research, development, special production, or testing establishment wholly
                        or principally devoted to one or more of the major programs of the contracting federal
                        agency. Federal Acquisition Regulation, 48 C.F.R. § 17.601.




                        Page 6                             GAO-12-806 Modernizing the Nuclear Security Enterprise
Figure 1: NNSA PPBE Schedule




                               a
                                NNSA policy for budget validation calls for validation activities to begin in the April-May time frame
                               and continue to September each year. However, in practice, the schedule of these activities can vary
                               from year to year. For example, the NNSA memorandum initiating the budget validation process in
                               2011 scheduled validation activities to occur from June to October of that year.


                               Planning. According to NNSA policy, in this phase, scheduled to begin
                               annually in November, NNSA is to identify the goals it needs to achieve
                               over the next 5 years and the program activities needed to meet these
                               goals. According to NNSA officials, these goals are defined in a variety of
                               documents, including presidential directives, policy statements, and DOE
                               and NNSA strategic plans. This phase begins with the issuance of
                               NNSA’s annual Strategic Planning Guidance, which provides any updates
                               to the strategic plans and identifies any emerging issues. The NNSA
                               program offices use this guidance to conduct their own internal planning
                               processes, update their multiyear plans, including revising or adding
                               program activities needed to meet the agency’s goals.




                               Page 7                                  GAO-12-806 Modernizing the Nuclear Security Enterprise
Programming. According to NNSA policy, in this phase, scheduled to
begin annually in February, NNSA is to determine which program
activities and funding levels it will include in its budget proposal to DOE
for the fiscal year beginning in October of the following calendar year.
This determination is based on analysis of the activities’ estimated costs,
as well as the need to meet the NNSA goals defined in the planning
process. To determine these activities, NNSA program offices are to work
with their contractors to obtain estimates for the cost of the program
activities identified in the planning phase and determine how to
accomplish these activities within anticipated funding levels, which are
defined in annual NNSA Program and Fiscal Guidance. NNSA program
offices are to then rank these activities in order of priority for meeting
program goals and document these decisions in integrated priority lists.
These lists can include proposed program activities above the anticipated
funding levels specified in NNSA guidance—these proposed activities are
known as unfunded requirements. Using these lists, as well as other
briefing materials, a group of senior NNSA officials including the heads of
all program offices—the Program Review Council—then is to meet with
the Principal Deputy Administrator to discuss and defend each program’s
proposed program activities. After reviewing the deliberations of the
Program Review Council and the associated documents provided by the
program offices, the NNSA Administrator is to decide on resources trade-
offs that result in the combination of program activities that best meet
NNSA’s goals over the 5-year period covered by the Future Years
Nuclear Security Program plan. The Administrator is responsible for
issuing the Administrator’s Final Recommendations (AFR), scheduled to
be completed in May at the conclusion of the programming phase, to
document NNSA’s justification for its priorities and to serve as the basis
for the agency’s participation in DOE’s program review process, the
Strategic Resources Review.

Budgeting. According to NNSA policy, this phase is to integrate NNSA
planning and programming priorities and budget estimates into DOE’s
departmental budget process and consists of the following three parts:

•   Budget formulation. During formulation, which is scheduled to begin
    annually in July for the fiscal year beginning in October of the
    following calendar year, NNSA submits its proposed budget to DOE
    and participates in the Strategic Resources Review. If DOE’s budget
    deliberations result in changes to NNSA’s proposed budget, NNSA
    may have to rebalance its work activities. In September each year,
    DOE submits its proposed budget to the Office of Management and
    Budget (OMB) for review. Depending on OMB revisions, NNSA may



Page 8                       GAO-12-806 Modernizing the Nuclear Security Enterprise
     need to again revise its work activities. These revisions are
     incorporated into the President’s final budget request for DOE, which
     is submitted to Congress in February.

•    Budget validation. According to NNSA guidance, the agency uses its
     budget validation review process to ensure its budget request is
     consistent with NNSA priorities and its budget estimating processes
     are reasonable. NNSA policy calls for NNSA’s Office of PPBE to
     manage a three-phase process of validating approximately 20 percent
     of NNSA’s programs each year, so that 100 percent of its budget is
     validated every 5 years. 12 Programs to undergo validations are to be
     determined by a combination of factors, including Program Managers’
     request, Administrator direction, and/or significant external
     interest/high program visibility. During Phase I of the process,
     scheduled for completion in June, before the beginning of the fiscal
     year in October, program officials determine if their activities conform
     with strategic guidance and program plans and review their methods
     for formulating budgets. In Phase II, scheduled annually for July to
     September, NNSA contractors or program offices, whichever
     developed the budget estimates, conduct a self-assessment of their
     budget planning, formulation, and cost-estimating processes. Phase II
     self-assessments are to be reviewed by a team—known as a
     validation review team—that comprises NNSA headquarters and site
     office staff. During Phase III, scheduled to occur from July through
     August, these validation review teams also review the cost-estimating
     practices used by the NNSA contractors and program offices.
     Importantly, NNSA’s validation guidance emphasizes that reviews
     should focus on the processes used to formulate budget plans and
     derive budget estimates rather than on the accuracy of the resulting
     estimates. According to this guidance, validation review teams are to
     issue a report on their findings on Phases II and III in September to
     inform NNSA, DOE, and OMB decisions for the following year’s
     budget cycle.

•    Budget execution. According to NNSA policy, during this process,
     DOE and NNSA are to allocate, distribute, and control funds to
     achieve the priorities established in the programming phase, and to
     maintain the fiscal limits set up in the budgeting phase, which are



12
  NNSA’s Office of PPBE is organizationally located within the NNSA Office of
Management and Budget.




Page 9                            GAO-12-806 Modernizing the Nuclear Security Enterprise
                                subject to appropriation of funds by Congress. The execution
                                coincides with the fiscal year and commences once appropriated
                                funds become available—whether by appropriation or Continuing
                                Resolution—at the beginning of the fiscal year every October.

                            Evaluation. According to NNSA policy, NNSA is to employ an ongoing
                            cycle of evaluations to review program performance. Evaluations are to
                            include annual and quarterly NNSA performance reviews, performance
                            reviews conducted as part of the Government Performance and Results
                            Act, reviews conducted by OMB, and DOE oversight activities.


                            NNSA does not thoroughly review budget estimates before it incorporates
NNSA Does Not               them into its annual budget request. Instead, it relies on undocumented,
Thoroughly Review           informal reviews of these estimates by site and headquarters program
                            office officials and the formal budget validation reviews, which conclude
Budget Estimates            after the submission of the President’s budget to Congress. Neither of
When Developing Its         these processes meets DOE provisions for ensuring the credibility and
Annual Budget               reliability of agency budgets, as defined in DOE Order 130.1. According
                            to senior NNSA officials, NNSA does not comply with DOE Order 130.1
                            because it believes the order expired in 2003 and therefore no longer
                            applies to NNSA budget activities. Furthermore, they stated that the need
                            for a formal review of budget estimates is minimized, in part, because of
                            the inherent trust between NNSA and its M&O contractors. Additionally,
                            we identified three key problems in NNSA’s formal budget validation
                            review process: it occurs too late to affect budget decisions, is not
                            sufficiently thorough, and includes other weaknesses that limit its
                            effectiveness.


NNSA’s Process for          NNSA does not have a thorough, documented process for assessing the
Reviewing Budget            validity of its budget estimates prior to their inclusion in the President’s
Estimates Is Not Thorough   budget submission to Congress, thereby limiting the reliability and
                            credibility of the budget submission. Specifically, according to NNSA
or Documented               officials from NNSA’s Offices of Management and Budget, Defense
                            Programs; Defense Nuclear Nonproliferation; and the site offices for Los
                            Alamos, Sandia, and Y-12, during the programming phase of PPBE, site
                            and headquarters program office officials conduct informal,
                            undocumented reviews of the budget estimates that M&O contractors
                            submitted to determine their reasonableness, though some officials noted




                            Page 10                       GAO-12-806 Modernizing the Nuclear Security Enterprise
that the level of review may vary across site and headquarters program
offices. 13 According to these officials, this informal review is often
conducted by comparing current budget estimates with those from
previous years because the work is largely the same from year to year. If
the estimates are similar, and no major programmatic change has taken
place, site office and headquarters program office officials said that they
generally view these budget estimates as reasonable for inclusion in
NNSA’s budget estimate. However, site office officials told us that their
ability to thoroughly review budget estimates is limited. For example,
according to NNSA officials at the Los Alamos Site Office, they do not
have the personnel needed or the time, because of other laboratory
management responsibilities, to oversee the laboratory’s budget
estimation practices. They told us that only one dedicated budget analyst
is employed at the site office and, because of insufficient personnel
resources in the office, a majority of this analyst’s time is spent
conducting work that is not directly related to budget oversight. NNSA
officials from the Y-12 Site Office also told us that they informally review
budget estimates when they initially submit them to headquarters
program offices. However, they also stated that they become more
involved in reviewing budget estimates when the agency is formulating its
final budget submission, and the M&O contractors are asked to develop
multiple iterations of budget estimates based on various hypothetical
funding scenarios. However, these officials also stated that their reviews
are not documented. NNSA officials from Defense Programs’ Office of
Analysis and Evaluation told us that the presence of certified cost
engineers—individuals with professional certification in the field of cost
assessment and program management—at the NNSA site offices could
enhance NNSA’s ability to understand how M&O contractors and
programs develop budget estimates and assess those estimates.

The practices the site and headquarters program offices follow do not
align with the criteria for thoroughness or documentation established in
DOE Order 130.1. Specifically, DOE Order 130.1 states that contractor-
developed budget estimates should be thoroughly reviewed and deemed



13
   According to NNSA officials, site office officials conduct these informal reviews of the
budget estimates for activities funded by the Weapons Activities appropriation because
they are developed by the M&O contractors. Conversely, they stated that the budget
estimates for the Defense Nuclear Nonproliferation Appropriation are developed by
program federal employees at NNSA Headquarters. For these activities, contractors
compete for projects based upon submitted proposals and funding availability.




Page 11                              GAO-12-806 Modernizing the Nuclear Security Enterprise
reasonable prior to their inclusion in agency budgets and that these
reviews should be documented. Senior officials from NNSA’s Office of
Management and Budget told us that the agency does not strictly adhere
to DOE Order 130.1 because it believes that the order has expired and no
longer applies to NNSA budget activities. According to these officials, this
order expired in 2003, and they are unaware of any other DOE or federal
government requirement to conduct budget validation reviews. 14 They
further stated that NNSA is conducting budget validation reviews only
because it considers them to be a good business practice and that NNSA
will work with DOE on updating the order if DOE initiates that process.
NNSA officials stated that, if DOE updated and reissued DOE Order
130.1, it would comply to the extent that it had the resources to do so.
However, DOE Order 130.1 remains listed on DOE’s “All Current
Directives” website, and a senior DOE budget official told us that DOE
Order 130.1 remains an active order. Additionally, this official stated that
a key principle of DOE Order 130.1—federal oversight of contractors’
practices for budget formulation—is appropriate and valid. This official
noted, however, that the order is outdated in terms of the terminology it
uses to describe DOE—it was issued in 1995, predating the 2000
establishment of NNSA—and should be updated to reflect the
department’s current organizational structure. Furthermore, in March
2009, we issued a cost-estimating guide—a compilation of cost-
estimating best practices drawn from across industry and government—in
which we reported that validation is considered a best practice to ensure
that cost data are credible and reliable for use in justifying estimates to
agency management. 15 As a result, NNSA’s site and headquarters
program office reviews of budget estimates are neither thorough nor
documented. According to the Principal Deputy Administrator, NNSA
continues to face challenges in moving away from its historical process
for developing budgets based solely on the unreviewed estimates
produced by NNSA M&O contractors and that NNSA’s practices for
understanding its program activity costs are not as sufficient as they need
to be.

In contrast, NNSA’s Office of Naval Reactors’ is jointly staffed and funded
by both NNSA and the Navy and is therefore subject to naval and DOD,


14
     DOE Notice 251.45 extended DOE Order 130.1 until May 1, 2003.
15
  GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and
Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009).




Page 12                            GAO-12-806 Modernizing the Nuclear Security Enterprise
as well as NNSA, standards for reviewing contractor-developed budget
estimates. The Office of Naval Reactors conducts a semiannual
process—known as budget confirmation—to review all contractor-
developed budget estimates. This review is conducted and documented
by NNSA technical experts and approved by the Director of the Office of
Naval Reactors; this director manages both NNSA and the Navy’s
activities within the office and has final budgetary decision authority.
Officials in NNSA’s Office of Management and Budget told us that the
Office of Naval Reactors’ process is much more rigorous than that used
by other NNSA program offices we reviewed. Furthermore, NNSA has
exempted the Office of Naval Reactors from NNSA’s formal budget
validation review process because of management’s confidence in the
quality of the office’s budget confirmation process.

Senior officials in NNSA’s Office of Management and Budget told us that
NNSA does not have the financial and personnel resources needed to
conduct budget estimate reviews with the same rigor as the Navy and
DOD. Furthermore, these officials said, the need for a formal review of
M&O contractor-developed budget estimates is minimized within NNSA
because site office officials have historical knowledge of work with
NNSA’s M&O contractors that allows them to assess the reasonableness
of M&O contractor-developed budget estimates without conducting a
formal review and because of the “inherent trust” between NNSA and its
M&O contractors that results from its contracting strategy with them.
Specifically, one of these officials stated that, to a large extent, only the
M&O contractors are in a position to know the scientific and engineering
details of nuclear weapons and the associated work scope and funding
necessary to ensure their safety and reliability. However, for the last 10
years, we have reported that NNSA has significant weaknesses in its
ability to control costs and effectively manage its M&O contractors. 16 We
are therefore concerned that NNSA management continues to deny the
need for NNSA to improve its processes for developing credible and
reliable budget estimates.




16
     GAO-01-48, GAO-07-36, and GAO-12-337.




Page 13                          GAO-12-806 Modernizing the Nuclear Security Enterprise
NNSA’s Formal Budget        We identified three key problems in NNSA’s annual budget validation
Validation Review Process   review process—its formal process for assessing M&O contractor- and
Occurs Too Late to Affect   program-developed budget estimates. First, NNSA’s annual budget
                            validation review process occurs too late in the budget cycle to inform
Budget Decisions and Is     NNSA, DOE, OMB, and congressional budget development or
Not Sufficiently Thorough   appropriations decisions. DOE Order 130.1, which is referenced in
                            NNSA’s policy for its budget validation review process, states that
                            agencies should thoroughly review budget estimates before using these
                            estimates to develop budgets. However, NNSA’s Phase II and Phase III
                            budget validation reviews are scheduled to begin 5 months after the
                            President submits his budget to Congress. Additionally, during each of
                            the past four budget validation cycles, NNSA did not complete its budget
                            validation reports for at least 12 months following the President’s budget
                            submission to Congress and at least 4 months after the beginning of the
                            fiscal year for which NNSA reviewed the budget estimates. Therefore,
                            Congress considered the budget request for NNSA and appropriated
                            funds to it, and NNSA executed these funds to M&O contractors, before
                            NNSA had published the results of the budget validation reviews.
                            Because of their timing, NNSA’s budget validation reviews cannot inform
                            NNSA’s budget development, DOE or OMB reviews, or Congress’
                            appropriation processes. According to NNSA policy, the timing of NNSA’s
                            budget validation review process is designed to inform the NNSA, DOE,
                            and OMB budgeting processes for the fiscal year following that for which
                            the budget validation reviews were conducted. However, the timing of the
                            publication of the budget validation review reports for each of the last 4
                            years precluded even such delayed consideration because they were
                            issued following the OMB budget formulation process for the following
                            fiscal year.

                            Second, NNSA’s budget validation review process is not sufficiently
                            thorough to ensure the credibility and reliability of NNSA’s budget. DOE
                            Order 130.1 states that budgets should be based on budget estimates
                            that have been thoroughly reviewed by site and headquarters program
                            offices. However, NNSA’s budget validation review process is limited to
                            assessing the processes M&O contractors and programs used to develop
                            budget estimates rather than the accuracy of the resulting budget
                            estimates. 17 NNSA’s 2010 budget validation review guidance states that


                            17
                              The criteria used by validation review teams to assess the reasonableness of budget
                            estimation processes include completeness, consistency, reasonableness, and
                            documentation.




                            Page 14                           GAO-12-806 Modernizing the Nuclear Security Enterprise
the agency lacks the resources and expertise needed to thoroughly
evaluate the accuracy of budget estimates on its own and therefore relies
on assessments of the reasonableness of the processes used by M&O
contractors to develop budget estimates. NNSA officials from the Los
Alamos and Y-12 Site Offices told us that they believe the budget
validation review process would benefit NNSA more if it more thoroughly
assessed the budgetary processes M&O contractors used to develop
their budget estimates. Furthermore, NNSA policy and budget validation
review guidance stipulate that 20 percent of the agency’s programs
should be reviewed annually to help ensure its validity, but NNSA’s formal
validation process actually results in significantly smaller portion of its
budget being reviewed. For example, in 2011, NNSA’s annual budget
validation guidance identified four programs subject to budget validation
review—the Engineering Campaign, Nuclear Counterterrorism Incident
Response, Global Threat Reduction Initiative, and Fissile Materials
Disposition—each of which is conducted at multiple NNSA sites. 18
However, NNSA conducted validation reviews at only one site for each of
these programs, which resulted in a formal validation review of
approximately 12, 21, 15, and 4 percent of each of those programs’ total
budgets, respectively, which, together, constituted 1.5 percent of NNSA’s
budget request for fiscal year 2012.

Third, other weaknesses in NNSA’s budget validation review process limit
its effectiveness as a resource to assess the validity of its budget
estimates. In particular, NNSA workgroups that reviewed the 2007 and
2008 budget validation review cycles recommended that NNSA formally
evaluate the status of recommendations made during previous budget
validation reviews. However, NNSA has not incorporated a formal
mechanism for implementing an evaluation into its budget validation
review process. NNSA officials at the Los Alamos and Y-12 site offices
also told us that not having such an evaluative mechanism was a



18
   The Engineering Campaign provides modern components and engineering science
capabilities to ensure the safety, security, reliability, and performance of the stockpile. The
Nuclear Counterterrorism Incident Response program responds to and mitigates nuclear
and radiological incidents worldwide. The Global Threat Reduction Initiative works in the
United States and internationally to convert research reactors and medical isotope
production processes from the use of highly enriched uranium, remove and dispose of
excess nuclear and radiological materials, and protect high-priority nuclear and
radiological sources from theft. The Fissile Materials Disposition program supports the
downblending of highly enriched uranium that is excess to U.S. defense needs and is
planning to assist Russia in modifying reactors in that country.




Page 15                              GAO-12-806 Modernizing the Nuclear Security Enterprise
weakness in NNSA’s budget validation process. Without a formal
mechanism, NNSA is limited in its ability to measure (1) any progress
M&O contractors or programs have made in their processes for
estimating budgets in response to recommendations from previous
budget validation reviews and (2) the effectiveness of NNSA’s budget
validation review process. For example, a 2010 budget validation review
of the Readiness Campaign recommended that the program more
formally document its budget processes, guidance, and estimating
assumptions. 19 Furthermore, a 2009 budget validation review of the
Elimination of Weapons Grade Plutonium Production program found that
the program could not provide documentation of its internal budget
processes. 20 However, in both instances, NNSA did not follow up to
determine if the programs had addressed these concerns during
subsequent budget validation reviews. Additionally, budget validation
reviews do not always include recommendations to improve M&O
contractor or program processes for estimating budgets when they
identify potentially serious weaknesses in those M&O contractor’s or
programs’ ability to develop cost estimates. For example, according to a
2010 budget validation review of budget estimation activities for the
Nonproliferation and Verification Research and Development program at
Sandia National Laboratories, six of the eight projects reviewed lacked
sufficient documentation to support their cost estimates, including two that
lacked any supporting documentation. 21 The report noted the importance
of credible cost estimates, but it did not formally recommend any remedial
improvements and rated the overall processes used to develop those cost
estimates as satisfactory. Additionally, NNSA officials in the Defense
Programs’ Office of Analysis and Evaluation told us that the cost
information used to support budget validation review reports is often
flawed or nonexistent.



19
  The Readiness Campaign identifies, develops, and deploys new or enhanced
processes, technologies and capabilities to meet current nuclear weapon design,
production, and dismantlement needs and provide quick response to national security
requirements.
20
   The Elimination of Weapons Grade Plutonium Production program seeks to facilitate
the closure of Russian nuclear reactors by building or refurbishing replacement fossil fuel
plants.
21
   The Nonproliferation and Verification Research and Development program’s mission is
to conduct needs-driven research, development, testing, and evaluation of new
technologies that are intended to strengthen the United States’ ability to prevent and
respond to nuclear, chemical, and biological attacks.




Page 16                             GAO-12-806 Modernizing the Nuclear Security Enterprise
                          During the programming phase of PPBE, NNSA uses a variety of
NNSA Has                  management tools, such as integrated priority lists and requirements and
Implemented Some          resources assessments, to support its programming phase and assist
                          senior managers in making decisions on resource trade-offs. However, it
Tools to Support          has stopped using these capabilities or developing others.
Decision Making on
Resource Trade-offs
but Has Stopped
Using or Developing
Other Capabilities
NNSA Has Developed and    NNSA uses the following management tools to decide on resource trade-
Implemented Some Tools    offs during the programming phase of its PPBE process:
to Assist Management in
                          Integrated priority lists. NNSA’s policy for the programming phase
Deciding on Resource      stipulates that each of NNSA’s nine program offices is to annually
Trade-offs                develop an integrated priority list that ranks program activities according
                          to their importance for meeting mission requirements. These lists provide
                          senior NNSA and DOE managers with an understanding of how various
                          funding scenarios would affect program activities. Specifically, these lists
                          rank the priority of program activities that are within anticipated
                          appropriation levels—which are of the highest priority—as well as those
                          that NNSA would fund if the appropriation levels were sufficiently high to
                          do so. For example, the program activity listed last on an integrated
                          priority list would be the first to forgo funding if appropriation levels are
                          lower than anticipated. Conversely, these lists define program activities—
                          unfunded requirements—that would be funded if appropriation levels are
                          higher than anticipated. NNSA instructions for the programming phase
                          stipulate that the agency is to combine the nine program office-developed
                          integrated priority lists into four that correspond to the four congressional
                          appropriations NNSA receives. Three of the integrated priority lists—
                          those of the Offices of the Administrator, Defense Nuclear
                          Nonproliferation, and Naval Reactors—correspond directly to specific
                          appropriations, 22 but NNSA does not combine the six others that
                          represent activities funded by the Weapons Activities appropriation into a


                          22
                            According to NNSA officials, the integrated priority list developed by NNSA’s Office of
                          Management and Budget includes all activities funded by the Office of the Administrator
                          appropriation.




                          Page 17                             GAO-12-806 Modernizing the Nuclear Security Enterprise
single integrated priority list. NNSA officials stated that these six others
represent separate and distinct mission areas and only the Administrator
can decide on the resource trade-offs among them. Of the six program
offices funded by the Weapons Activities appropriation, Defense
Programs accounts for a large majority—approximately 82 percent—of
the funding requested in NNSA’s fiscal year 2013 budget submission to
Congress. According to officials in NNSA’s Office of Management and
Budget, the Administrator is responsible for deciding on how to allocate
resources across program offices. However, these decisions are not
documented or reflected in a single integrated priority list for program
activities funded by the Weapons Activities appropriation. By not
combining these lists, NNSA is limiting the formal documentation
available to inform DOE about how various Weapons Activities
appropriation funding scenarios would affect the program activities
conducted by these six program offices. Specifically, because these six
lists are not integrated, it is unclear which program activities would be
affected by changes to appropriation levels or which programs across the
six lists are of the highest priority.

Requirements and Resources Assessments. During the 2010 and 2011
programming cycles, NNSA used its Requirements and Resources
Assessment process to independently assess the need to conduct new
program activities and unfunded requirements. According to the NNSA
handbook for this process, officials in NNSA’s Office of Management and
Budget review program offices’ budget submissions for activities that are
either above anticipated funding levels or are for new activities. For these
assessments, officials are to analyze specific factors related to these
activities, such as their need for meeting agency priorities and the
reasonableness of the assumptions used to produce their budget
estimates. The objective of this process is to ensure that new program
activities and unfunded requirements are needed to meet NNSA priorities.
For example, according to officials in NNSA’s Office of Management and
Budget, the use of the Requirements and Resources Assessment
process was a contributing factor in reducing the amount of unfunded
program activities included in NNSA’s budget from approximately $1
billion for fiscal year 2012 to approximately $80 million for fiscal year
2013. Furthermore, draft NNSA guidance states that the process has
identified inconsistencies in the quality of estimates and the level of
insight and understanding program managers have regarding the fidelity
of the estimates supporting their budgets. According to officials in NNSA’s
Office of Management and Budget, this process is a simple and effective
tool for providing management with additional information on the need to
conduct proposed new program activities or unfunded requirements.


Page 18                       GAO-12-806 Modernizing the Nuclear Security Enterprise
However, these officials also stated that this process is time-consuming
and would not be practical or efficient to apply to the entirety of NNSA
program activities because it was designed to assess program
components rather than entire programs; they added that other types of
program reviews or validations would be better suited for conducting
program needs analysis on an enterprise-wide basis. Additionally,
because the NNSA Office of Integration and Assessments, which was
responsible for conducting these assessments, was dissolved in 2010,
officials in NNSA’s Office of Management and Budget told us that they
may discontinue the use of Requirements and Resources Assessment
process in future programming cycles. Furthermore, in the current austere
budget environment, they do not foresee any programs proposing
activities that are either new or above anticipated funding levels.
Therefore, the continued use of this process in future programming cycles
is uncertain. However, we believe that NNSA has demonstrated this
process can be an important tool for assessing the necessity to fund
certain activities in order to meet its mission requirements.

Enterprise Portfolio Analysis Tool. NNSA’s Office of Defense Programs is
implementing a data system—the Enterprise Portfolio Analysis Tool—
designed to provide a consistent framework for managing the PPBE
process within Defense Programs, which accounts for 54 percent, or $6.2
billion, of the President’s $11.5 billion fiscal year 2013 budget request for
NNSA. As we testified in February 2012, a tool such as this could help
NNSA obtain the basic data it needs to make informed management
decisions, determine return on investment, and identify opportunities for
cost saving. 23 Currently, this tool includes a mechanism to identify when
decisions on resource trade-offs must be made if, for example, M&O
contractor-developed budget estimates for program requirements exceed
the budget targets NNSA provided for those programs. Additionally, the
tool is to incorporate Defense Programs’ common work activity
structure—known as its work breakdown structure—to facilitate an
analysis of consistent budget data from across the NNSA enterprise.
Specifically, the tool may allow Defense Programs managers to compare
the budget estimates for analogous activities across the nuclear security
enterprise regardless of which M&O contractor or program is conducting
them. Furthermore, Defense Programs officials stated that they eventually
plan to use this tool to compare budget estimates of program activities



23
     GAO-12-473T.




Page 19                       GAO-12-806 Modernizing the Nuclear Security Enterprise
                            with the amounts the programs ultimately expended, but they said that
                            the introduction of this capability is not imminent. According to Defense
                            Programs and M&O contractor officials, the implementation of this tool is
                            placing an additional labor burden on NNSA M&O contractors because of
                            the quantity of historical budget data that need to be entered into it.
                            However, according to Defense Programs officials, once these initial
                            historical data are entered, the M&O contractors will need to annually
                            update the system with the most recent year’s data.


NNSA Does Not Use           NNSA no longer has an independent analytical capability to perform such
Capabilities That Were      functions as reviewing proposals for program activities and verifying cost
Previously Implemented or   estimates. In addition, since 2009, the NNSA Administrator has not
                            formally documented his decisions on resource trade-offs at the close of
Partially Developed to      the programming phase in the AFR. Furthermore, NNSA has not
Support Its Programming     completed cost estimating guidance to assist NNSA program managers in
Activities                  identifying reliable M&O contractor practices for estimating costs for
                            operations and sustainment activities. By not using these capabilities,
                            NNSA has reduced its ability to decide on resource trade-offs because it
                            has not enhanced, made formal, or implemented capabilities that it had
                            already, to varying degrees, developed or used.

Independent Analytical      The DOE Inspector General and GAO, recommended in 2003 and 2007,
Capability                  respectively, that NNSA establish an independent analysis unit to perform
                            such functions as reviewing proposals for program activities and verifying
                            cost estimates. 24 NNSA agreed with these recommendations and, in
                            2009, instituted the Office of Integration and Assessments to identify,
                            analyze, assess, and present to senior NNSA management options for
                            managing its programs and making decisions on resource trade-offs. The
                            specific responsibilities of this office included analyzing program
                            performance, evaluating programming and funding alternatives, and
                            assessing the implementation and effectiveness of process improvement
                            initiatives. Furthermore, this office managed the Requirements and
                            Resources Assessment process during the 2010 programming cycle.
                            However, NNSA disbanded the office in 2010, 18 months after it was
                            formally created. NNSA officials also told us that it was never properly
                            staffed, which thereby limited its effectiveness.




                            24
                                 DOE/IG-0614 and GAO-07-36.




                            Page 20                           GAO-12-806 Modernizing the Nuclear Security Enterprise
In the memorandum establishing the Office of Integration and
Assessments, NNSA stated that it expected the office, in conjunction with
DOE’s Office of Cost Analysis, to provide DOD-like analytical resources
across NNSA. 25 Since then, however, DOE has also eliminated its Office
of Cost Analysis. With both of these offices now gone, neither NNSA nor
DOE has independent cost assessment or program evaluation
capabilities analogous to those of DOD. In contrast, because NNSA’s
Office of Naval Reactors is organized as a separate entity within NNSA
reporting both to NNSA and the U.S. Navy, unlike the rest of NNSA, it is
subject to the Navy’s independent analytical resources, such as
assessments by the Naval Sea Systems Command’s Office of Cost
Engineering and Industrial Analysis and the Naval Center for Cost
Analysis, which conduct independent reviews and analyses of program
cost estimates. Furthermore, DOD has an Office of Cost Assessment and
Program Evaluation, which has a similar function but with a purview that
extends across DOD, including the Navy. These layers of independent
review approximate NNSA’s vision for independent analysis, as described
in the memorandum establishing the Office of Integration and
Assessments. Following the dissolution of the Office of Integration and
Assessments in 2010, NNSA’s Office of Defense Programs created the
Office of Analysis and Evaluation to conduct similar program review
functions. However, the capabilities of this office are limited by several
factors. For example, because the office is positioned within Defense
Programs, it does not have purview to conduct analysis on any of NNSA’s
other programs, which, in total, constitute nearly half of the agency’s
budget request for fiscal year 2013. Additionally, according to Defense
Programs officials, this office does not have the capability to self-initiate
reviews of programs but rather is instructed by Defense Programs’
management on what activities to assess, thereby limiting the office’s
independence. Furthermore, NNSA officials from this office stated that
properly staffing the office remains a challenge because many qualified
individuals left DOE and NNSA when they eliminated the Offices of Cost
Analysis and Integration and Assessments, respectively.

Even though NNSA has had difficulty in maintaining an agencywide
independent analytical capability, NNSA’s Principal Deputy Administrator
told us that NNSA remains supportive of the concept of an independent


25
   DOE established the Office of Cost Analysis in order to improve the department’s cost-
estimating capabilities and better ensure that its project cost estimates are reliable by
providing a new independent cost-estimating function for the department.




Page 21                            GAO-12-806 Modernizing the Nuclear Security Enterprise
                           analytical unit to conduct assessments of programs agencywide.
                           However, senior NNSA officials told us that creating and developing the
                           capabilities of such an office would be difficult in the current budget
                           environment and that therefore NNSA has no current plans to institute
                           such a capability.

Administrator’s Final      The NNSA Administrator has not formally documented his decisions on
Recommendations            resource trade-offs at the close of the programming phase in the AFR
                           since 2009, which is inconsistent with NNSA policy and instructions.
                           When issued, this document articulated the Administrator’s rationale and
                           methodology for deciding on resource trade-offs during the programming
                           phase of the PPBE process—which one senior official in NNSA’s Office of
                           Management and Budget described as an important component of the
                           PPBE process—to support in his budget proposal to DOE and to better
                           facilitate NNSA’s participation in DOE’s Strategic Resources Review.
                           According to senior NNSA officials, the Administrator considered the AFR
                           to be a useful management tool but decided to discontinue issuing it
                           because of concerns that its contents, which are predecisional Executive
                           Branch deliberative material and embargoed from public release by OMB
                           Circular A-11, could be leaked and thereby reduce the flexibility of DOE
                           and OMB in making final decisions regarding the President’s Budget.
                           Instead of the AFR, the Administrator now develops an internal document
                           called “Administrator’s Preliminary Decisions,” which is not required in
                           NNSA policy, guidance, or instructions; contains more generalized
                           information; and does not have the rationales, methodologies, and
                           justifications for decision making on resource trade-offs that were
                           previously incorporated into the AFR.

Issuance of Supplemental   NNSA developed a draft guide—the Program Managers’ Guide to
Guide on Cost Estimating   Understanding and Reviewing Cost Estimates for Operations and
                           Sustainment Activities—in 2010 to assist NNSA program managers in
                           identifying reliable M&O contractor practices for estimating costs for
                           operations and sustainment activities—activities not related to
                           construction; according to this guide, these activities constitute
                           approximately 80 percent of NNSA’s annual budget. This guide was also
                           created to supplement the information provided in NNSA’s Business
                           Operating Procedure 50.005, Establishment of an Independent Cost
                           Estimate Policy and interim Cost Estimating Guide 50.005, which




                           Page 22                     GAO-12-806 Modernizing the Nuclear Security Enterprise
              identifies best practices for preparing cost estimates. 26 However, the
              Program Managers’ Guide to Understanding and Reviewing Cost
              Estimates for Operations and Sustainment Activities was largely
              completed but never finalized before NNSA dissolved the Office of
              Integration and Assessments, which had drafted the guide, and NNSA
              officials said the agency has no plans to complete or issue it. According to
              officials in NNSA’s Office of Management and Budget, NNSA drafted this
              guide because it recognized that supplemental information focused on
              operations and sustainment activities cost estimates—the development of
              which, according to this guide, are not governed by any specific NNSA
              guidance or processes—could enhance the tools available to program
              managers in evaluating cost estimates and how they are translated into
              budget estimates. The objective of the guide was to provide an instructive
              document to facilitate program managers’ ability to understand what
              constitutes a rigorous process for ensuring quality operations and
              sustainment cost estimates on an ongoing basis and evaluate the
              reasonableness those estimates. This guide also defined key
              components of cost estimating to clarify the responsibilities and
              expectations of NNSA program managers and included instructions for
              how NNSA program managers can assess the quality of budget
              estimates submitted by M&O contractors. NNSA officials with Defense
              Programs’ Office of Analysis and Evaluation told us that additional
              guidance on how to assess the costs of operations and sustainment
              activities could enhance program managers’ ability to assess the
              reliability and credibility of cost estimates.


              NNSA has established a formal four-phase PPBE process that uses
Conclusions   short- and long-term planning to define program priorities and match them
              to available budgetary resources. However, DOE and NNSA have not
              taken adequate steps to make this process as effective and efficient as
              possible. In particular, DOE Order 130.1, which defines DOE’s provisions
              for budget activities, references outdated terminology and organizations
              that no longer exist within the department, leading to confusion regarding


              26
                 The information included in NNSA’s interim Cost Estimating Guide 50.005 is applicable
              to all cost estimating activities, but one senior official in NNSA’s Office of Acquisition and
              Project Management told us that it is most useful for construction and capital asset
              projects, which are land, structures, equipment, and information technology used by the
              federal government and have an estimated useful life of 2 years or more, as well as
              strategic assets that are unique physical or intellectual property that is of long-term or
              ongoing value to an enterprise.




              Page 23                              GAO-12-806 Modernizing the Nuclear Security Enterprise
the order’s applicability and requirement for implementation. As a result,
NNSA believes that the order has expired and that it is not required to
adhere to its provisions. By not adhering to these provisions, NNSA is
reducing the credibility of its budget proposals. Moreover, NNSA’s
process for developing budget estimates continues to rely heavily on its
M&O contractors to develop budget estimates without an effective,
thorough review of the validity of those estimates. Without thorough
reviews by site and headquarters program offices of budget estimates,
NNSA cannot have a high level of confidence in its budget estimates or in
its ability to make informed decisions on resource trade-offs and to
enhance the credibility and reliability of its budget. Furthermore, NNSA’s
formal budget validation review process does not sufficiently ensure the
credibility and reliability of NNSA’s budget, primarily because of
deficiencies in the timing of these reviews. Also, without a formal
mechanism to evaluate the status of recommendations made during
previous budget validation reviews, NNSA is limited in its ability to
measure any progress M&O contractors or programs have made to their
budget estimating processes.

NNSA has reduced its ability to decide on resource trade-offs because it
has not enhanced, made formal, or implemented capabilities that it has
already, to varying degrees, developed or used. In particular, NNSA does
not follow its instructions for preparing an integrated priority list for each
congressional appropriation, as it does not combine the six priority lists
that represent activities funded by the Weapons Activities appropriation
into a single integrated list. By not combining these lists into a single
integrated priority list, NNSA is limiting the formal documentation
available to inform DOE which program activities would be affected by
changes to this appropriation. Moreover, NNSA instituted and then
disbanded an independent analytic capability that would provide it with an
independent cost assessment or evaluation capabilities of the
reasonableness and affordability of various programs and projects
proposed by NNSA offices. By disbanding its independent analytical
capability, NNSA is losing its ability to improve its cost-estimating
capabilities and better ensure that its project cost estimates are credible
and reliable. Because of the fiscal constraints in the current budget
environment, it is all the more critical that NNSA have the capability to
conduct independent cost analyses to enhance its ability to make the
most effective and efficient resource decisions on resource trade-offs.
Despite previous recommendations that DOE’s Inspector General made
in 2003, and that we made in 2007, to institute an independent analytical
capability to assess programs throughout all of NNSA, NNSA continues to
lack such a function. Not having this capability could preclude NNSA from


Page 24                       GAO-12-806 Modernizing the Nuclear Security Enterprise
                      making the best decisions about what activities to fund and whether they
                      are affordable. In addition, NNSA may cease using its Requirements and
                      Resources Assessment process—which is intended to provide some
                      independent analysis of new program activities and unfunded
                      requirements—in future PPBE budget cycles because it does not
                      anticipate program proposals for new activities or unfunded requirements.
                      By not retaining this process, NNSA would lose an important tool for
                      assessing the necessity to fund certain activities in order to meet its
                      mission requirements.

                      Furthermore, NNSA no longer follows its policy to issue the AFR. Without
                      a formal document, NNSA and DOE have no formal record of the
                      Administrator’s rationale and methodology for deciding on resource trade-
                      offs during the programming phase of the PPBE process. We recognize
                      that NNSA needs to hold confidential, internal budgetary and resource
                      trade-off deliberations; however, we do not believe that this need
                      supersedes NNSA policy or the benefits provided by documented
                      decision making during programming, which one senior NNSA official
                      described to us an important component in NNSA’s PPBE process. Not
                      issuing the AFR (or some similarly precise documentation) places the
                      Administrator in conflict with official NNSA policy and with an important
                      PPBE precept—the importance of transparency.

                      Finally, NNSA developed draft guidance in 2010 to assist NNSA program
                      managers in identifying reliable M&O contractor practices for estimating
                      costs for operations and sustainment activities. Such guidance would
                      better equip NNSA program managers to more accurately evaluate the
                      reasonableness of cost estimates, but this guidance is in draft form and
                      NNSA has no plans to complete and issue it. Without such guidance,
                      NNSA program managers are limited in their ability to assess the
                      reliability and credibility of budget estimates.


                      To enhance NNSA’s ability to better ensure the validity of its budget
Recommendations for   submissions, and to decide on resource trade-offs, we recommend that
Executive Action      the Secretary of Energy take the following seven actions:

                      •   Direct the DOE Office of Budget to formally evaluate DOE Order
                          130.1 and revise as necessary, and communicate any revisions to the
                          NNSA Administrator so that the agency will have updated provisions
                          for assessing the quality of its budget estimates.




                      Page 25                     GAO-12-806 Modernizing the Nuclear Security Enterprise
                     •   Direct the Administrator of NNSA to:
                         •   Develop a formal process, or amend its budget validation review
                             process, to ensure that all budget estimates are thoroughly
                             reviewed by site and headquarters program offices, and that these
                             reviews are timed to inform NNSA, DOE, OMB, and congressional
                             budget decisions.
                         •   Once this process is developed, incorporate a formal mechanism
                             to evaluate the status of recommendations made during previous
                             budget validation reviews so that NNSA can measure M&O
                             contractors’ and programs’ progress in responding to deficiencies
                             with their budget estimates.
                         •   Combine the integrated priorities lists for each of the program
                             offices funded within the Weapons Activities appropriation into a
                             single integrated priorities list, as stipulated in NNSA instructions,
                             to better inform DOE which program activities would be affected
                             by changes to this appropriation.
                         •   Reinstitute an independent analytical capability to provide senior
                             decision makers with independent program reviews, including an
                             analysis of different options for deciding on resource trade-offs,
                             and facilitate NNSA making the best decisions about what
                             activities to fund and whether they are affordable. As part of this
                             capability, formally retain the Requirements and Resources
                             Assessment process to review proposed new activities and
                             unfunded requirements.
                         •   Reinstitute the issuance of the Administrator’s Final
                             Recommendations to document the Administrator’s rationale and
                             methodology for deciding on resource trade-offs to support in his
                             budget proposal to DOE and to better facilitate NNSA’s
                             participation in DOE’s budget process.
                         •   Complete and formally issue the Program Managers’ Guide to
                             Understanding and Reviewing Cost Estimates for Operations and
                             Sustainment Activities so that program managers will be better
                             equipped to evaluate the reasonableness of cost estimates.



                     We provided DOE with a draft of this report for its review and comment. In
Agency Comments      its written comments, NNSA, responding on behalf of DOE, provided
and Our Evaluation   observations on the report’s findings and stated that it generally agreed in
                     principle with six of our seven recommendations and did not concur with
                     one. NNSA did not concur with our recommendation to combine the
                     integrated priorities lists for all program offices funded by the Weapons
                     Activities appropriation into a single integrated priorities list, as is
                     stipulated by NNSA instructions for the programming phase of PPBE.



                     Page 26                       GAO-12-806 Modernizing the Nuclear Security Enterprise
NNSA agrees that the integrated priorities lists are a useful tool to
facilitate NNSA and DOE decision-making. However, NNSA states that it
believes reaching management consensus on a single integrated
priorities list for these program offices would be a difficult, time consuming
process and that its current approach for deciding on resource trade-offs
is effective and efficient. We acknowledge that NNSA uses a variety of
tools in addition to integrated priorities lists to conduct programming
activities, but we continue to believe that combining the integrated
priorities lists for all program offices funded by the Weapons Activities
appropriation could enhance the agency’s ability to support its decisions
on resource trade-offs for DOE consideration during the Strategic
Resources Review. However, NNSA stated in its comments that it would
consider the development of more robust integrated priority lists if
circumstances require changes to its current approach.

NNSA further acknowledged that aspects of its PPBE process could be
improved but disagreed with our report’s characterization of its budget
estimate review processes as not being thorough. NNSA commented that
it believes that our conclusions overemphasize some procedural areas for
potential improvement, without accurately considering the cumulative
effectiveness of NNSA’s PPBE process as a whole. We continue to
believe that the agency’s processes for reviewing budget estimates are
not sufficiently thorough to ensure the credibility and reliability of those
estimates and do not meet the provisions defined in DOE Order 130.1.
Specifically, the reviews conducted by site and headquarters program
office officials are informal and undocumented, and NNSA’s budget
validation review process—the agency’s formal process for assessing
M&O contractor- and program-developed budget estimates—does not
assess the accuracy of budget estimates and is conducted for a small
portion of the agency’s annual budget. NNSA’s letter is reproduced in
appendix II.

NNSA also provided technical comments, which we incorporated
throughout the report as appropriate.


We are sending this report to the Secretary of Energy, the Administrator
of NNSA, the appropriate congressional committees, and other interested
parties. In addition, the report is available at no charge on the GAO
website at http://www.gao.gov.




Page 27                       GAO-12-806 Modernizing the Nuclear Security Enterprise
If you or your staff have any questions about this report, please contact
me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Office
of Congressional Relations and Public Affairs may be found on the
last page of this report. Key contributors to this report are listed in
appendix III.




Gene Aloise
Director, Natural Resources and Environment




Page 28                      GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              The objectives of our review were to determine (1) the current structure of
              the National Nuclear Security Administration’s (NNSA) planning,
              programming, budgeting, and evaluation (PPBE) process; (2) the extent
              to which NNSA reviews its budget estimates; and (3) how NNSA decides
              on resource trade-offs in its PPBE process.

              To determine the current structure of NNSA’s PPBE process, we
              reviewed the NNSA policies and other headquarters-developed
              instructions and guidance documents that define how the process is
              designed to function. We also reviewed program-specific PPBE
              documentation from the Offices of Defense Programs, Defense Nuclear
              Nonproliferation, and Naval Reactors; these three offices correspond to
              NNSA’s primary missions and collectively account for approximately 85
              percent of the President’s fiscal year 2013 NNSA budget submission to
              Congress. We also interviewed officials from NNSA’s Office of
              Management and Budget, which is responsible for managing NNSA’s
              PPBE process, as well as the offices of Defense Programs, Defense
              Nuclear Nonproliferation, and Naval Reactors to discuss how NNSA’s
              PPBE process is designed to function.

              To determine the extent to which NNSA reviews its budget estimates, we
              reviewed DOE Order 130.1 and NNSA policies, instructions, and
              guidance that define how such reviews are to be conducted. We also
              analyzed documentation of the formal budget validation reviews
              conducted by NNSA for the last five review cycles, as well as the results
              of two NNSA workgroups that evaluated the budget validation review
              process. Furthermore, we interviewed officials involved in the
              development, oversight, or execution of NNSA budget estimate reviews
              from the NNSA Offices of the Administrator, Management and Budget,
              Defense Programs, Defense Nuclear Nonproliferation, Naval Reactors,
              and Acquisition and Project Management; the site offices for Los Alamos,
              Sandia, and the Y-12 National Security Complex, and the Naval Reactors
              Laboratory Field Office; DOE officials from the Office of Budget; and M&O
              contractor officials from Los Alamos and Sandia National Laboratories,
              the Y-12 National Security Complex, and Bettis Atomic Power
              Laboratory. 1 Because NNSA’s Office of Naval Reactors is organized as a


              1
                We selected a nonprobability sample of NNSA sites based on the levels of funding they
              receive from NNSA for the programs included in the scope of our work. Because we used
              a nonprobability sample, the results we obtained from these sites are not generalizable
              across the nuclear security enterprise; however, they did provide us with specific
              examples of how NNSA conducts PPBE activities at the site level.




              Page 29                              GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix I: Objectives, Scope, and
Methodology




separate entity within NNSA reporting both to NNSA and the U.S. Navy,
we also met with Navy officials from its Offices of Financial Management
and Budgeting, and Cost Engineering and Industrial Analysis.

To determine how NNSA decides on resource trade-offs, we reviewed
NNSA policies, instructions, and guidance for its programming process.
Based on these documents, we identified the tools that NNSA uses, or
has used, to assist NNSA management in deciding on resource trade-offs
and reviewed documentation of how these tools were applied by program
offices and NNSA management during the programming phases of
previous PPBE cycles. We also interviewed officials from the NNSA
Offices of the Administrator, Management and Budget, Defense
Programs, Defense Nuclear Nonproliferation, Naval Reactors to discuss
how they decide on, and document, resource trade-offs.

We conducted this performance audit from July 2011 to July 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




Page 30                              GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
              Appendix II: Comments from the U.S.
              Department of Energy



Department of Energy




              Page 31                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
Department of Energy




Page 32                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
Department of Energy




Page 33                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
Department of Energy




Page 34                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
Department of Energy




Page 35                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix II: Comments from the U.S.
Department of Energy




Page 36                           GAO-12-806 Modernizing the Nuclear Security Enterprise
Appendix III: GAO Contact and Staff
                  Appendix III: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Gene Aloise, (202) 512-3841, or aloisee@gao.gov
GAO Contact
                  In addition to the contact named above, Dan Feehan (Assistant Director),
Staff             Robert Baney, Matthew Tabbert, and Eugene Wisnoski made significant
Acknowledgments   contributions to this report. Cheryl Peterson, Jeremy Sebest, Rebecca
                  Shea, Carol Herrnstadt Shulman, and Kiki Theodoropoulos provided
                  technical assistance.




(361316)
                  Page 37                               GAO-12-806 Modernizing the Nuclear Security Enterprise
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.