oversight

Modernizing the Nuclear Security Enterprise: New Plutonium Research Facility at Los Alamos May Not Meet All Mission Needs

Published by the Government Accountability Office on 2012-03-26.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States Government Accountability Office

GAO          Report to the Subcommittee on Energy
             and Water Development, Committee on
             Appropriations, U.S. Senate


March 2012
             MODERNIZING THE
             NUCLEAR SECURITY
             ENTERPRISE
             New Plutonium
             Research Facility at
             Los Alamos May Not
             Meet All Mission
             Needs




GAO-12-337
                                            March 2012

                                            MODERNIZING THE NUCLEAR SECURITY
                                            ENTERPRISE
                                            New Plutonium Research Facility at Los Alamos May
Highlights of GAO-12-337, a report to the
                                            Not Meet All Mission Needs
Subcommittee on Energy and Water
Development, Committee on Appropriations,
U.S. Senate



Why GAO Did This Study                      What GAO Found
Plutonium—a man-made element                The estimated cost to construct the CMRR has greatly increased since NNSA’s
produced by irradiating uranium in          initial plans, and the project’s schedule has been significantly delayed. According
nuclear reactors—is vital to the nuclear    to its most recent estimates prepared in April 2010, NNSA determined that the
weapons stockpile. Much of the              CMRR will cost between $3.7 billion and $5.8 billion—nearly a six-fold increase
nation’s current plutonium research         from the initial estimate. Construction has also been repeatedly delayed and, in
capabilities are housed in aging            February 2012 after GAO provided its draft report to NNSA for comment, NNSA
facilities at Los Alamos National           decided to defer CMRR construction by at least an additional 5 years, bringing
Laboratory in New Mexico. These             the total delay to between 8 and 12 years from NNSA’s original plans.
facilities pose safety hazards. The
                                            Infrastructure-related design changes and longer-than-expected overall project
National Nuclear Security
                                            duration have contributed to these cost increases and delays. GAO’s review of
Administration (NNSA) has decided to
construct a multibillion dollar Chemistry
                                            NNSA’s April 2010 cost and schedule estimates for CMRR found that the
and Metallurgy Research Replacement         estimates were generally well prepared, but important weaknesses remain. For
Nuclear Facility (CMRR) to modernize        example, a high-quality schedule requires a schedule risk analysis that
the laboratory’s capabilities to analyze    incorporates known risks to predict the level of confidence in meeting a project’s
and store plutonium. GAO was asked          completion date and the amount of contingency time needed to cover
to examine (1) the cost and schedule        unexpected delays. CMRR project officials identified hundreds of risks to the
estimates to construct CMRR and the         project, but GAO found that these risks were not used in preparing a schedule
extent to which its most recent             risk analysis. As a result of these weaknesses, NNSA cannot be fully confident,
estimates reflect best practices, (2)       once it decides to resume the CMRR project, that the project will be completed
options NNSA considered to ensure           on time and within estimated costs.
that needed plutonium research
activities could continue, and (3) the      NNSA considered several options to preserve its plutonium-related research
extent to which NNSA's plans reflected      capabilities in its decision to build CMRR at Los Alamos. NNSA assessed three
changes in stockpile requirements and       different sizes for a new facility—22,500, 31,500, and 40,500 square feet. In
other plutonium research needs. GAO         2004, NNSA chose the smallest option. NNSA officials stated that cost was the
reviewed NNSA and contractor project        primary driver of the decision, but that building a smaller facility would result in
design documents and visited Los            trade-offs, including the elimination of contingency space. In the end, NNSA
Alamos and another plutonium facility       decided to build a minimally-sized CMRR facility at Los Alamos with a broad
at Lawrence Livermore National              suite of capabilities to meet nuclear weapons stockpile needs over the long-term.
Laboratory in California.                   These capabilities would also be used to support plutonium-related research
                                            needs of other departmental missions.
What GAO Recommends
                                            NNSA’s plans to construct CMRR focused on meeting nuclear weapons stockpile
GAO is making recommendations to            requirements, but CMRR may not meet all stockpile and other plutonium-related
improve CMRR’s schedule risk
                                            research needs. NNSA analyzed data on past workload and the expected need
analysis and to conduct an
                                            for new weapon components to help ensure CMRR’s design included the
assessment of plutonium research
needs. NNSA agreed with GAO’s               necessary plutonium-related research capabilities for maintaining the safety and
recommendations to assess plutonium         reliability of the nuclear stockpile. However, some plutonium research, storage,
research needs, but disagreed that its      and environmental testing capabilities that exist at Lawrence Livermore National
schedule risk analysis should be            Laboratory may no longer be available after NNSA consolidates plutonium-
revised, citing its recent decision to      related research at Los Alamos. Furthermore, NNSA conducts important
defer the project. GAO clarified the        plutonium-related research in other areas such as homeland security and nuclear
recommendation to specify that NNSA         nonproliferation, but it has not comprehensively analyzed plutonium research and
should take action when it resumes the      storage needs of these other programs outside of its nuclear weapons stockpile
project.                                    work and therefore cannot be sure that the CMRR plans will effectively
                                            accommodate these needs. As a result, expansion of CMRR or construction of
                                            more plutonium research and storage facilities at Los Alamos or elsewhere may
View GAO-12-337. For more information,      be needed in the future, potentially further adding to costs.
contact Gene Aloise at (202) 512-3841 or
aloisee@gao.gov.
                                                                                     United States Government Accountability Office
Contents


Letter                                                                                       1
               Background                                                                    5
               CMRR’s Initial Cost Estimate Has Significantly Increased and Its
                 Schedule Has Been Delayed                                                   9
               NNSA Considered Several Options to Preserve Plutonium-Related
                 Research Capabilities, but Ultimately Chose to Build a Minimally
                 Sized Facility at Los Alamos                                              14
               CMRR May Meet Nuclear Weapons Stockpile Requirements but
                 May Not Accommodate Other Plutonium-Related Research
                 Needs                                                                     17
               Conclusions                                                                 22
               Recommendations for Executive Action                                        23
               Agency Comments and Our Evaluation                                          24

Appendix I     Objectives, Scope, and Methodology                                          26



Appendix II    Summary Assessment of CMRR’s Cost Estimate Compared to
               Industry Best Practices                                                     28



Appendix III   Summary Assessment of CMRR’s Schedule Estimate Compared to
               Industry Best Practices                                                     33



Appendix IV    Comments from the National Nuclear Security Administration                  36



Appendix V     GAO Contact and Staff Acknowledgments                                       39



Tables
               Table 1: Extent to Which CMRR’s Cost Estimate Met Industry Best
                        Practices                                                          11
               Table 2: Extent to Which CMRR’s Schedule Estimate Met Industry
                        Best Practices                                                     12




               Page i                               GAO-12-337 Plutonium Research at Los Alamos
Abbreviations

CMRR                       Chemistry and Metallurgy Research Replacement
                           Nuclear Facility
DOE                        Department of Energy
NNSA                       National Nuclear Security Administration




This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                     GAO-12-337 Plutonium Research at Los Alamos
United States Government Accountability Office
Washington, DC 20548




                                   March 26, 2012

                                   The Honorable Dianne Feinstein
                                   Chairman
                                   The Honorable Lamar Alexander
                                   Ranking Member
                                   Subcommittee on Energy and Water Development
                                   Committee on Appropriations
                                   United States Senate

                                   Plutonium—a man-made radioactive element produced by irradiating
                                   uranium in nuclear reactors—is vital to the nation’s nuclear weapons
                                   stockpile. Plutonium is used in “pits”—the spherical central core of a
                                   nuclear weapon that is compressed with high explosives to create a
                                   nuclear explosion. Several kilograms of plutonium are sufficient to make a
                                   nuclear bomb, so plutonium must be stored under extremely high security
                                   to protect it from theft. In addition, exposure to small quantities is
                                   dangerous to human health, so that even inhaling a few micrograms
                                   creates a long-term risk of lung, liver, and bone cancer and inhaling larger
                                   doses can cause immediate lung injuries and death. Also, if not safely
                                   contained and managed, plutonium can be unstable and spontaneously
                                   ignite under certain conditions. Therefore, any facility that stores or
                                   conducts research on plutonium must be robustly designed to safely
                                   prevent the uncontrolled release of hazardous material to the
                                   environment and to securely store the material to protect it from potential
                                   theft.

                                   Much of the nation’s current plutonium research and development
                                   capabilities are housed at the Los Alamos National Laboratory in New
                                   Mexico. The laboratory is one of the National Nuclear Security
                                   Administration’s (NNSA) two primary laboratories responsible for
                                   designing nuclear weapons components that contain plutonium; the other
                                   is Lawrence Livermore National Laboratory in California. 1 Los Alamos has
                                   been supporting the production of new pits since the closure of the
                                   Department of Energy’s (DOE) Rocky Flats Plant near Denver, Colorado,


                                   1
                                    NNSA was created by the National Defense Authorization Act for Fiscal Year 2000, Pub.
                                   L. No. 106-65 (1999). It is a separate, semiautonomous agency within the Department of
                                   Energy, with responsibility for the nation’s nuclear weapons, nonproliferation, and naval
                                   reactors programs.




                                   Page 1                                     GAO-12-337 Plutonium Research at Los Alamos
in 1989, as well as homeland security activities, energy programs, and
nuclear nonproliferation activities.

The Chemistry and Metallurgy Research nuclear facility at Los Alamos
conducts plutonium-related research that is crucial to effectively maintain
the nuclear weapons stockpile. However, the facility is nearly 60 years
old, and its aging infrastructure poses safety hazards. In addition, the
facility is situated on a seismic fault line, raising concerns about the effect
of earthquakes on the safety and security of plutonium used for research
or stored at the facility. Because of these concerns, NNSA has decided to
construct a Chemistry and Metallurgy Research Replacement Nuclear
Facility (CMRR) at Los Alamos that will (1) modernize the laboratory’s
capabilities to analyze plutonium and (2) store plutonium in vaults that
provide a secure environment that protects against its accidental or
intentional misuse and minimizes health risks for workers and the
surrounding communities. 2 Originally estimated to begin construction in
2008, the project has experienced several delays and, in February 2012
after we had provided a draft of this report to NNSA for its comments,
NNSA announced that it had decided to defer construction of the facility
for at least 5 years.

NNSA’s proposed construction of the CMRR is part of a larger strategic
effort to consolidate nuclear materials from other locations across the
United States and to modernize nuclear research, development, and
production facilities that support the nuclear weapons stockpile. For
example, NNSA’s plans call for some of the plutonium-related research
currently conducted at a high security facility at Lawrence Livermore
National Laboratory known as Superblock to be transferred to CMRR.
NNSA’s plans also call for much of the plutonium currently stored at
Livermore’s Superblock to be consolidated at Los Alamos for continued
research activities, and for the other material not needed for research at
Livermore or Los Alamos to be stored at DOE’s Savannah River Site
pending final disposition. In addition, the nuclear weapons stockpile’s
requirements for plutonium are evolving. Specifically, the New Strategic
Arms Reduction Treaty that the United States and Russia signed in April
2010 is to reduce the number of deployed strategic nuclear warheads by


2
 For the purposes of this report, CMRR refers to the design and construction of the
nuclear facility portion of NNSA’s CMRR project. The scope of this report does not include
the first phase of the project—the Radiological Laboratory, Utility, and Office Building—
which is much smaller in scope and cost and is substantially complete.




Page 2                                      GAO-12-337 Plutonium Research at Los Alamos
30 percent. As a result of this treaty and NNSA’s approach for
modernizing the stockpile through the refurbishment of existing weapons,
demand for newly manufactured pits has fluctuated in recent years. In
light of these fluctuations, NNSA’s current strategy is to design the CMRR
around a broad suite of capabilities—equipment, processes, and
expertise—that it anticipates may be needed to fulfill the stockpile’s
requirements regardless of the specific demand for pits.

Because of the extensive safety and security measures required to
analyze and store plutonium, the cost of constructing new nuclear
facilities is typically a multibillion dollar venture. In the past, we have
reported on several major DOE and NNSA construction projects that
faced cost increases and schedule delays. 3 DOE’s long-standing
difficulties in preparing cost and schedule estimates is one reason
contract management in NNSA and DOE’s Office of Environmental
Management is on our list of federal programs at high risk of fraud, waste,
abuse, and mismanagement. 4 Because other federal agencies have also
had problems developing high-quality cost and schedule estimates, we
issued a cost-estimating guide in March 2009, consisting of best practices
drawn from across industry and government, to assist agencies to
develop cost and schedule estimates that are well-documented,
comprehensive, accurate, and credible. 5

In this context, you asked us to review NNSA’s plans for constructing the
CMRR. Specifically, our objectives were to examine (1) NNSA’s cost and
schedule estimates for the construction of the facility and the extent to



3
 GAO, Nuclear Weapons: National Nuclear Security Administration’s Plans for Its Uranium
Processing Facility Should Better Reflect Funding Estimates and Technology Readiness,
GAO-11-103 (Washington, D.C.: Nov. 19, 2010); GAO, Department of Energy: Actions
Needed to Develop High-Quality Cost Estimates for Construction and Environmental
Cleanup Projects, GAO-10-199 ( Washington, D.C.: Jan. 14, 2010); GAO, Nuclear
Weapons: National Nuclear Security Administration Needs To Better Manage Risks
Associated with the Modernization of Its Kansas City Plant, GAO-10-115 (Washington,
D.C.: Oct. 23, 2009): GAO, Nuclear Waste: Uncertainties and Questions about Costs and
Risks Persist with DOE’s Tank Waste Cleanup Strategy at Hanford, GAO-09-913
(Washington, D.C.: Sept. 30, 2009); and GAO, Department of Energy: Contract and
Project Management Concerns at the National Nuclear Security Administration and Office
of Environmental Management, GAO-09-406T (Washington, D.C.: Mar. 4, 2009).
4
GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: Feb. 16, 2011).
5
 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and
Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009).




Page 3                                    GAO-12-337 Plutonium Research at Los Alamos
which its most recent estimates reflect best practices, (2) options NNSA
considered to ensure that plutonium-related research activities will
continue as needed, and (3) the extent to which NNSA’s plans to
construct the CMRR and its consideration of options reflected changes in
nuclear weapons stockpile requirements and the plutonium-related
research needs of other departmental missions.

To examine NNSA’s cost and schedule estimates for the CMRR project
and the extent to which its current estimates reflect best practices, we
reviewed relevant NNSA documents and met with agency and Los
Alamos project officials to discuss the changes in the estimates that have
occurred to date and the reasons for them. We compared NNSA’s most
recent cost and schedule estimates––prepared in April 2010––with best
practices contained in our cost estimating guide and gave project officials
the opportunity to provide and discuss feedback on our assessment. To
examine the options NNSA considered to ensure that its plutonium-
related work will continue, we reviewed NNSA and contractor documents
on plutonium research needs and the various options available to meet
those needs. We also met with NNSA and contractor officials to better
understand how these options were analyzed to determine the best
approach to fulfill NNSA’s mission. To determine the extent to which
NNSA’s plans reflect changes in nuclear weapons stockpile requirements,
we reviewed NNSA analyses that were used to support CMRR project
decisions and met with NNSA officials to determine if these analyses
were comprehensive and reflected up-to-date nuclear weapons stockpile
requirements. We also visited the Los Alamos and Lawrence Livermore
national laboratories. To ensure the data we used were sufficiently
reliable, we compared information gathered from a variety of data
sources. For example, we interviewed officials from both Los Alamos and
Livermore to obtain separate and independent perspectives on CMRR
project plans. We determined the data were sufficiently reliable for our
purposes.

We conducted this performance audit from February 2011 through
February 2012 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives. Appendix I
contains a detailed description of our scope and methodology.




Page 4                              GAO-12-337 Plutonium Research at Los Alamos
             In the mid-1990s, Congress directed DOE to develop the Stockpile
Background   Stewardship Program to provide a single, highly integrated technical
             program for maintaining the continued safety and reliability of the nuclear
             weapons stockpile. Stockpile stewardship comprises activities associated
             with conducting nuclear weapons research, design, and development;
             maintaining the knowledge base and capabilities to support nuclear
             weapons testing; and assessing and certifying nuclear weapons safety
             and reliability. Stockpile stewardship includes operations associated with
             producing, maintaining, refurbishing, surveilling, and dismantling the
             nuclear weapons stockpile. The Stockpile Stewardship Program’s
             objectives were updated as a result of the 2010 Nuclear Posture Review,
             which establishes the U.S. nuclear policy for the next 5 to 10 years,
             including the nation’s nuclear weapons stockpile requirements. 6 The
             Nuclear Posture Review and the Stockpile Stewardship Program
             reinforce the New Strategic Arms Reduction Treaty between the United
             States and Russia. As part of this treaty, the United States has agreed to
             reduce the size of its strategic nuclear weapons stockpile from a
             maximum of 2,200 to 1,550 weapons, with the remaining weapons in the
             stockpile continuing to be an essential element of U.S. defense strategy.

             Nuclear stockpile requirements include a pit production capacity that is
             defined by estimating the number of pits NNSA needs to manufacture
             annually to effectively support the nuclear weapons stockpile. The
             demand for pits has fluctuated over the past decade for various reasons.
             Until 2005, NNSA planned to produce pits in a large-scale manufacturing
             plant to be built called the Modern Pit Facility, which would have
             increased pit production capacity per year to a range of 125 to 450 pits.
             This project was terminated and, at around the same time, NNSA began
             to study a new approach for modernizing the stockpile, called the Reliable
             Replacement Warhead program, which would have produced 50 pits per
             year and which was also short-lived. Through this program, NNSA would
             have designed new weapon components, including pits, to be safer and
             easier to manufacture, maintain, dismantle, and certify without nuclear
             testing. Since 2008, NNSA’s guidance has established pit capacity for
             future production at about 20 pits per year, with an upper range limit of 80
             pits per year. In addition, NNSA has recently determined that pit lifetimes
             are longer than anticipated and that it may increase the reuse of existing
             pits, reducing the demand for newly manufactured pits. Currently, pit



             6
             Department of Defense, Nuclear Posture Review Report, (Washington, D.C.: April 2010).




             Page 5                                   GAO-12-337 Plutonium Research at Los Alamos
capacity requirements are uncertain and still in flux. Demand may again
fluctuate as a result of the Nuclear Posture Review and changes to the
Stockpile Stewardship Program. For example, there are still unknowns in
implementing the Nuclear Posture Review and modernization work on
each nuclear weapon type may require a varied number of new pits.

To execute the activities to maintain and refurbish the nation’s existing
nuclear weapons stockpile, NNSA oversees eight sites that comprise its
nuclear security enterprise—formerly known as the nuclear weapons
complex—which includes three national weapons laboratories, four
production plants, and a test site, all of which carry out missions to
support NNSA’s programs. One of these sites, Los Alamos National
Laboratory, plays a crucial role in carrying out NNSA’s maintenance of
the nuclear weapons stockpile, including (1) production of weapons
components, (2) assessment and certification of the nuclear weapons
stockpile, (3) surveillance of weapons components and weapon systems,
(4) assurance of the safe and secure storage of strategic materials, and
(5) management of excess plutonium inventories. Los Alamos was
established in 1943 during the Manhattan Project in northern New
Mexico. It is a multidisciplinary, multipurpose institution primarily engaged
in theoretical and experimental research and development. A significant
portion of Los Alamos’ work is focused on ensuring that nuclear weapons
stockpile needs are met. Since 2000, pit production has been established
within the Plutonium Facility Complex at Los Alamos’s Technical Area 55,
and certified pits have been produced over the past 5 years in that facility.

A particularly important facility at Los Alamos within Technical Area 55 is
the nearly 60-year-old Chemistry and Metallurgy Research facility. The
facility has unique capabilities for performing analytical chemistry,
material characterization, and research and development related to
plutonium. This includes activities that support the manufacturing,
development, and surveillance of nuclear weapons pits; programs to
extend the life of nuclear weapons in the stockpile; and nuclear weapon
dismantlement efforts. This pit production mission support work was first
assigned to Los Alamos in 1996. NNSA also currently maintains some
plutonium-related research capabilities at other facilities, such as
Livermore’s Superblock facility. These capabilities are necessary
components of NNSA’s overall stockpile management strategy. NNSA
and DOE also use the unique plutonium-related capabilities located at
Los Alamos and Livermore to support the plutonium-related research
needs of other national security missions and activities outside of the
nuclear weapons stockpile work, including nuclear nonproliferation
activities; homeland security activities, such as nuclear forensics and


Page 6                               GAO-12-337 Plutonium Research at Los Alamos
nuclear counterterrorism; waste management; and material recycle and
recovery programs.

The Chemistry and Metallurgy Research facility was initially designed and
constructed to comply with building codes in effect during the late 1940s
and early 1950s. In 1992, recognizing that some of the utility systems and
structural components were aging, outmoded, and generally deteriorating,
DOE began upgrading the facility. These upgrades addressed specific
safety, reliability, consolidation, and security issues with the intent of
extending the useful life of the facility for an additional 20 to 30 years.
However, beginning in about 1997 and continuing to the present, a series
of additional operational and safety concerns have surfaced. In particular,
a 1998 seismic study identified two small parallel faults beneath the
northern portion of the Chemistry and Metallurgy Research facility. The
presence of these faults raised concerns about the structural integrity of
the building in the event of an earthquake. DOE and NNSA determined
that, over the long term, Los Alamos could not continue to operate the
mission-critical support capabilities in the existing Chemistry and
Metallurgy Research facility at an acceptable level of risk to worker safety
and health. To ensure that NNSA can fulfill its national security mission
for the next 50 years in a safe, secure, and environmentally sound
manner, NNSA decided in 2004 to construct a replacement facility, known
as the CMRR. 7

Federal agencies, including DOE and NNSA, have experienced long-
standing difficulties in completing major projects within cost and on
schedule. To provide assistance in preparing high-quality cost and
schedule estimates, we compiled best practices used throughout
government and industry and, in March 2009, issued a guide outlining the
criteria for high-quality cost and schedule estimates. Specifically, our
guide identified four characteristics of a high-quality, reliable cost
estimate: (1) credible, (2) well-documented, (3) accurate, and (4)




7
 In 2004, NNSA determined that it needed a nuclear facility to relocate certain analytical
capabilities from existing facilities at Los Alamos, which are near end-of-life, as part of
NNSA’s plans for maintaining and certifying the nation’s nuclear weapons stockpile. In
deciding whether to build a new facility or instead use or refurbish other existing facilities,
a 2006 Los Alamos study determined that a new nuclear facility should be built because
the fundamental objectives of NNSA’s strategic planning for the nuclear weapons complex
could not be achieved without it.




Page 7                                        GAO-12-337 Plutonium Research at Los Alamos
comprehensive. 8 In addition, our cost guide lays out 12 key steps that
should result in high-quality cost estimates and hundreds of best
practices drawn from across industry and government for carrying out
these steps. For example, one of the key steps includes conducting an
independent cost estimate––that is, one generated by an entity that has
no stake in the approval of the project but uses the same detailed
technical information as the project estimate. Having an independent
entity perform such a cost estimate and comparing it to the project team’s
estimate provides an unbiased test of whether the project team’s cost
estimate is reasonable.

Our guide also identified nine best practices for effectively estimating
schedules: (1) capturing key activities, (2) sequencing key activities, (3)
assigning resources to key activities, (4) establishing the duration of key
activities, (5) integrating key activities horizontally and vertically, (6)
establishing the critical path for key activities, (7) identifying total float
(i.e., the time that activities can slip before the delay affects the
completion date), (8) performing a risk analysis of the schedule, and (9)
updating the schedule using logic and durations to determine dates. 9
Many of these practices have also been incorporated into DOE’s recent
guidance for establishing performance baselines. 10




8
 In the context of our cost guide, a cost estimate is the summation of individual cost
elements, using established methods and valid data, to estimate the future costs of a
project, based on what is known today.
9
GAO-09-3SP.
10
  DOE, Performance Baseline Guide, DOE G 413.3-5A (Washington, D.C.: Sept. 23,
2011). Although there is not a one-to-one correlation, many of the GAO-identified best
practices, are also suggested schedule development practices in DOE’s Performance
Baseline Guide. DOE also requires that NNSA establish a project performance baseline to
document estimated project cost and schedule for planned capital projects in order to
measure the project’s performance. See DOE, Program and Project Management for the
Acquisition of Capital Assets, DOE O 413.3B (Washington, D.C.: Nov. 29, 2010).




Page 8                                      GAO-12-337 Plutonium Research at Los Alamos
                           The estimated cost to construct the CMRR, according to estimates
CMRR’s Initial Cost        prepared in April 2010, is nearly six times higher than the project’s initial
Estimate Has               cost estimate that was prepared in 2005. The project’s estimated
                           completion date has also been delayed by at least 8 to 12 years. Our
Significantly              review of these most recent detailed cost and schedule estimates for the
Increased and Its          CMRR project found that the estimates generally reflect best practices,
Schedule Has Been          but are not yet entirely reliable.

Delayed
Estimated Cost to          Since CMRR was first proposed, its costs have risen significantly, and its
Construct CMRR Has         schedule has been repeatedly delayed. Specifically, in 2005, when DOE
Increased by Almost Six-   developed initial plans for CMRR, it estimated that the project would cost
                           from $745 million to $975 million and would be completed between 2013
Fold, and Its Scheduled    and 2017. This estimate was prepared using preliminary information—
Completion Has Been        before a detailed project design was substantially under way—and was
Delayed                    therefore considered by DOE to be a rough estimate. In April 2010, NNSA
                           estimated that the CMRR will cost between $3.7 and $5.8 billion—a
                           nearly six-fold increase from the initial estimate—and that construction
                           will be complete by 2020—a 3- to 7-year delay. In February 2012, after
                           we had provided NNSA with a draft of this report for its comments, NNSA
                           announced that it had decided to defer CMRR construction by at least an
                           additional 5 years, bringing the total delay from NNSA’s original plans to 8
                           to 12 years.

                           NNSA officials explained that the majority of the cost increases occurred
                           because of changes to the facility’s design and because of project delays.
                           Specifically,

                           •   Modifications to the facility’s design. To address concerns about
                               seismic activity, the project design was modified to strengthen the
                               facility to withstand a potential earthquake. For example, significant
                               design changes resulted from the need to thicken the concrete walls
                               to satisfy increasingly stringent seismic requirements. In addition, to
                               proceed to final design, project officials had to evaluate the potential
                               effects of an earthquake on the facility’s complex ventilation system.
                               This effort included several studies, consultations with vendors and
                               other designers, and an assessment of the availability of equipment
                               that would meet seismic requirements. Overall, Los Alamos estimates
                               the seismic related design changes increased the project costs by
                               almost $500 million.




                           Page 9                                GAO-12-337 Plutonium Research at Los Alamos
                              •     Delays in the construction start date and longer overall project
                                    duration. CMRR construction was originally expected to begin in
                                    2008, but was first delayed until 2013 and is now not expected to
                                    begin before 2018. The initial delay in starting construction from 2008
                                    to 2013 had varying causes, including facility design changes
                                    described previously as well as the additional time needed for NNSA
                                    to determine where and how to consolidate plutonium operations in
                                    the nuclear security enterprise, according to project officials. This
                                    delay starting construction pushed the estimated construction
                                    completion date from between 2013 and 2017 to 2020—3 to 7 years
                                    later than initially expected. At the time, the facility was expected to be
                                    operational in 2022. 11 These delays further increased costs, partly
                                    because inflation meant that equipment and materials became more
                                    expensive as time passed. In addition, the longer project duration also
                                    contributes to increases in the cost of workers’ wages and salaries.
                                    Overall, project officials estimate that about $1.2 billion in additional
                                    costs resulted from these schedule delays. In February 2012, NNSA
                                    announced another significant project delay—at least an additional 5-
                                    year deferral in starting the construction of the CMRR—resulting in a
                                    total of an 8 to 12 year delay from NNSA’s original plans. However,
                                    NNSA has not yet determined the impact to the project’s costs as a
                                    result of this additional delay.



NNSA’s Most Recent Cost       Our review of NNSA’s most recent cost and schedule estimates for the
and Schedule Estimates        CMRR construction project found that the estimates were generally well
Generally Meet Industry       prepared but that important weaknesses remain. Specifically, we found
                              that the CMRR cost estimate prepared in April 2010 exhibits most of the
Best Practices, but Are Not   characteristics of high-quality, reliable cost estimates. As identified by the
Yet Entirely Reliable         professional cost-estimating community and documented in our cost-
                              estimating guide, a high-quality cost estimate is comprehensive, well-
                              documented, accurate, and credible. 12 Our review of the CMRR cost
                              estimate found that the cost estimate exhibits three of the four
                              characteristics of a high-quality estimate by being substantially
                              comprehensive, well documented, and accurate, but only partially



                              11
                                The delay between completion of construction and the date of operation allows for
                              equipment to be prepared for use and workers to be trained on new equipment, among
                              other things.
                              12
                                  GAO-09-3SP.




                              Page 10                                   GAO-12-337 Plutonium Research at Los Alamos
credible, as shown in table 1. Appendix II contains additional information
about each of the four general best practice characteristics and our
assessment of the estimate compared to detailed best practices.

Table 1: Extent to Which CMRR’s Cost Estimate Met Industry Best Practices
                                                                                    a
    Best practice characteristic                                Overall assessment
    Comprehensive                                               Substantially met
    Well documented                                             Substantially met
    Accurate                                                    Substantially met
    Credible                                                    Partially met
Source: GAO analysis of CMRR project cost information.

a
 The ratings we used in this analysis are as follows: “Not met” means CMRR provided no evidence
that satisfies any of the characteristic. “Minimally met” means CMRR provided evidence that satisfies
a small portion of the characteristic. “Partially met” means CMRR provided evidence that satisfies
about half of the characteristic. “Substantially met” means CMRR provided evidence that satisfies a
large portion of the characteristic. “Fully met” means CMRR provided complete evidence that satisfies
the entire characteristic.


The CMRR cost estimate only partially met industry best practices for
credibility because project officials did not use alternate methods to
crosscheck major cost elements to see whether the results were similar
under different estimating methods. In addition, according to our guide,
there are varying methods of validating an estimate, but the most rigorous
method is the independent cost estimate that is generated by an entity
that has no stake in the approval of the project. Conducting an
independent cost estimate is especially important at major milestones
because it provides senior decision makers with a more objective
assessment of the likely cost of a project. A second, less rigorous method
for validating a project’s cost estimate—an independent cost review—
focuses on examining the estimate’s supporting documentation and
interviewing relevant staff. Independent cost reviews address only the
cost estimate’s high-value, high-risk, and high-interest aspects without
evaluating the remainder of the estimate. An independent cost review on
the entire CMRR project was initiated in 2011, but the more rigorous
method of validating—conducting an independent cost estimate—has
only been used on a small portion of the project representing about 6




Page 11                                                  GAO-12-337 Plutonium Research at Los Alamos
percent of the project’s total costs. 13 According to NNSA officials, DOE
orders do not require NNSA to seek an independent cost estimate until
just prior to establishing the project baseline, and project officials told us
NNSA is preparing to have one conducted before the project baseline is
established. However, until a quality independent cost estimate is
completed on the entire project or another means of validating the
estimate for the project, DOE and NNSA officials cannot be confident that
the current cost estimate is completely credible.

With regard to CMRR’s schedule, the project’s schedule estimate fully
met two and substantially met six out of nine best practices for a high-
quality schedule as identified by our guide and minimally met one. For
example, two of the best practices the estimate fully met concerned how
well it (1) captured all of the project’s activities, including design,
construction, and other tasks that collectively form a comprehensive
schedule, and (2) is successfully kept up-to-date. Table 2 lists best
practices along with our assessment of the extent to which the project’s
schedule met each best practice.

Table 2: Extent to Which CMRR’s Schedule Estimate Met Industry Best Practices
                                                                                                a
Best practice                                                               Overall assessment
Capturing all activities                                                    Fully met
Sequencing all activities                                                   Substantially met
Assigning resources to all activities                                       Substantially met
Establishing the duration of all activities                                 Substantially met
Integrating schedule activities                                             Substantially met
Establishing the critical path for all activities                           Substantially met
Identifying float between activities                                        Substantially met
Conducting a schedule risk analysis                                         Minimally met
Updating the schedule using logic and durations to                          Fully met
determine dates
Source: GAO analysis of CMRR project schedule information.




13
  An independent cost estimate was initiated in 2011 covering the design and
infrastructure needed to complete the CMRR nuclear facility project, such as concrete
batch plants and equipment storage, which represents only $250 million of the estimated
total project cost.




Page 12                                                      GAO-12-337 Plutonium Research at Los Alamos
a
 The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no
evidence that satisfies any part of a practice. “Minimally met” means the CMRR provided evidence
that satisfies a small portion of a practice. “Partially met” means the CMRR provided evidence that
satisfies about half of a practice. “Substantially met” means the CMRR provided evidence that
satisfies a large portion of a practice. “Fully met” means the CMRR provided evidence that completely
satisfies a practice.


The CMRR schedule estimate minimally met industry best practices for
conducting a schedule risk analysis. Namely, according to our guide, a
high-quality schedule requires a schedule risk analysis that uses already
identified risks, among other things, to predict the level of confidence in
meeting a project’s completion date and the amount of contingency time
needed to cover unexpected delays. CMRR project officials identified and
documented hundreds of risks to the project, but these risks were not
used in preparing a schedule risk analysis. For example, project officials
identified the following three risks that are likely to occur: (1) a necessary
electrical system upgrade that might not be completed in time for
construction activities, (2) uncertainties associated with the flow of
simultaneous design changes, and (3) noncompliance with certain quality
assurance standards for nuclear facilities. These risks could cause
delays, ranging anywhere from 1 to 5 years. Nevertheless, the project’s
schedule risk analysis identified only a 1-year schedule contingency for
the entire project. If NNSA is unable to successfully mitigate these risks
and if they occur together, there is a high likelihood that the 1-year
contingency that NNSA established may be exceeded. As a result, project
officials cannot be certain the schedule estimate contains all identified
risks in its risk analysis. Project officials told us that, before the project
baseline is established, they expect to have a schedule risk analysis that
includes identified risks and that they are in the early stages of developing
a plan to do so.

NNSA is taking steps to mitigate the risks that have been identified and,
because the project is still in early stages, many risks may be resolved.
For example, to mitigate the risk that the electrical system upgrade would
not be completed in time to avoid a delay in construction activities, project
officials have identified specific steps to help ensure that the upgrade is
performed in a timely manner. However, without a schedule risk analysis
that contains risks identified by CMRR project officials, NNSA cannot be
fully confident, once it decides to resume CMRR construction plans, that
sufficient schedule contingency is established to ensure that the project
will be completed on time and within estimated costs. As a result, overall
project costs could potentially exceed NNSA’s April 2010 estimate of
between $3.7 billion and $5.8 billion and NNSA had not yet determined
the impact to the project’s costs of its recent decision to defer CMRR
construction for at least 5 years. Appendix III contains additional


Page 13                                          GAO-12-337 Plutonium Research at Los Alamos
                        information on each practice and our assessment of the estimate
                        compared to best practices.


                        To replace the plutonium-related research capabilities in Los Alamos’s
NNSA Considered         deteriorating Chemistry and Metallurgy Research facility, NNSA
Several Options to      considered several options. In the end, NNSA decided to build a
                        minimally sized CMRR facility at Los Alamos with a broad suite of
Preserve Plutonium-     capabilities to meet nuclear weapons stockpile needs over the long-term.
Related Research        These capabilities would also be used to support plutonium-related
Capabilities, but       research needs of other departmental missions. NNSA evaluated these
                        options based on their expected effect on cost, schedule, risk, and ability
Ultimately Chose to     to meet the plutonium-related research needs of the nuclear weapons
Build a Minimally       stockpile stewardship program.

Sized Facility at Los   NNSA first focused on identifying and replacing the capabilities necessary
Alamos                  to maintain and modernize the nuclear weapons stockpile. Specifically,
                        these capabilities included those necessary to study the chemical and
                        metallurgic properties of plutonium pits to ensure that they are properly
                        produced, certified, and monitored over time so they remain safe and
                        reliable. 14 For example, to ensure that a nuclear weapon will function as
                        intended, the plutonium inside of the pits needs to meet strict
                        specifications. Meeting these specifications requires having the capability
                        to analyze and characterize the plutonium’s chemistry and material
                        properties. The specifications require NNSA to measure several chemical
                        attributes, including chemical composition and impurities, as well as the
                        pit’s structural attributes, such as the metal’s microscopic grain size, its
                        texture, any potential defects, and its weld characteristics. NNSA
                        identified at least 58 distinct capabilities that will be required in the new
                        facility to allow it to conduct the analyses necessary to build at least one




                        14
                          Actual pit production will not take place in CMRR, but it will provide plutonium-related
                        analytical capabilities to support pit production being done at another facility known as PF-
                        4, which is located within Los Alamos’s Technical Area 55. The purpose of the pit
                        production program is to re-establish the capability to produce pits, which were formerly
                        produced at the Rocky Flats Plant outside of Denver, Colorado, until 1989 when
                        operations there ceased. Responsibility for pit production was then assigned to Los
                        Alamos in 1996.




                        Page 14                                      GAO-12-337 Plutonium Research at Los Alamos
pit of every type currently in the stockpile. 15 NNSA determined that as
many as 79 capabilities may be required if NNSA needs to manufacture a
larger quantity of pits—up to its high estimate of 80 pits per year, which is
the Department of Defense’s published military requirement for pit
production. In addition to research capabilities, NNSA determined that the
new facility would need to provide other capabilities to support research
operations. In particular, long-term plutonium storage space is needed to
support plutonium-related research at CMRR.

To house these needed capabilities, NNSA assessed three potential
sizes for a new facility—22,500 square feet, 31,500 square feet, and
40,500 square feet. The 40,500 square foot option included about 10,500
square feet of unequipped space—known as contingency space—to
allow for program changes, such as increased pit manufacturing. In
addition, this contingency space could accommodate users outside Los
Alamos, such as researchers from Livermore. However, in 2004, NNSA
chose the smallest and least expensive option—22,500 square feet.
NNSA officials told us that cost was the primary driver of this decision.

NNSA’s choice to build a minimally sized facility was questioned in two
studies conducted subsequent to NNSA’s decision in 2004. Specifically, a
Los Alamos study conducted in 2006 found that increasing CMRR’s size
by 9,000 square feet—to a total of 31,500 square feet—would be the best
option based on cost, schedule, risk, and the facility’s ability to meet
plutonium-related research needs. 16 Furthermore, a separate
independent study prepared for NNSA in 2006 determined that adding
9,000 square feet to CMRR would lower risk and increase facility flexibility




15
  Weapon types in the nuclear weapons stockpile include the W78 and W87 warheads for
intercontinental ballistic missiles used by the Air Force; W76 and W88 warheads for
submarine launched ballistic missiles used by the Navy; B61 and B83 bombs used by the
Air Force; and the W80 warhead for missiles used by the Navy and Air Force.
16
  Los Alamos National Laboratory, Options for Plutonium-Related Missions and
Associated Facilities Between 2007 and 2022, LA-CP-06-0957 (Los Alamos, NM: Oct. 10,
2006).




Page 15                                  GAO-12-337 Plutonium Research at Los Alamos
but could cost an additional $179 million. 17 Nevertheless, NNSA officials
told us that a smaller sized facility had the best chance of minimizing
costs. NNSA officials acknowledge that the smaller size option poses
more risk because the facility will include no contingency space. This
space may be necessary, for example, to respond to potential increases
in pit production needs if in the future they unexpectedly approach or
exceed 80 pits per year. If this occurs, and no contingency space is
available, other plutonium-related research beyond that required for the
nuclear weapons stockpile will also likely be affected. According to NNSA
and Los Alamos officials, these risks could be mitigated by conducting
some nonnuclear weapons plutonium-related research at other facilities,
such as Los Alamos’s PF-4 pit production facility. However, PF-4 also has
ongoing laboratory and storage limitations and may not be able to
accommodate these other nonweapons plutonium activities.

Subsequent to its 2004 decision to build CMRR at Los Alamos, NNSA
continued to study other locations for consolidating plutonium-related
research within the nuclear security enterprise. Specifically, as part of its
development of a complexwide strategy to modernize nuclear research,
development, and production facilities that support the nuclear weapons
stockpile, NNSA studied consolidating the nation’s plutonium-related
research capabilities at Los Alamos, the Pantex Plant in Texas, the
Nevada National Security Site in Nevada, the Savannah River Site in
South Carolina, and the Y-12 National Security Complex in Tennessee. In
December 2008, NNSA decided to consolidate plutonium research at Los
Alamos and reaffirmed its earlier 2004 decision to locate the new CMRR
at Los Alamos. Consolidating plutonium-related research capabilities at
Los Alamos presented several advantages, including lower costs and
risks when compared to other locations. For example, colocating
plutonium analytical capabilities with Los Alamos’s pit manufacturing
capabilities reduced the costs and risks of protecting plutonium from
potential theft. As part of NNSA’s decision to consolidate plutonium
research at Los Alamos, NNSA also decided that the CMRR would be
used to support plutonium-related research needs of other non-weapons


17
  Techsource Incorporated, Independent Business Case Analysis for Construction of the
Chemistry and Metallurgy Research Replacement Nuclear Facility (Washington, D.C.:
Dec. 21, 2006). The $179 million cost difference represents the $4.175 billion estimate for
the larger 31,500 square foot facility less the $3.996 billion estimate for a smaller 22,500
square foot facility. The study results are based on estimated project costs from fiscal year
2007 through fiscal year 2022. Estimated costs are shown in fiscal year 2006 dollars and
are not adjusted to reflect present worth or net residual value.




Page 16                                      GAO-12-337 Plutonium Research at Los Alamos
                            activities, including nuclear nonproliferation activities; homeland security
                            activities, such as nuclear forensics and nuclear counterterrorism; waste
                            management; and material recycle and recovery programs. However, the
                            size of the planned CMRR facility—22,500 square feet—has not changed
                            since NNSA’s initial 2004 decision, which calls into question the facility’s
                            ability to support the needs of these other activities.


                            NNSA’s plans to construct the CMRR focused on meeting changing
CMRR May Meet               nuclear weapons stockpile requirements. However, CMRR may not be
Nuclear Weapons             able to accommodate all stockpile and other plutonium-related research
                            needs, particularly as other NNSA facilities reduce or end their plutonium
Stockpile                   research activities as a result of broader NNSA plans to consolidate its
Requirements but            plutonium activities.
May Not
Accommodate Other
Plutonium-Related
Research Needs
NNSA’s CMRR Plans Focus     NNSA’s plans to construct the CMRR primarily focus on maintaining
on Meeting Nuclear          plutonium-related research capabilities that are necessary for meeting
Weapons Stockpile           nuclear weapons stockpile requirements. NNSA designed the CMRR to
                            support the capabilities necessary for maintaining the safety and reliability
Requirements, But Some      of the nuclear stockpile––namely, the testing, manufacturing, and
Plutonium-Related Work      certification of the pits––and, in particular, plutonium-related research
for the Stockpile May Not   capabilities, such as analytical chemistry and materials characterization,
Be Accommodated             and associated special nuclear materials vault storage. More specifically,
                            in designing the CMRR, NNSA analyzed detailed data on past nuclear
                            weapons activities conducted at Los Alamos, including information on the
                            frequency of plutonium samples analyzed over time and the expected
                            annual requirement for manufacturing new pits to determine the
                            plutonium-related research capabilities the new facility would need to
                            meet NNSA weapons program requirements. For example, NNSA studied
                            the number of plutonium samples that had been processed in 2007 at the
                            old Chemistry and Metallurgy Research facility for analytical chemistry
                            and materials characterization work and used the number as an average
                            representation in assuming future workloads. In addition, NNSA
                            considered the numbers of specific pieces of equipment and the
                            associated square footage of laboratory space needed to conduct specific
                            analytical chemistry and material characterization work.



                            Page 17                              GAO-12-337 Plutonium Research at Los Alamos
In its planning, NNSA considered how plutonium-related capabilities in
the CMRR could meet changing stockpile requirements, including
NNSA’s established upper limit of producing 80 pits per year. NNSA
designed the facility to ensure that it can meet the pit production
requirements regardless of the specific number of pits produced—or, in
other words, the number of pits produced each year will not significantly
affect the capabilities NNSA will need in the new facility, although
capacity limits cap the quantity of new pits at 80 pits per year. For
example, NNSA’s 2009 CMRR Program Requirements document states
that the new facility will have laboratory spaces designed in a way that is
flexible and modular to accommodate changes in the mission and the
dynamic conditions associated with normal processing and maintenance
activities in a laboratory environment.

NNSA officials indicated that they are confident that the CMRR will
generally meet nuclear weapons activities needs and accommodate
changes in the nuclear weapons stockpile requirements, including the
ability to produce up to 80 pits per year. However, some weapons
activities capabilities that currently exist at other NNSA sites may no
longer be available to the nuclear security enterprise because of broader
NNSA modernization plans to consolidate plutonium activities. As part of
NNSA’s plan to consolidate plutonium related work at Los Alamos, the
CMRR was designed to absorb some plutonium-related research from
other facilities as those other facilities reduce or end their weapons
activities work. For example, Livermore’s Superblock facility is equipped
with the necessary systems to safely work with plutonium and to support
extending the life of certain warheads in the nuclear weapons stockpile.
Under NNSA’s strategy to consolidate plutonium work at Los Alamos, the
majority of Livermore’s plutonium is scheduled to be removed in 2012,
and some of this research will be discontinued at Superblock. NNSA
plans to have the CMRR take on much of this work; however, Livermore
officials told us they believe that NNSA may still lose some plutonium-
related capabilities once some research is discontinued at Superblock.
For example, NNSA may face a gap in the plutonium-related capabilities
necessary to help improve nuclear warhead surety—that is, safety,
security, and use control. NNSA has not planned for another facility to
take over this work, and NNSA officials told us that the CMRR has not
been designed to support this surety research. Furthermore, NNSA and
Los Alamos officials told us that NNSA may also lose some pit testing
capabilities that only take place in the Superblock at Livermore and are
expected to be discontinued there in 2013. Pit testing includes thermal,
vibration, and other environmental tests on pits that ensure that the
weapon can successfully function from the time it is in the stockpile until it


Page 18                               GAO-12-337 Plutonium Research at Los Alamos
                          is deployed and reaches a target. Livermore officials told us that CMRR
                          will not accommodate pit environmental testing because the systems
                          used to conduct the environmental tests could cause vibrations through
                          the rest of the facility. This could disrupt other work that requires precision
                          instrumentation. Livermore officials also told us that these pit
                          environmental testing capabilities are necessary to help meet nuclear
                          weapons stockpile requirements. Because the CMRR was not intended to
                          support all of these capabilities, NNSA will need to find another location if
                          this plutonium-related work currently being conducted at Livermore is to
                          be continued. NNSA has begun studying the extent to which the
                          environmental pit testing capabilities will be needed, and if so, where they
                          will be located. However, NNSA currently has no final plans for relocating
                          them elsewhere. 18


CMRR May Not Meet         DOE and NNSA conduct important plutonium-related research in other
Other Plutonium-Related   mission areas outside of nuclear weapons stockpile work, and it is
Research and Storage      unclear whether the CMRR as designed will be large enough to
                          accommodate these nonweapons activities because they have not
Needs As DOE and NNSA     comprehensively studied their long-term research and storage needs. A
Have Not Fully Analyzed   NNSA record of decision states that the CMRR will support other national
Programs Outside of       security missions involving plutonium-related research, including
Nuclear Weapons           nonproliferation, nuclear forensics, and nuclear counterterrorism
Stockpile Work            programs. For example, NNSA plans to use analytical chemistry
                          capabilities in CMRR to perform nuclear forensics work that would be
                          needed to, among other things, identify the source of and individuals
                          responsible for any planned or actual use of a nuclear device.

                          However, DOE and NNSA have not comprehensively studied the long-
                          term plutonium-related research and storage needs of programs outside
                          of NNSA’s nuclear weapons stockpile work and therefore cannot be sure
                          that the CMRR can accommodate them. In particular, DOE does not have
                          important information on departmentwide analytical chemistry and
                          material characterization research and storage needs, which can be
                          helpful in making fully informed planning decisions about its long-term
                          infrastructure and consolidation plans for the nuclear security enterprise.


                          18
                            NNSA has initiated a study considering implications of potentially upgrading Livermore’s
                          nuclear facility security and hazard categories for short periods to allow NNSA to continue
                          and maintain needed plutonium-related capabilities. An NNSA official told us that NNSA is
                          confident that the environmental pit testing capabilities will be maintained somewhere.




                          Page 19                                     GAO-12-337 Plutonium Research at Los Alamos
As we have previously reported, conceptual planning for a building—a
process by which an organization’s facility needs are identified and
understood—is the critical phase of any successful building project
development. 19 This conceptual planning results in a building design that
should be well defined according to an organization’s needs and include
input from all key stakeholders before it is designed. NNSA and Los
Alamos officials told us that the programs supporting mission areas
outside of the nuclear stockpile work—including NNSA’s Office of
National Technical Nuclear Forensics and Office of Fissile Materials
Disposition—were generally not involved in planning the CMRR. Los
Alamos officials said that they thought that there was too much time
before the new facility would be operating for other mission areas to know
their specific needs. However, by not including input from all the mission
areas during the design of CMRR, NNSA has risked not knowing all of the
potential needs and uses for the new facility to complement its important
missions outside of the nuclear weapons stockpile work.

NNSA and Los Alamos officials told us they are confident that the CMRR
will be able to support other missions’ needs for plutonium-related
research, but the facility’s design does not include dedicated space for
other missions’ research needs and includes little to no contingency
space. Los Alamos officials told us that shifting nuclear stockpile
requirements and changing pit production rates may impact specific
workloads and space capacity issues but that the CMRR is still too far
from becoming operational to estimate these impacts. For example, if
stockpile requirements are such that the higher boundary of pit production
capacity is needed—up to 80 pits per year—then the new facility will have
little, if any, space to address other missions’ research. 20 Moreover, in a
2008 analysis of the CMRR’s design, NNSA stated that Los Alamos is
uncertain that it will be able to conduct all of NNSA’s plutonium-related
research operations within the 22,500 square feet of laboratory space in
the facility. NNSA planning documents indicate that CMRR is intended to



19
  GAO, Guide to the Building Development Process: The First Phase: Conceptual
Planning, GAO-04-859G (Washington, D.C.: July 2004).
20
  NNSA and Los Alamos have considered using space in Los Alamos’ PF-4 plutonium
facility to handle additional plutonium-related research. However, NNSA officials told us
that operating at this high pit production range would also likely use all of PF-4’s capacity.
As a result, NNSA would have to consider reducing or eliminating other mission work
currently supported in PF-4 or modify CMRR to incorporate additional needed space at
additional cost.




Page 20                                       GAO-12-337 Plutonium Research at Los Alamos
support nonweapons activity needs only if additional capacity remains
after all weapons-related activities are supported. If additional capacity is
not available, NNSA may face the prospect of not being able to use the
new facility for one of its intended purposes of supporting certain
plutonium-related research for missions outside of nuclear weapons
stockpile work. A 2004 NNSA study suggested that this could effectively
result in national security, nonproliferation, and environmental
management programs potentially not performing in a cost-effective,
compliant, and timely manner. 21

In addition, the CMRR has been designed to support Los Alamos and
NNSA’s mission need to store significant quantities of nuclear material
associated with the plutonium operations in a safe and secure manner
using vault storage. Specifically, NNSA plans to shift all of Los Alamos’
current vault storage materials from its existing chemistry and metallurgy
facility and overflow inventory from the PF-4 facility to the CMRR. 22
However, Los Alamos officials told us that Los Alamos may not have
enough storage space even after the CMRR is complete. NNSA plans to
first use the newly available vault space in the CMRR for short-term, daily
storage of nuclear materials being used for programmatic work and then
use any remaining space for long-term storage. NNSA designed the
CMRR without much long-term vault storage because these materials
were initially planned to be shipped offsite for disposal. However, due to
broader departmental challenges with other NNSA sites receiving
materials for disposal, Los Alamos may not be able to ship its nuclear
material off-site. If this is the case, Los Alamos officials told us that they
may have to find additional long-term vault storage. This could also
potentially affect Los Alamos’ ability to receive nuclear materials from
other sites under NNSA’s consolidation strategy. In addition, Los Alamos
officials told us that NNSA is still considering facility layout options that
would allow for vault storage space to be configured for other operations
and lab space. If this space is used for functional laboratory space rather
than storage, less space will be available for short-term vault storage than
NNSA originally thought.



21
  NNSA, Los Alamos National Laboratory Chemistry and Metallurgy Research Facility
Replacement Project Mission Need Statement (Jan. 15, 2004).
22
  Los Alamos officials told us that one of the major uses of CMRR storage space will be to
relieve vault storage space at its plutonium facility that has already reached its available
storage capacity.




Page 21                                     GAO-12-337 Plutonium Research at Los Alamos
              Once NNSA resumes the CMRR project and constructs the facility,
Conclusions   CMRR will play an important role in ensuring the continued safety and
              reliability of the U.S. nuclear weapons stockpile. The CMRR can
              potentially offer NNSA the opportunity to improve efficiency, save costs,
              and reduce safety hazards for workers. Because of the facility’s
              importance to the stockpile, multibillion dollar price tag, the inherent
              challenges in building facilities that can safely and securely store
              plutonium, and NNSA’s ongoing difficulties managing large projects, it is
              critical that NNSA and Congress have accurate estimates of the project’s
              costs and schedules, particularly when the CMRR project is resumed.
              After facing a nearly six-fold increase in estimated cost and schedule
              delays, NNSA’s most recent cost and schedule estimates generally meet
              industry best practices, but there are important weaknesses that call
              these estimates’ reliability into question. For example, an independent
              cost estimate—the most rigorous method to validate major cost elements
              that is performed by an entity that has no stake in the approval of the
              project—has not yet been conducted. To its credit, NNSA plans to have
              an independent cost estimate conducted prior to the completion of
              CMRR’s project baseline once the project is resumed. With regard to the
              project’s schedule estimate, however, NNSA cannot yet provide high
              assurance that all project risks are fully accounted for in the project’s
              schedule risk analysis that is used for updating the project’s schedule
              contingency estimates. As a result, NNSA cannot yet be fully confident
              that, once it decides to resume the CMRR project, the project will meet its
              estimated completion date, which could lead to further delays and
              additional costs.

              However, reliable cost and schedule estimates for CMRR that fully meet
              industry best practices are of little use if DOE’s and NNSA’s mission
              needs are not met. Constructing CMRR is an important part of NNSA’s
              strategy to modernize its nuclear weapons facilities into a smaller and
              more responsive, efficient, and secure infrastructure to meet the changing
              requirements of the nuclear weapons stockpile. The CMRR was intended
              to support the plutonium-related research and storage needs of other
              DOE and NNSA national security missions and activities outside of the
              nuclear weapons stockpile work, including homeland security and nuclear
              nonproliferation activities; but because NNSA decided early in the project
              to reduce the size of the proposed facility to save money, CMRR may
              now lack the ability to accommodate these other research needs. In
              particular, the planned removal of most plutonium from Livermore
              presents NNSA with a dilemma in that the primary benefit of consolidating
              plutonium at Los Alamos—lower security costs—may be offset by the
              need to replace Lawrence Livermore National Laboratory’s plutonium


              Page 22                             GAO-12-337 Plutonium Research at Los Alamos
                      research, storage, and environmental testing capabilities. Importantly,
                      when NNSA decided to consolidate plutonium operations at Los Alamos,
                      it did not fully consider whether planned or existing facilities at Los
                      Alamos would be capable of continuing plutonium work being conducted
                      elsewhere. For example, CMRR was not intended to accommodate the
                      thermal, vibration, and other environmental pit testing that Livermore
                      currently conducts because the vibrations this type of testing creates
                      could disrupt other work at CMRR that requires precision instrumentation.
                      Nevertheless, this type of testing is necessary to meet nuclear weapons
                      stockpile requirements and so must be conducted somewhere. The full
                      extent of the potential shortfall in plutonium research capabilities is not
                      well-understood because DOE and NNSA have not comprehensively
                      assessed their plutonium-related research, storage, and environmental
                      testing needs. Plutonium research for the nuclear weapons stockpile and
                      for other missions may have to compete for limited laboratory and storage
                      space in CMRR and other facilities at Los Alamos, especially if the
                      demand for newly manufactured pits unexpectedly increases. As a result,
                      expansion of CMRR or construction of costly additional plutonium
                      research, storage, and testing facilities at Los Alamos or elsewhere may
                      be needed sometime in the future.


                      To strengthen cost and schedule estimates for the CMRR and ensure
Recommendations for   needed plutonium research needs are sufficiently accommodated, we
Executive Action      recommend that the Secretary of Energy take the following three actions:

                      1. Once NNSA resumes the CMRR project and prior to establishing a
                         new cost and schedule baseline, incorporate all key risks identified by
                         CMRR project officials into the project’s schedule risk analysis, and
                         ensure that this information is then used to update schedule
                         contingency estimates, as appropriate.

                      2. Conduct a comprehensive assessment of needed plutonium-related
                         research, storage, and environmental testing needs for nuclear
                         weapons stockpile activities as well as other missions currently
                         conducted at other NNSA and DOE facilities, with particular emphasis
                         on mitigating the consequences associated with eliminating plutonium
                         research, storage, and environmental testing capabilities from NNSA’s
                         Lawrence Livermore National Laboratory.

                      3. Using the results of this assessment, report to Congress detailing any
                         modifications to existing or planned facilities or any new facilities that
                         will be needed to support plutonium-related research, storage, and



                      Page 23                              GAO-12-337 Plutonium Research at Los Alamos
                         environmental testing needs for nuclear weapons stockpile activities
                         as well as other missions conducted by NNSA and DOE.

                     We provided NNSA with a draft of this report for its review and comment.
Agency Comments      In its written comments, reproduced in appendix IV, NNSA generally
and Our Evaluation   agreed with our recommendations to conduct a comprehensive
                     assessment of needed plutonium-related research, storage, and
                     environmental testing needs and to report to Congress on any
                     modifications to existing or planned facilities or any new facilities that will
                     be needed to support these needs. However, NNSA disagreed with our
                     recommendation to incorporate all key risks identified by project officials
                     into the project’s schedule risk analysis.

                     Specifically, NNSA stated that, subsequent to receiving our draft report
                     for its comments, the President’s budget request for fiscal year 2013 was
                     released and resulted in several changes to the funding and execution of
                     the CMRR project. In particular, construction of the CMRR is now to be
                     deferred for at least 5 years. Therefore, NNSA stated that it is conducting
                     additional analysis to determine the most effective way to provide
                     analytical chemistry, materials characterization, and storage capabilities
                     that were originally intended for the CMRR through the use of existing
                     infrastructure. As part of this analysis, NNSA stated that it will evaluate
                     options to use existing facilities at other sites. We believe this is
                     consistent with our recommendation that NNSA conduct a comprehensive
                     assessment of needed plutonium-related research, storage, and
                     environmental testing needs and that NNSA’s decision to defer
                     construction of the CMRR will give it sufficient time to conduct this
                     assessment.

                     NNSA also commented that it will continue to work with Congress and
                     other stakeholders as it adjusts its plutonium strategy. In our view, this is
                     also consistent with our recommendation to report to Congress on any
                     modifications to existing or planned facilities or any new facilities that will
                     be needed to support plutonium-related research, storage, and
                     environmental testing needs for nuclear weapons stockpile activities as
                     well as other missions conducted by NNSA and DOE.

                     With regard to our recommendation to incorporate all key risks identified
                     by CMRR project officials into the project’s schedule risk analysis, NNSA
                     commented that spending project money to update the CMRR project’s
                     schedule would not be prudent because of the construction delay.
                     Therefore, NNSA disagreed with the recommendation. NNSA stated that
                     its efforts in the near term would be focused on closing out the current


                     Page 24                                GAO-12-337 Plutonium Research at Los Alamos
design and that any future efforts will require updated cost and schedule
estimates. We agree with NNSA that it is not necessary to update the
project’s schedule at this time because of the recently announced
construction delay; however, we maintain that it is important that all
project risks are fully accounted for in the CMRR’s schedule once the
project is resumed. Therefore, we clarified our recommendation to specify
that NNSA should take action to ensure that the CMRR’s schedule risk
analysis is appropriately revised to account for all project risks when
NNSA resumes the project and before it establishes a new cost and
schedule baseline.


We are sending copies of this report to the Secretary of Energy; the
Administrator of NNSA; the Director, Office of Management and Budget;
the appropriate congressional committees; and other interested parties. In
addition, the report is available at no charge on the GAO website at
http://www.gao.gov.

If you or your staffs have any questions about this report, please contact
me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made major contributions to this report
are listed in appendix V.




Gene Aloise
Director, Natural Resources and Environment




Page 25                             GAO-12-337 Plutonium Research at Los Alamos
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              Our objectives were to examine (1) changes in the cost and schedule
              estimates for the construction of the facility and the extent to which its
              most recent estimates reflect best practices, (2) options the National
              Nuclear Security Administration (NNSA) considered to ensure that
              plutonium-related research activities could continue as needed, and (3)
              the extent to which NNSA’s plans to construct the Chemistry and
              Metallurgy Research Replacement Nuclear Facility (CMRR) and its
              consideration of options reflected changes in nuclear weapons stockpile
              requirements and other plutonium-related research needs.

              To examine the project’s cost and schedule estimates and the extent to
              which its current estimates reflect best practices, we reviewed relevant
              NNSA documents and met with agency and contractor officials on the
              changes that have occurred to date and the reasons for them. We
              compared NNSA’s most recent detailed cost and schedule estimates with
              industry best practices contained in our cost estimating and assessment
              guide and discussed them with project officials to give them the
              opportunity to provide feedback on our assessment. Our review examined
              specifically those NNSA cost estimates that were prepared in April 2010
              and schedule estimates, which at the time of our review were updated as
              of May 2011 or more recent for some portions of the schedule. As such,
              the cost and schedule estimates we reviewed do not reflect NNSA’s 5-
              year construction deferral recently announced in February 2012 and
              NNSA has not yet determined the potential long-term cost impact of this
              delay.

              To examine the options NNSA considered to continue plutonium-related
              analytical work, we reviewed NNSA and contractor documents on
              plutonium research needs and the various options available to meet those
              needs. We also met with NNSA and contractor officials to better
              understand how these options were analyzed to determine the best
              approach to fulfill NNSA’s mission. While NNSA evaluated options on
              how to best meet its mission needs, it may have also evaluated
              alternatives based on the environmental impact of building the CMRR. As
              such, our review examined the options NNSA assessed to maintain the
              capabilities for plutonium-related analytical chemistry, material
              characterization, and storage and did not address NNSA’s compliance
              with requirements of the National Environmental Policy Act. We also met
              with NNSA and contractor officials to gain a better understanding of how
              these options were analyzed to determine the best approach to fulfill
              NNSA’s mission.




              Page 26                              GAO-12-337 Plutonium Research at Los Alamos
Appendix I: Objectives, Scope, and
Methodology




To determine the extent to which NNSA’s plans reflect changes in nuclear
weapons stockpile requirements, we reviewed NNSA analyses that were
used to support CMRR project decisions and met with NNSA officials to
determine if these analyses were comprehensive and reflected up-to-date
nuclear weapons stockpile requirements. We also visited Los Alamos and
Lawrence Livermore National Laboratories. To ensure the data we used
were sufficiently reliable, we compared information gathered from a
variety of data sources. For example, we interviewed officials from both
Los Alamos and Lawrence Livermore National Laboratories to obtain
separate and independent perspectives on CMRR project plans. We
determined the data were sufficiently reliable for our purposes.

We conducted this performance audit from February 2011 through
February 2012 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.




Page 27                              GAO-12-337 Plutonium Research at Los Alamos
Appendix II: Summary Assessment of
                                        Appendix II: Summary Assessment of CMRR’s
                                        Cost Estimate Compared to Industry Best
                                        Practices


CMRR’s Cost Estimate Compared to Industry
Best Practices

Best practice     Overall
                            a                                                                           a
characteristic    assessment          Detailed best practice                     Detailed assessment
Comprehensive     Substantially met   The cost estimate includes all life        Substantially met. The total project cost for the
                                      cycle costs.                               construction of the Nuclear Facility is $4.2 billion.
                                                                                 Government and contractor costs are included.
                                                                                 However, operations and retirement costs are not
                                                                                 included. These costs were not included because
                                                                                 there was no mandate to estimate them. The cost
                                                                                 estimate spans from start of construction in June
                                                                                 2010 to completion in 2020 with a schedule
                                                                                 contingency through 2022.
                                      The cost estimate completely defines       Fully met. Technical descriptions were provided in
                                      the program, reflects the current          multiple documents such as the “CMRR Nuclear
                                      schedule, and is technically               Facility (NF) Estimate at Complete Forecast–April
                                      reasonable.                                2010,” the Los Alamos CMRR Mission Need
                                                                                 Statement, the Program Requirements Documents,
                                                                                 the WBS dictionary, and the “Final Environmental
                                                                                 Impact Statement for the Chemistry and Metallurgy
                                                                                 Project.”
                                      The cost estimate work breakdown           Partially met. The work breakdown structure and
                                      structure is product-oriented,             work breakdown structure dictionary are product
                                      traceable to the statement of              oriented and the work breakdown structure flows
                                      work/objective, and at an appropriate      down to level 4 of the program, project, or task. A
                                      level of detail to ensure that cost        statement of work was provided in the form of a
                                      elements are neither omitted nor           mission need statement; however, it is not easily
                                      double-counted.                            reconciled with the work breakdown structure
                                                                                 dictionary.
                                      The estimate documents all cost-           Fully met. Cost influencing ground rules and
                                      influencing ground rules and               assumptions can be found in the CMRR Estimate
                                      assumptions.                               Update Execution Plan. Budget constraints and
                                                                                 escalation are addressed. A list of high-level risk
                                                                                 drivers along with the handling costs and risk input
                                                                                 information was provided. Exclusions to the cost
                                                                                 estimate are noted in the documents.
Well documented   Substantially met   The documentation captures the             Partially met. The data was analyzed and high-level
                                      source data used, the reliability of the   cost drivers have been addressed as well as unit
                                      data, and how the data were                rates and quantities. Source data used to develop
                                      normalized.                                the estimate were found. The cost estimate was
                                                                                 based on historical data from other Department of
                                                                                 Energy (DOE) sites and the data was normalized.
                                                                                 However, the independent review team found
                                                                                 inconsistencies and discrepancies of quantities
                                                                                 (hours) and costs. In addition, the review team
                                                                                 reported that even though the basis of estimate
                                                                                 referred to current contract awards or proposals, no
                                                                                 reference was made to specific contracts or
                                                                                 proposals by date and number.




                                        Page 28                                        GAO-12-337 Plutonium Research at Los Alamos
                                 Appendix II: Summary Assessment of CMRR’s
                                 Cost Estimate Compared to Industry Best
                                 Practices




Best practice    Overall
                           a                                                                   a
characteristic   assessment    Detailed best practice                    Detailed assessment
                               The documentation describes in            Substantially met. While not explicitly stating what
                               sufficient detail the calculations        methodology was used, the pricing approach
                               performed and the estimating              summary indicates that the estimate was developed
                               methodology used to derive each           using a combination of the build-up method and
                               element’s cost.                           extrapolation from pricing information and
                                                                         productivity rates from other DOE sites. However
                                                                         the calculations involved were not clearly shown.
                               The documentation describes, step by      Substantially met. The documentation for the
                               step, how the estimate was developed      estimate contains a summary narrative about the
                               so that a cost analyst unfamiliar with    project as well as high-level cost summaries. The
                               the program could understand what         documentation discusses risk and contingency
                               was done and replicate it.                reserve. However, it does not address sensitivity
                                                                         although a sensitivity analysis was performed.
                                                                         Narrative on how the sensitivity analysis was
                                                                         conducted was not provided.
                               The documentation discusses the           Substantially met. There are technical descriptions
                               technical baseline description and the    discussed in the documentation that are consistent
                               data in the baseline is consistent with   with the basis of estimate and the work outlined in
                               the estimate.                             the detail cost estimate spreadsheets. However, we
                                                                         are unable to map specific technical descriptions as
                                                                         outlined in the requirements document to cost
                                                                         elements in the high-level or detailed cost
                                                                         estimates. During the site visit, project officials
                                                                         showed us how the scope of work in the work
                                                                         breakdown structure dictionary was written in a way
                                                                         to illustrate how the scope of work was captured.
                               The documentation provides evidence Partially met. Los Alamos policy states that reviews
                               that the cost estimate was reviewed shall be performed. According to project officials,
                               and accepted by management.         these reviews typically include an integrated project
                                                                   team review, functional manager review, directorate
                                                                   review, and in the case of projects of high
                                                                   complexity or risk, an external corporate review
                                                                   and/or DOE Los Alamos Site Office review. A
                                                                   CMRR functional review was held March 12, 2010,
                                                                   and the review of the current estimate was listed on
                                                                   the meeting agenda. However, without further
                                                                   documentation we are unable to determine whether
                                                                   or not a briefing was given to management that
                                                                   clearly explains the detail of the cost estimate—
                                                                   including presentation of lifecycle costs, ground
                                                                   rules and assumptions, estimating methods and
                                                                   data sources as they relate to each work
                                                                   breakdown structure element, results of sensitivity
                                                                   analysis, risk and uncertainty analysis, and if a
                                                                   desired level of confidence was reached.
                                                                   Additionally, it is not clear that an affordability
                                                                   analysis, contingency reserve, conclusions, or
                                                                   recommendations were discussed with
                                                                   management. The documentation also does not
                                                                   show management’s acceptance of the cost
                                                                   estimate.




                                 Page 29                                       GAO-12-337 Plutonium Research at Los Alamos
                                       Appendix II: Summary Assessment of CMRR’s
                                       Cost Estimate Compared to Industry Best
                                       Practices




Best practice    Overall
                           a                                                                         a
characteristic   assessment          Detailed best practice                   Detailed assessment
Accurate         Substantially met   The cost estimate results are            Substantially met. Risk and uncertainty analyses
                                     unbiased, not overly conservative or     were performed providing an 84 percent confidence
                                     optimistic, and based on an              level. There are three components that contribute to
                                     assessment of most likely costs.         the total contingency value established for the
                                                                              project—schedule, estimate, and technical and
                                                                              programmatic risk analysis.
                                     The estimate has been adjusted           Substantially met. The documentation contained
                                     properly for inflation.                  information on escalation rates. However, it is
                                                                              unclear how the cost estimate data were
                                                                              normalized. For example, costs are listed but are
                                                                              not labeled as constant or then-year dollars.
                                                                              Detailed calculations on how escalation was
                                                                              applied to the cost estimate are not documented.
                                     The estimate contains few, if any,       Substantially met. The numbers shown in the
                                     minor mistakes.                          estimate at complete document and the cost
                                                                              estimate spreadsheet are accurate and the
                                                                              independent review team found only one minor
                                                                              mistake in their review of the estimate. However,
                                                                              we were not provided access to the detailed
                                                                              calculations behind the spreadsheet to check that
                                                                              the estimate was calculated correctly.
                                     The cost estimate is regularly updated   Substantially met. The CMRR Project Control Plan
                                     to reflect significant changes in the    outlines a formal change control process that is to
                                     program so that it always reflects       be executed in accordance with the Los Alamos
                                     current status.                          Project Management and Site Services Directorate
                                                                              as well as the CMRR Baseline Change Control
                                                                              Board. These documents provide an approach to
                                                                              document, communicate, and approve potential
                                                                              changes to scope, cost, and schedule, and they
                                                                              provide the basis for incorporating changes into the
                                                                              project baseline and/or the forecast estimate at
                                                                              completion. These documents also describe the
                                                                              activities and responsibilities for making changes to
                                                                              the baseline.
                                     Any variances between planned and        Substantially met. Earned value is entered for each
                                     actual costs are documented,             work package based on the earned value method
                                     explained, and reviewed.                 indicated for that work package. Progress is
                                                                              reported in terms of percent complete by work
                                                                              package and is verified, analyzed, and reported to
                                                                              the project controls team. This information is then
                                                                              analyzed by the project controls team and control
                                                                              account managers and reviewed with CMRR
                                                                              management as the final reports are completed and
                                                                              published. However, there is no evidence of the
                                                                              cost estimate being updated to capture variances
                                                                              from the earned value system.




                                       Page 30                                      GAO-12-337 Plutonium Research at Los Alamos
                                   Appendix II: Summary Assessment of CMRR’s
                                   Cost Estimate Compared to Industry Best
                                   Practices




Best practice    Overall
                           a                                                                      a
characteristic   assessment      Detailed best practice                  Detailed assessment
                                 The estimate is based on a historical   Substantially met. Part of the estimate was
                                 record of cost estimating and actual    developed using the engineering build up method
                                 experiences from other comparable       which includes historical data from other
                                 programs.                               DOE/NNSA sites (Waste Treatment Plant, Mixed
                                                                         Oxide Fuel Fabrication Facility, and two chemical
                                                                         demilitarization facilities). The reliability of the data
                                                                         is documented where confidence levels associated
                                                                         with quantity, productivity, labor, and nonlabor
                                                                         pricing are addressed. However, for some of the
                                                                         data, the sources were not provided and there was
                                                                         no evidence that earned value data was used to
                                                                         develop or update the estimate.
Credible         Partially met   The cost estimate includes a            Substantially met. CMRR conducted some sort of
                                 sensitivity analysis—a technique that   sensitivity analysis. No documentation was given
                                 identifies a range of possible costs    providing a narrative on how the sensitivity analysis
                                 based on varying major assumptions,     was conducted—including whether high
                                 parameters, and data inputs.            percentages of cost were determined and how their
                                                                         parameters and assumptions were examined.
                                                                         Additionally, it cannot be determined whether the
                                                                         outcomes were evaluated for parameters most
                                                                         sensitive to change or how this analysis was
                                                                         applied to the estimate. However, during a site visit,
                                                                         Los Alamos officials provided a copy of a report that
                                                                         shows how a sensitivity analysis was applied to the
                                                                         nuclear facility cost estimate. For this assessment,
                                                                         a high and low range was determined. Some of the
                                                                         factors that were varied included overhead and
                                                                         General and Administrative rates, and escalation.




                                   Page 31                                      GAO-12-337 Plutonium Research at Los Alamos
                                 Appendix II: Summary Assessment of CMRR’s
                                 Cost Estimate Compared to Industry Best
                                 Practices




Best practice    Overall
                           a                                                                                a
characteristic   assessment    Detailed best practice                                 Detailed assessment
                               A risk and uncertainty analysis was                    Substantially met. The cost estimate includes
                               conducted that quantified the                          contingency costs for schedule ($99 million), cost
                               imperfectly understood risks and                       estimate ($508 million) and technical and
                               identified the effects of changing key                 programmatic risks ($404 million). While a schedule
                               cost driver assumptions and factors.                   risk analysis was performed that identified $99
                                                                                      million in schedule contingency, it is not clear how
                                                                                      this analysis was done as no supporting
                                                                                      documentation was provided. An independent
                                                                                      review team assessed the schedule risk analysis
                                                                                      and found that the risk model did not contain
                                                                                      enough detail to allow specific risk events to be
                                                                                      associated with the schedule activities they affect.
                                                                                      Documentation supporting the cost estimate ($508
                                                                                      million) risk and uncertainty analysis was conducted
                                                                                      via a Monte Carlo simulation which established an
                                                                                      84 percent confidence level for cost estimate
                                                                                      uncertainty. The process by which this analysis was
                                                                                      done is well documented and includes the
                                                                                      contingency level range results. However, this risk
                                                                                      and uncertainty analysis only reviewed classic cost
                                                                                      estimate contingency and did not assess technical,
                                                                                      programmatic or schedule risks. In addition, the
                                                                                      independent review team found that the cost risk
                                                                                      uncertainty analysis was done at a summary level
                                                                                      so it does not fully reflect the uncertainty of the
                                                                                      design costs associated with uncertainty related to
                                                                                      quantities or prices listed.
                               Major cost elements were crossed                       Partially met. Documentation was provided that
                               checked to see whether results were                    shows comparison of selected CMRR cost
                               similar.                                               elements against cost estimates of other sites.
                               An independent cost estimate was                       Partially met. An independent cost estimate was not
                               conducted by a group outside the                       conducted by a group outside of the acquiring
                               acquiring organization to determine                    organization. However, an independent cost review
                               whether other estimating methods                       was performed by the U.S. Army Corps of
                               produce similar results.                               Engineers in conjunction with an experienced
                                                                                      contractor. This independent cost review resulted in
                                                                                      the identification of key findings which require a
                                                                                      Corrective Action Plan. The independent cost
                                                                                      review focused on engineering design, and nuclear
                                                                                      facility special facility equipment engineering
                                                                                      design. The independent cost review team had 24
                                                                                      key findings and recommendations.
                                 Source: GAO analysis of CMRR project cost information.
                                 a
                                  The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no
                                 evidence that satisfies any of the practice. “Minimally met” means the CMRR provided evidence that
                                 satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that
                                 satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that
                                 satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that
                                 completely satisfies the practice.




                                 Page 32                                                    GAO-12-337 Plutonium Research at Los Alamos
Appendix III: Summary Assessment of
                                               Appendix III: Summary Assessment of CMRR’s
                                               Schedule Estimate Compared to Industry Best
                                               Practices


CMRR’s Schedule Estimate Compared to
Industry Best Practices

                                                                                                          a
Best practice           Explanation                                               Detailed assessment
Capturing all           The schedule should reflect all activities as defined     Fully met. The schedule integrates all of the effort of
activities              in the program’s work breakdown structure, to             NNSA, its contractor, and its major subcontractors.
                        include activities to be performed by both the
                        government and its contractors.
Sequencing all          The schedule should be planned so that it can meet        Substantially met. While we found that about 16 percent
activities              critical program dates. To meet this objective, key       of the activities were missing predecessors and
                        activities need to be logically sequenced in the          successors, or had constraints, lags, and leads, the
                        order that they are to be carried out. In particular,     majority (84 percent) of the activities were logically
                        activities that must finish before the start of other     sequenced. There are more than 2,400 activities (5
                        activities (i.e., predecessor activities) as well as      percent) with missing or dangling predecessors or
                        activities that cannot begin until other activities are   successors. There are summary tasks linked with logic (3
                        completed (i.e., successor activities) should be          percent), but we have determined that they do not affect
                        identified. By doing so, interdependencies among          the credibility of the schedule. There are 123 activities
                        activities that collectively lead to the                  (less than 1 percent) with start-to-finish logic. There are
                        accomplishment of events or milestones can be             460 activities (less than 1 percent) that have 10
                        established and used as a basis for guiding work          predecessors or more. There are 590 activities (1
                        and measuring progress.                                   percent) scheduled with constraints, in addition to or
                                                                                  substituting for complete logic.
Assigning resources     The schedule should reflect what resources (i.e.,         Substantially met. Not all activities in the project schedule
to all activities       labor, material, and overhead) are needed to do the       are resource loaded—only 3,757 activities (8 percent) out
                        work, whether all required resources will be              of the 45,429 activities with positive remaining duration
                        available when they are needed, and whether any           have resources assigned in the schedule we received.
                        funding or time constraints exist.                        However, there is credible evidence that the program and
                                                                                  Los Alamos manage resources in various ways outside
                                                                                  the project schedule and that their resource solutions are
                                                                                  fed back to the project schedule so that it is feasible
                                                                                  given resource limits.
Establishing the        The schedule should realistically reflect how long        Substantially met. There are 1,642 activities (4 percent)
duration of all         each activity will take to execute. In determining the    with durations 44 days or greater, which means that the
activities              duration of each activity, the same rationale, data,      majority of the activities (96 percent) have activities that
                        and assumptions used for cost estimating should           are of short duration. Contributing to this is the rolling
                        be used. Further, these durations should be as            wave approach to the schedule, where the near-term
                        short as possible and they should have specific           activities are detailed while activities further in the future
                        start and end dates. Excessively long periods             are left in large planning packages until they become
                        needed to execute an activity should prompt further       near-term, at which point they are broken down into their
                        decomposition of the activity so that shorter             component activities.
                        execution durations will result.
Integrating schedule The schedule should be horizontally integrated,              Substantially met. As discussed previously in the
activities horizontally meaning that it should link the products and              “sequencing all activities,” there are activities missing
                        outcomes associated with already sequenced                predecessor and successor logic as well as the presence
                        activities. These links are commonly referred to as       of constraints, lags, and leads that call into question the
                        handoffs and serve to verify that activities are          adequacy of horizontal traceability. Vertical traceability
                        arranged in the right order to achieve aggregated         was confirmed. The schedule hierarchy includes five
                        products or outcomes. The schedule should also be         levels, increasing in detail and specificity from top to
                        vertically integrated, meaning that traceability exists   bottom.
                        among varying levels of activities and supporting
                        tasks and subtasks. Such mapping or alignment
                        among levels enables different groups to work to
                        the same master schedule.




                                               Page 33                                         GAO-12-337 Plutonium Research at Los Alamos
                                              Appendix III: Summary Assessment of CMRR’s
                                              Schedule Estimate Compared to Industry Best
                                              Practices




                                                                                                       a
Best practice           Explanation                                             Detailed assessment
Establishing the        Using scheduling software, the critical path—the        Substantially met. This schedule’s critical path has 5,479
critical path for all   longest duration path through the sequenced list of     activities with zero or negative total float. There are so
activities              key activities—should be identified. The                many critical activities because of a number of
                        establishment of a program’s critical path is           constraints on intermediate milestones which is causing
                        necessary for examining the effects of any activity     negative float on paths to those activities. However, these
                        slipping along this path. Potential problems that       activities do not all drive the final delivery. Los Alamos
                        may occur on or near the critical path should also      officials said that when they baseline the schedule, they
                        be identified and reflected in the scheduling of the    plan to remove many of the constraints that are causing
                        time for high-risk activities.                          negative float. Many of these constraints are there to
                                                                                enable Los Alamos to monitor status of intermediate
                                                                                milestones.
Identifying float       The schedule should identify float so that schedule     Substantially met. Of the remaining activities, 22 percent
between activities      flexibility can be determined. As a general rule,       have unexplained large positive and large negative total
                        activities along the critical path typically have the   float values. Even with agency review, these were
                        least amount of float.                                  present in the schedule. The total float values in many
                                                                                cases are several years long. There are 4,611 activities
                                                                                (10 percent) that have total float over 1,000 days or about
                                                                                3.8 years. These high total float values are likely related
                                                                                to the incomplete logic described in the “sequencing all
                                                                                activities” best practice.
Conducting a            A schedule risk analysis should be performed using      Minimally met. There is no evidence that a risk analysis
schedule risk           a schedule built using a good critical path method      has been conducted on this schedule or any summary
analysis                and data about project schedule risks, as well as       schedule derived from this schedule. Los Alamos officials
                        statistical analysis techniques (such as Monte          said that they have conducted a risk analysis using
                        Carlo) to predict the level of confidence in meeting    Monte Carlo simulation based on a prior and more
                        a program’s completion date. This analysis focuses      concise schedule a full year before the version we
                        not only on critical path activities but also on        reviewed was developed. The version we reviewed
                        activities near the critical path, since they can       contained 90,000 activities and was developed in the
                        potentially affect program status.                      Spring of 2010—a full year after Los Alamos conducted
                                                                                its risk analysis and Monte Carlo simulation. Los Alamos
                                                                                did not conduct a risk analysis on this more recent
                                                                                schedule, nor did it prepare and simulate a summary
                                                                                schedule based on this more recent schedule. The
                                                                                summary schedule that Los Alamos simulated was based
                                                                                on critical and near critical paths. This schedule
                                                                                comprised the main, secondary and tertiary critical paths.
                                                                                As a result, we believe that the schedule did not cover
                                                                                the entire work of the project, and therefore may have
                                                                                excluded some activities or paths that have risk sufficient
                                                                                to affect the finish date. Instead, Los Alamos selected
                                                                                about 2,100 activities based on total float, but this
                                                                                practice is risky because they may not have included all
                                                                                of the activities that risks in the risk register may affect.




                                              Page 34                                        GAO-12-337 Plutonium Research at Los Alamos
                                             Appendix III: Summary Assessment of CMRR’s
                                             Schedule Estimate Compared to Industry Best
                                             Practices




                                                                                                                   a
Best practice          Explanation                                                       Detailed assessment
Using logic and        The schedule should use logic and durations in                    Fully met. The CMRR schedule is updated at least
durations to           order to reflect realistic start and completion dates             monthly, although much of it is updated weekly. The
determine the start    for program activities. The schedule should be                    schedule integrity is checked after each update and
and completion dates   continually monitored to determine when forecasted                metrics are compiled on problems to determine if the
                       completion dates differ from the planned dates,                   schedule’s integrity is improving with each update.
                       which can be used to determine whether schedule                   There are no activities in the past that lack the
                       variances will affect downstream work. Maintaining                designation of actual start or actual finish. There are
                       the integrity of the schedule logic is not only                   some activities on or after the data date that have actual
                       necessary to reflect true status, but is also required            start or finish designations, but that may be because
                       before conducting a schedule risk analysis.                       there are 15 schedules combined in the Integrated
                                                                                         Master Schedule and some were updated somewhat
                                                                                         after May 9, 2011.
                                             Source: GAO analysis of CMRR project schedule information.
                                             a
                                              The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no
                                             evidence that satisfies any part of the practice. “Minimally met” means the CMRR provided evidence
                                             that satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that
                                             satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that
                                             satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that
                                             completely satisfies the practice.




                                             Page 35                                                      GAO-12-337 Plutonium Research at Los Alamos
Appendix IV: Comments from the National
              Appendix IV: Comments from the National
              Nuclear Security Administration



Nuclear Security Administration




              Page 36                                   GAO-12-337 Plutonium Research at Los Alamos
Appendix IV: Comments from the National
Nuclear Security Administration




Page 37                                   GAO-12-337 Plutonium Research at Los Alamos
Appendix IV: Comments from the National
Nuclear Security Administration




Page 38                                   GAO-12-337 Plutonium Research at Los Alamos
Appendix V: GAO Contact and Staff
                  Appendix V: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Gene Aloise, (202) 512-3841 or aloisee@gao.gov
GAO Contact
                  In addition to the contact named above, Ryan T. Coles, Assistant
Acknowledgments   Director; John Bauckman; Jennifer Echard; Eugene Gray; David T. Hulett;
                  Jonathan Kucskar; Alison O’Neill; Christopher Pacheco; Tim Persons;
                  Karen Richey; Stacey Steele; Vasiliki Theodoropoulos; and Mary Welch
                  made key contributions to this report.




(361267)
                  Page 39                             GAO-12-337 Plutonium Research at Los Alamos
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.