United States Government Accountability Office GAO Report to the Subcommittee on Energy and Water Development, Committee on Appropriations, U.S. Senate March 2012 MODERNIZING THE NUCLEAR SECURITY ENTERPRISE New Plutonium Research Facility at Los Alamos May Not Meet All Mission Needs GAO-12-337 March 2012 MODERNIZING THE NUCLEAR SECURITY ENTERPRISE New Plutonium Research Facility at Los Alamos May Highlights of GAO-12-337, a report to the Not Meet All Mission Needs Subcommittee on Energy and Water Development, Committee on Appropriations, U.S. Senate Why GAO Did This Study What GAO Found Plutonium—a man-made element The estimated cost to construct the CMRR has greatly increased since NNSA’s produced by irradiating uranium in initial plans, and the project’s schedule has been significantly delayed. According nuclear reactors—is vital to the nuclear to its most recent estimates prepared in April 2010, NNSA determined that the weapons stockpile. Much of the CMRR will cost between $3.7 billion and $5.8 billion—nearly a six-fold increase nation’s current plutonium research from the initial estimate. Construction has also been repeatedly delayed and, in capabilities are housed in aging February 2012 after GAO provided its draft report to NNSA for comment, NNSA facilities at Los Alamos National decided to defer CMRR construction by at least an additional 5 years, bringing Laboratory in New Mexico. These the total delay to between 8 and 12 years from NNSA’s original plans. facilities pose safety hazards. The Infrastructure-related design changes and longer-than-expected overall project National Nuclear Security duration have contributed to these cost increases and delays. GAO’s review of Administration (NNSA) has decided to construct a multibillion dollar Chemistry NNSA’s April 2010 cost and schedule estimates for CMRR found that the and Metallurgy Research Replacement estimates were generally well prepared, but important weaknesses remain. For Nuclear Facility (CMRR) to modernize example, a high-quality schedule requires a schedule risk analysis that the laboratory’s capabilities to analyze incorporates known risks to predict the level of confidence in meeting a project’s and store plutonium. GAO was asked completion date and the amount of contingency time needed to cover to examine (1) the cost and schedule unexpected delays. CMRR project officials identified hundreds of risks to the estimates to construct CMRR and the project, but GAO found that these risks were not used in preparing a schedule extent to which its most recent risk analysis. As a result of these weaknesses, NNSA cannot be fully confident, estimates reflect best practices, (2) once it decides to resume the CMRR project, that the project will be completed options NNSA considered to ensure on time and within estimated costs. that needed plutonium research activities could continue, and (3) the NNSA considered several options to preserve its plutonium-related research extent to which NNSA's plans reflected capabilities in its decision to build CMRR at Los Alamos. NNSA assessed three changes in stockpile requirements and different sizes for a new facility—22,500, 31,500, and 40,500 square feet. In other plutonium research needs. GAO 2004, NNSA chose the smallest option. NNSA officials stated that cost was the reviewed NNSA and contractor project primary driver of the decision, but that building a smaller facility would result in design documents and visited Los trade-offs, including the elimination of contingency space. In the end, NNSA Alamos and another plutonium facility decided to build a minimally-sized CMRR facility at Los Alamos with a broad at Lawrence Livermore National suite of capabilities to meet nuclear weapons stockpile needs over the long-term. Laboratory in California. These capabilities would also be used to support plutonium-related research needs of other departmental missions. What GAO Recommends NNSA’s plans to construct CMRR focused on meeting nuclear weapons stockpile GAO is making recommendations to requirements, but CMRR may not meet all stockpile and other plutonium-related improve CMRR’s schedule risk research needs. NNSA analyzed data on past workload and the expected need analysis and to conduct an for new weapon components to help ensure CMRR’s design included the assessment of plutonium research needs. NNSA agreed with GAO’s necessary plutonium-related research capabilities for maintaining the safety and recommendations to assess plutonium reliability of the nuclear stockpile. However, some plutonium research, storage, research needs, but disagreed that its and environmental testing capabilities that exist at Lawrence Livermore National schedule risk analysis should be Laboratory may no longer be available after NNSA consolidates plutonium- revised, citing its recent decision to related research at Los Alamos. Furthermore, NNSA conducts important defer the project. GAO clarified the plutonium-related research in other areas such as homeland security and nuclear recommendation to specify that NNSA nonproliferation, but it has not comprehensively analyzed plutonium research and should take action when it resumes the storage needs of these other programs outside of its nuclear weapons stockpile project. work and therefore cannot be sure that the CMRR plans will effectively accommodate these needs. As a result, expansion of CMRR or construction of more plutonium research and storage facilities at Los Alamos or elsewhere may View GAO-12-337. For more information, be needed in the future, potentially further adding to costs. contact Gene Aloise at (202) 512-3841 or email@example.com. United States Government Accountability Office Contents Letter 1 Background 5 CMRR’s Initial Cost Estimate Has Significantly Increased and Its Schedule Has Been Delayed 9 NNSA Considered Several Options to Preserve Plutonium-Related Research Capabilities, but Ultimately Chose to Build a Minimally Sized Facility at Los Alamos 14 CMRR May Meet Nuclear Weapons Stockpile Requirements but May Not Accommodate Other Plutonium-Related Research Needs 17 Conclusions 22 Recommendations for Executive Action 23 Agency Comments and Our Evaluation 24 Appendix I Objectives, Scope, and Methodology 26 Appendix II Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices 28 Appendix III Summary Assessment of CMRR’s Schedule Estimate Compared to Industry Best Practices 33 Appendix IV Comments from the National Nuclear Security Administration 36 Appendix V GAO Contact and Staff Acknowledgments 39 Tables Table 1: Extent to Which CMRR’s Cost Estimate Met Industry Best Practices 11 Table 2: Extent to Which CMRR’s Schedule Estimate Met Industry Best Practices 12 Page i GAO-12-337 Plutonium Research at Los Alamos Abbreviations CMRR Chemistry and Metallurgy Research Replacement Nuclear Facility DOE Department of Energy NNSA National Nuclear Security Administration This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-12-337 Plutonium Research at Los Alamos United States Government Accountability Office Washington, DC 20548 March 26, 2012 The Honorable Dianne Feinstein Chairman The Honorable Lamar Alexander Ranking Member Subcommittee on Energy and Water Development Committee on Appropriations United States Senate Plutonium—a man-made radioactive element produced by irradiating uranium in nuclear reactors—is vital to the nation’s nuclear weapons stockpile. Plutonium is used in “pits”—the spherical central core of a nuclear weapon that is compressed with high explosives to create a nuclear explosion. Several kilograms of plutonium are sufficient to make a nuclear bomb, so plutonium must be stored under extremely high security to protect it from theft. In addition, exposure to small quantities is dangerous to human health, so that even inhaling a few micrograms creates a long-term risk of lung, liver, and bone cancer and inhaling larger doses can cause immediate lung injuries and death. Also, if not safely contained and managed, plutonium can be unstable and spontaneously ignite under certain conditions. Therefore, any facility that stores or conducts research on plutonium must be robustly designed to safely prevent the uncontrolled release of hazardous material to the environment and to securely store the material to protect it from potential theft. Much of the nation’s current plutonium research and development capabilities are housed at the Los Alamos National Laboratory in New Mexico. The laboratory is one of the National Nuclear Security Administration’s (NNSA) two primary laboratories responsible for designing nuclear weapons components that contain plutonium; the other is Lawrence Livermore National Laboratory in California. 1 Los Alamos has been supporting the production of new pits since the closure of the Department of Energy’s (DOE) Rocky Flats Plant near Denver, Colorado, 1 NNSA was created by the National Defense Authorization Act for Fiscal Year 2000, Pub. L. No. 106-65 (1999). It is a separate, semiautonomous agency within the Department of Energy, with responsibility for the nation’s nuclear weapons, nonproliferation, and naval reactors programs. Page 1 GAO-12-337 Plutonium Research at Los Alamos in 1989, as well as homeland security activities, energy programs, and nuclear nonproliferation activities. The Chemistry and Metallurgy Research nuclear facility at Los Alamos conducts plutonium-related research that is crucial to effectively maintain the nuclear weapons stockpile. However, the facility is nearly 60 years old, and its aging infrastructure poses safety hazards. In addition, the facility is situated on a seismic fault line, raising concerns about the effect of earthquakes on the safety and security of plutonium used for research or stored at the facility. Because of these concerns, NNSA has decided to construct a Chemistry and Metallurgy Research Replacement Nuclear Facility (CMRR) at Los Alamos that will (1) modernize the laboratory’s capabilities to analyze plutonium and (2) store plutonium in vaults that provide a secure environment that protects against its accidental or intentional misuse and minimizes health risks for workers and the surrounding communities. 2 Originally estimated to begin construction in 2008, the project has experienced several delays and, in February 2012 after we had provided a draft of this report to NNSA for its comments, NNSA announced that it had decided to defer construction of the facility for at least 5 years. NNSA’s proposed construction of the CMRR is part of a larger strategic effort to consolidate nuclear materials from other locations across the United States and to modernize nuclear research, development, and production facilities that support the nuclear weapons stockpile. For example, NNSA’s plans call for some of the plutonium-related research currently conducted at a high security facility at Lawrence Livermore National Laboratory known as Superblock to be transferred to CMRR. NNSA’s plans also call for much of the plutonium currently stored at Livermore’s Superblock to be consolidated at Los Alamos for continued research activities, and for the other material not needed for research at Livermore or Los Alamos to be stored at DOE’s Savannah River Site pending final disposition. In addition, the nuclear weapons stockpile’s requirements for plutonium are evolving. Specifically, the New Strategic Arms Reduction Treaty that the United States and Russia signed in April 2010 is to reduce the number of deployed strategic nuclear warheads by 2 For the purposes of this report, CMRR refers to the design and construction of the nuclear facility portion of NNSA’s CMRR project. The scope of this report does not include the first phase of the project—the Radiological Laboratory, Utility, and Office Building— which is much smaller in scope and cost and is substantially complete. Page 2 GAO-12-337 Plutonium Research at Los Alamos 30 percent. As a result of this treaty and NNSA’s approach for modernizing the stockpile through the refurbishment of existing weapons, demand for newly manufactured pits has fluctuated in recent years. In light of these fluctuations, NNSA’s current strategy is to design the CMRR around a broad suite of capabilities—equipment, processes, and expertise—that it anticipates may be needed to fulfill the stockpile’s requirements regardless of the specific demand for pits. Because of the extensive safety and security measures required to analyze and store plutonium, the cost of constructing new nuclear facilities is typically a multibillion dollar venture. In the past, we have reported on several major DOE and NNSA construction projects that faced cost increases and schedule delays. 3 DOE’s long-standing difficulties in preparing cost and schedule estimates is one reason contract management in NNSA and DOE’s Office of Environmental Management is on our list of federal programs at high risk of fraud, waste, abuse, and mismanagement. 4 Because other federal agencies have also had problems developing high-quality cost and schedule estimates, we issued a cost-estimating guide in March 2009, consisting of best practices drawn from across industry and government, to assist agencies to develop cost and schedule estimates that are well-documented, comprehensive, accurate, and credible. 5 In this context, you asked us to review NNSA’s plans for constructing the CMRR. Specifically, our objectives were to examine (1) NNSA’s cost and schedule estimates for the construction of the facility and the extent to 3 GAO, Nuclear Weapons: National Nuclear Security Administration’s Plans for Its Uranium Processing Facility Should Better Reflect Funding Estimates and Technology Readiness, GAO-11-103 (Washington, D.C.: Nov. 19, 2010); GAO, Department of Energy: Actions Needed to Develop High-Quality Cost Estimates for Construction and Environmental Cleanup Projects, GAO-10-199 ( Washington, D.C.: Jan. 14, 2010); GAO, Nuclear Weapons: National Nuclear Security Administration Needs To Better Manage Risks Associated with the Modernization of Its Kansas City Plant, GAO-10-115 (Washington, D.C.: Oct. 23, 2009): GAO, Nuclear Waste: Uncertainties and Questions about Costs and Risks Persist with DOE’s Tank Waste Cleanup Strategy at Hanford, GAO-09-913 (Washington, D.C.: Sept. 30, 2009); and GAO, Department of Energy: Contract and Project Management Concerns at the National Nuclear Security Administration and Office of Environmental Management, GAO-09-406T (Washington, D.C.: Mar. 4, 2009). 4 GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: Feb. 16, 2011). 5 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). Page 3 GAO-12-337 Plutonium Research at Los Alamos which its most recent estimates reflect best practices, (2) options NNSA considered to ensure that plutonium-related research activities will continue as needed, and (3) the extent to which NNSA’s plans to construct the CMRR and its consideration of options reflected changes in nuclear weapons stockpile requirements and the plutonium-related research needs of other departmental missions. To examine NNSA’s cost and schedule estimates for the CMRR project and the extent to which its current estimates reflect best practices, we reviewed relevant NNSA documents and met with agency and Los Alamos project officials to discuss the changes in the estimates that have occurred to date and the reasons for them. We compared NNSA’s most recent cost and schedule estimates––prepared in April 2010––with best practices contained in our cost estimating guide and gave project officials the opportunity to provide and discuss feedback on our assessment. To examine the options NNSA considered to ensure that its plutonium- related work will continue, we reviewed NNSA and contractor documents on plutonium research needs and the various options available to meet those needs. We also met with NNSA and contractor officials to better understand how these options were analyzed to determine the best approach to fulfill NNSA’s mission. To determine the extent to which NNSA’s plans reflect changes in nuclear weapons stockpile requirements, we reviewed NNSA analyses that were used to support CMRR project decisions and met with NNSA officials to determine if these analyses were comprehensive and reflected up-to-date nuclear weapons stockpile requirements. We also visited the Los Alamos and Lawrence Livermore national laboratories. To ensure the data we used were sufficiently reliable, we compared information gathered from a variety of data sources. For example, we interviewed officials from both Los Alamos and Livermore to obtain separate and independent perspectives on CMRR project plans. We determined the data were sufficiently reliable for our purposes. We conducted this performance audit from February 2011 through February 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix I contains a detailed description of our scope and methodology. Page 4 GAO-12-337 Plutonium Research at Los Alamos In the mid-1990s, Congress directed DOE to develop the Stockpile Background Stewardship Program to provide a single, highly integrated technical program for maintaining the continued safety and reliability of the nuclear weapons stockpile. Stockpile stewardship comprises activities associated with conducting nuclear weapons research, design, and development; maintaining the knowledge base and capabilities to support nuclear weapons testing; and assessing and certifying nuclear weapons safety and reliability. Stockpile stewardship includes operations associated with producing, maintaining, refurbishing, surveilling, and dismantling the nuclear weapons stockpile. The Stockpile Stewardship Program’s objectives were updated as a result of the 2010 Nuclear Posture Review, which establishes the U.S. nuclear policy for the next 5 to 10 years, including the nation’s nuclear weapons stockpile requirements. 6 The Nuclear Posture Review and the Stockpile Stewardship Program reinforce the New Strategic Arms Reduction Treaty between the United States and Russia. As part of this treaty, the United States has agreed to reduce the size of its strategic nuclear weapons stockpile from a maximum of 2,200 to 1,550 weapons, with the remaining weapons in the stockpile continuing to be an essential element of U.S. defense strategy. Nuclear stockpile requirements include a pit production capacity that is defined by estimating the number of pits NNSA needs to manufacture annually to effectively support the nuclear weapons stockpile. The demand for pits has fluctuated over the past decade for various reasons. Until 2005, NNSA planned to produce pits in a large-scale manufacturing plant to be built called the Modern Pit Facility, which would have increased pit production capacity per year to a range of 125 to 450 pits. This project was terminated and, at around the same time, NNSA began to study a new approach for modernizing the stockpile, called the Reliable Replacement Warhead program, which would have produced 50 pits per year and which was also short-lived. Through this program, NNSA would have designed new weapon components, including pits, to be safer and easier to manufacture, maintain, dismantle, and certify without nuclear testing. Since 2008, NNSA’s guidance has established pit capacity for future production at about 20 pits per year, with an upper range limit of 80 pits per year. In addition, NNSA has recently determined that pit lifetimes are longer than anticipated and that it may increase the reuse of existing pits, reducing the demand for newly manufactured pits. Currently, pit 6 Department of Defense, Nuclear Posture Review Report, (Washington, D.C.: April 2010). Page 5 GAO-12-337 Plutonium Research at Los Alamos capacity requirements are uncertain and still in flux. Demand may again fluctuate as a result of the Nuclear Posture Review and changes to the Stockpile Stewardship Program. For example, there are still unknowns in implementing the Nuclear Posture Review and modernization work on each nuclear weapon type may require a varied number of new pits. To execute the activities to maintain and refurbish the nation’s existing nuclear weapons stockpile, NNSA oversees eight sites that comprise its nuclear security enterprise—formerly known as the nuclear weapons complex—which includes three national weapons laboratories, four production plants, and a test site, all of which carry out missions to support NNSA’s programs. One of these sites, Los Alamos National Laboratory, plays a crucial role in carrying out NNSA’s maintenance of the nuclear weapons stockpile, including (1) production of weapons components, (2) assessment and certification of the nuclear weapons stockpile, (3) surveillance of weapons components and weapon systems, (4) assurance of the safe and secure storage of strategic materials, and (5) management of excess plutonium inventories. Los Alamos was established in 1943 during the Manhattan Project in northern New Mexico. It is a multidisciplinary, multipurpose institution primarily engaged in theoretical and experimental research and development. A significant portion of Los Alamos’ work is focused on ensuring that nuclear weapons stockpile needs are met. Since 2000, pit production has been established within the Plutonium Facility Complex at Los Alamos’s Technical Area 55, and certified pits have been produced over the past 5 years in that facility. A particularly important facility at Los Alamos within Technical Area 55 is the nearly 60-year-old Chemistry and Metallurgy Research facility. The facility has unique capabilities for performing analytical chemistry, material characterization, and research and development related to plutonium. This includes activities that support the manufacturing, development, and surveillance of nuclear weapons pits; programs to extend the life of nuclear weapons in the stockpile; and nuclear weapon dismantlement efforts. This pit production mission support work was first assigned to Los Alamos in 1996. NNSA also currently maintains some plutonium-related research capabilities at other facilities, such as Livermore’s Superblock facility. These capabilities are necessary components of NNSA’s overall stockpile management strategy. NNSA and DOE also use the unique plutonium-related capabilities located at Los Alamos and Livermore to support the plutonium-related research needs of other national security missions and activities outside of the nuclear weapons stockpile work, including nuclear nonproliferation activities; homeland security activities, such as nuclear forensics and Page 6 GAO-12-337 Plutonium Research at Los Alamos nuclear counterterrorism; waste management; and material recycle and recovery programs. The Chemistry and Metallurgy Research facility was initially designed and constructed to comply with building codes in effect during the late 1940s and early 1950s. In 1992, recognizing that some of the utility systems and structural components were aging, outmoded, and generally deteriorating, DOE began upgrading the facility. These upgrades addressed specific safety, reliability, consolidation, and security issues with the intent of extending the useful life of the facility for an additional 20 to 30 years. However, beginning in about 1997 and continuing to the present, a series of additional operational and safety concerns have surfaced. In particular, a 1998 seismic study identified two small parallel faults beneath the northern portion of the Chemistry and Metallurgy Research facility. The presence of these faults raised concerns about the structural integrity of the building in the event of an earthquake. DOE and NNSA determined that, over the long term, Los Alamos could not continue to operate the mission-critical support capabilities in the existing Chemistry and Metallurgy Research facility at an acceptable level of risk to worker safety and health. To ensure that NNSA can fulfill its national security mission for the next 50 years in a safe, secure, and environmentally sound manner, NNSA decided in 2004 to construct a replacement facility, known as the CMRR. 7 Federal agencies, including DOE and NNSA, have experienced long- standing difficulties in completing major projects within cost and on schedule. To provide assistance in preparing high-quality cost and schedule estimates, we compiled best practices used throughout government and industry and, in March 2009, issued a guide outlining the criteria for high-quality cost and schedule estimates. Specifically, our guide identified four characteristics of a high-quality, reliable cost estimate: (1) credible, (2) well-documented, (3) accurate, and (4) 7 In 2004, NNSA determined that it needed a nuclear facility to relocate certain analytical capabilities from existing facilities at Los Alamos, which are near end-of-life, as part of NNSA’s plans for maintaining and certifying the nation’s nuclear weapons stockpile. In deciding whether to build a new facility or instead use or refurbish other existing facilities, a 2006 Los Alamos study determined that a new nuclear facility should be built because the fundamental objectives of NNSA’s strategic planning for the nuclear weapons complex could not be achieved without it. Page 7 GAO-12-337 Plutonium Research at Los Alamos comprehensive. 8 In addition, our cost guide lays out 12 key steps that should result in high-quality cost estimates and hundreds of best practices drawn from across industry and government for carrying out these steps. For example, one of the key steps includes conducting an independent cost estimate––that is, one generated by an entity that has no stake in the approval of the project but uses the same detailed technical information as the project estimate. Having an independent entity perform such a cost estimate and comparing it to the project team’s estimate provides an unbiased test of whether the project team’s cost estimate is reasonable. Our guide also identified nine best practices for effectively estimating schedules: (1) capturing key activities, (2) sequencing key activities, (3) assigning resources to key activities, (4) establishing the duration of key activities, (5) integrating key activities horizontally and vertically, (6) establishing the critical path for key activities, (7) identifying total float (i.e., the time that activities can slip before the delay affects the completion date), (8) performing a risk analysis of the schedule, and (9) updating the schedule using logic and durations to determine dates. 9 Many of these practices have also been incorporated into DOE’s recent guidance for establishing performance baselines. 10 8 In the context of our cost guide, a cost estimate is the summation of individual cost elements, using established methods and valid data, to estimate the future costs of a project, based on what is known today. 9 GAO-09-3SP. 10 DOE, Performance Baseline Guide, DOE G 413.3-5A (Washington, D.C.: Sept. 23, 2011). Although there is not a one-to-one correlation, many of the GAO-identified best practices, are also suggested schedule development practices in DOE’s Performance Baseline Guide. DOE also requires that NNSA establish a project performance baseline to document estimated project cost and schedule for planned capital projects in order to measure the project’s performance. See DOE, Program and Project Management for the Acquisition of Capital Assets, DOE O 413.3B (Washington, D.C.: Nov. 29, 2010). Page 8 GAO-12-337 Plutonium Research at Los Alamos The estimated cost to construct the CMRR, according to estimates CMRR’s Initial Cost prepared in April 2010, is nearly six times higher than the project’s initial Estimate Has cost estimate that was prepared in 2005. The project’s estimated completion date has also been delayed by at least 8 to 12 years. Our Significantly review of these most recent detailed cost and schedule estimates for the Increased and Its CMRR project found that the estimates generally reflect best practices, Schedule Has Been but are not yet entirely reliable. Delayed Estimated Cost to Since CMRR was first proposed, its costs have risen significantly, and its Construct CMRR Has schedule has been repeatedly delayed. Specifically, in 2005, when DOE Increased by Almost Six- developed initial plans for CMRR, it estimated that the project would cost from $745 million to $975 million and would be completed between 2013 Fold, and Its Scheduled and 2017. This estimate was prepared using preliminary information— Completion Has Been before a detailed project design was substantially under way—and was Delayed therefore considered by DOE to be a rough estimate. In April 2010, NNSA estimated that the CMRR will cost between $3.7 and $5.8 billion—a nearly six-fold increase from the initial estimate—and that construction will be complete by 2020—a 3- to 7-year delay. In February 2012, after we had provided NNSA with a draft of this report for its comments, NNSA announced that it had decided to defer CMRR construction by at least an additional 5 years, bringing the total delay from NNSA’s original plans to 8 to 12 years. NNSA officials explained that the majority of the cost increases occurred because of changes to the facility’s design and because of project delays. Specifically, • Modifications to the facility’s design. To address concerns about seismic activity, the project design was modified to strengthen the facility to withstand a potential earthquake. For example, significant design changes resulted from the need to thicken the concrete walls to satisfy increasingly stringent seismic requirements. In addition, to proceed to final design, project officials had to evaluate the potential effects of an earthquake on the facility’s complex ventilation system. This effort included several studies, consultations with vendors and other designers, and an assessment of the availability of equipment that would meet seismic requirements. Overall, Los Alamos estimates the seismic related design changes increased the project costs by almost $500 million. Page 9 GAO-12-337 Plutonium Research at Los Alamos • Delays in the construction start date and longer overall project duration. CMRR construction was originally expected to begin in 2008, but was first delayed until 2013 and is now not expected to begin before 2018. The initial delay in starting construction from 2008 to 2013 had varying causes, including facility design changes described previously as well as the additional time needed for NNSA to determine where and how to consolidate plutonium operations in the nuclear security enterprise, according to project officials. This delay starting construction pushed the estimated construction completion date from between 2013 and 2017 to 2020—3 to 7 years later than initially expected. At the time, the facility was expected to be operational in 2022. 11 These delays further increased costs, partly because inflation meant that equipment and materials became more expensive as time passed. In addition, the longer project duration also contributes to increases in the cost of workers’ wages and salaries. Overall, project officials estimate that about $1.2 billion in additional costs resulted from these schedule delays. In February 2012, NNSA announced another significant project delay—at least an additional 5- year deferral in starting the construction of the CMRR—resulting in a total of an 8 to 12 year delay from NNSA’s original plans. However, NNSA has not yet determined the impact to the project’s costs as a result of this additional delay. NNSA’s Most Recent Cost Our review of NNSA’s most recent cost and schedule estimates for the and Schedule Estimates CMRR construction project found that the estimates were generally well Generally Meet Industry prepared but that important weaknesses remain. Specifically, we found that the CMRR cost estimate prepared in April 2010 exhibits most of the Best Practices, but Are Not characteristics of high-quality, reliable cost estimates. As identified by the Yet Entirely Reliable professional cost-estimating community and documented in our cost- estimating guide, a high-quality cost estimate is comprehensive, well- documented, accurate, and credible. 12 Our review of the CMRR cost estimate found that the cost estimate exhibits three of the four characteristics of a high-quality estimate by being substantially comprehensive, well documented, and accurate, but only partially 11 The delay between completion of construction and the date of operation allows for equipment to be prepared for use and workers to be trained on new equipment, among other things. 12 GAO-09-3SP. Page 10 GAO-12-337 Plutonium Research at Los Alamos credible, as shown in table 1. Appendix II contains additional information about each of the four general best practice characteristics and our assessment of the estimate compared to detailed best practices. Table 1: Extent to Which CMRR’s Cost Estimate Met Industry Best Practices a Best practice characteristic Overall assessment Comprehensive Substantially met Well documented Substantially met Accurate Substantially met Credible Partially met Source: GAO analysis of CMRR project cost information. a The ratings we used in this analysis are as follows: “Not met” means CMRR provided no evidence that satisfies any of the characteristic. “Minimally met” means CMRR provided evidence that satisfies a small portion of the characteristic. “Partially met” means CMRR provided evidence that satisfies about half of the characteristic. “Substantially met” means CMRR provided evidence that satisfies a large portion of the characteristic. “Fully met” means CMRR provided complete evidence that satisfies the entire characteristic. The CMRR cost estimate only partially met industry best practices for credibility because project officials did not use alternate methods to crosscheck major cost elements to see whether the results were similar under different estimating methods. In addition, according to our guide, there are varying methods of validating an estimate, but the most rigorous method is the independent cost estimate that is generated by an entity that has no stake in the approval of the project. Conducting an independent cost estimate is especially important at major milestones because it provides senior decision makers with a more objective assessment of the likely cost of a project. A second, less rigorous method for validating a project’s cost estimate—an independent cost review— focuses on examining the estimate’s supporting documentation and interviewing relevant staff. Independent cost reviews address only the cost estimate’s high-value, high-risk, and high-interest aspects without evaluating the remainder of the estimate. An independent cost review on the entire CMRR project was initiated in 2011, but the more rigorous method of validating—conducting an independent cost estimate—has only been used on a small portion of the project representing about 6 Page 11 GAO-12-337 Plutonium Research at Los Alamos percent of the project’s total costs. 13 According to NNSA officials, DOE orders do not require NNSA to seek an independent cost estimate until just prior to establishing the project baseline, and project officials told us NNSA is preparing to have one conducted before the project baseline is established. However, until a quality independent cost estimate is completed on the entire project or another means of validating the estimate for the project, DOE and NNSA officials cannot be confident that the current cost estimate is completely credible. With regard to CMRR’s schedule, the project’s schedule estimate fully met two and substantially met six out of nine best practices for a high- quality schedule as identified by our guide and minimally met one. For example, two of the best practices the estimate fully met concerned how well it (1) captured all of the project’s activities, including design, construction, and other tasks that collectively form a comprehensive schedule, and (2) is successfully kept up-to-date. Table 2 lists best practices along with our assessment of the extent to which the project’s schedule met each best practice. Table 2: Extent to Which CMRR’s Schedule Estimate Met Industry Best Practices a Best practice Overall assessment Capturing all activities Fully met Sequencing all activities Substantially met Assigning resources to all activities Substantially met Establishing the duration of all activities Substantially met Integrating schedule activities Substantially met Establishing the critical path for all activities Substantially met Identifying float between activities Substantially met Conducting a schedule risk analysis Minimally met Updating the schedule using logic and durations to Fully met determine dates Source: GAO analysis of CMRR project schedule information. 13 An independent cost estimate was initiated in 2011 covering the design and infrastructure needed to complete the CMRR nuclear facility project, such as concrete batch plants and equipment storage, which represents only $250 million of the estimated total project cost. Page 12 GAO-12-337 Plutonium Research at Los Alamos a The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no evidence that satisfies any part of a practice. “Minimally met” means the CMRR provided evidence that satisfies a small portion of a practice. “Partially met” means the CMRR provided evidence that satisfies about half of a practice. “Substantially met” means the CMRR provided evidence that satisfies a large portion of a practice. “Fully met” means the CMRR provided evidence that completely satisfies a practice. The CMRR schedule estimate minimally met industry best practices for conducting a schedule risk analysis. Namely, according to our guide, a high-quality schedule requires a schedule risk analysis that uses already identified risks, among other things, to predict the level of confidence in meeting a project’s completion date and the amount of contingency time needed to cover unexpected delays. CMRR project officials identified and documented hundreds of risks to the project, but these risks were not used in preparing a schedule risk analysis. For example, project officials identified the following three risks that are likely to occur: (1) a necessary electrical system upgrade that might not be completed in time for construction activities, (2) uncertainties associated with the flow of simultaneous design changes, and (3) noncompliance with certain quality assurance standards for nuclear facilities. These risks could cause delays, ranging anywhere from 1 to 5 years. Nevertheless, the project’s schedule risk analysis identified only a 1-year schedule contingency for the entire project. If NNSA is unable to successfully mitigate these risks and if they occur together, there is a high likelihood that the 1-year contingency that NNSA established may be exceeded. As a result, project officials cannot be certain the schedule estimate contains all identified risks in its risk analysis. Project officials told us that, before the project baseline is established, they expect to have a schedule risk analysis that includes identified risks and that they are in the early stages of developing a plan to do so. NNSA is taking steps to mitigate the risks that have been identified and, because the project is still in early stages, many risks may be resolved. For example, to mitigate the risk that the electrical system upgrade would not be completed in time to avoid a delay in construction activities, project officials have identified specific steps to help ensure that the upgrade is performed in a timely manner. However, without a schedule risk analysis that contains risks identified by CMRR project officials, NNSA cannot be fully confident, once it decides to resume CMRR construction plans, that sufficient schedule contingency is established to ensure that the project will be completed on time and within estimated costs. As a result, overall project costs could potentially exceed NNSA’s April 2010 estimate of between $3.7 billion and $5.8 billion and NNSA had not yet determined the impact to the project’s costs of its recent decision to defer CMRR construction for at least 5 years. Appendix III contains additional Page 13 GAO-12-337 Plutonium Research at Los Alamos information on each practice and our assessment of the estimate compared to best practices. To replace the plutonium-related research capabilities in Los Alamos’s NNSA Considered deteriorating Chemistry and Metallurgy Research facility, NNSA Several Options to considered several options. In the end, NNSA decided to build a minimally sized CMRR facility at Los Alamos with a broad suite of Preserve Plutonium- capabilities to meet nuclear weapons stockpile needs over the long-term. Related Research These capabilities would also be used to support plutonium-related Capabilities, but research needs of other departmental missions. NNSA evaluated these options based on their expected effect on cost, schedule, risk, and ability Ultimately Chose to to meet the plutonium-related research needs of the nuclear weapons Build a Minimally stockpile stewardship program. Sized Facility at Los NNSA first focused on identifying and replacing the capabilities necessary Alamos to maintain and modernize the nuclear weapons stockpile. Specifically, these capabilities included those necessary to study the chemical and metallurgic properties of plutonium pits to ensure that they are properly produced, certified, and monitored over time so they remain safe and reliable. 14 For example, to ensure that a nuclear weapon will function as intended, the plutonium inside of the pits needs to meet strict specifications. Meeting these specifications requires having the capability to analyze and characterize the plutonium’s chemistry and material properties. The specifications require NNSA to measure several chemical attributes, including chemical composition and impurities, as well as the pit’s structural attributes, such as the metal’s microscopic grain size, its texture, any potential defects, and its weld characteristics. NNSA identified at least 58 distinct capabilities that will be required in the new facility to allow it to conduct the analyses necessary to build at least one 14 Actual pit production will not take place in CMRR, but it will provide plutonium-related analytical capabilities to support pit production being done at another facility known as PF- 4, which is located within Los Alamos’s Technical Area 55. The purpose of the pit production program is to re-establish the capability to produce pits, which were formerly produced at the Rocky Flats Plant outside of Denver, Colorado, until 1989 when operations there ceased. Responsibility for pit production was then assigned to Los Alamos in 1996. Page 14 GAO-12-337 Plutonium Research at Los Alamos pit of every type currently in the stockpile. 15 NNSA determined that as many as 79 capabilities may be required if NNSA needs to manufacture a larger quantity of pits—up to its high estimate of 80 pits per year, which is the Department of Defense’s published military requirement for pit production. In addition to research capabilities, NNSA determined that the new facility would need to provide other capabilities to support research operations. In particular, long-term plutonium storage space is needed to support plutonium-related research at CMRR. To house these needed capabilities, NNSA assessed three potential sizes for a new facility—22,500 square feet, 31,500 square feet, and 40,500 square feet. The 40,500 square foot option included about 10,500 square feet of unequipped space—known as contingency space—to allow for program changes, such as increased pit manufacturing. In addition, this contingency space could accommodate users outside Los Alamos, such as researchers from Livermore. However, in 2004, NNSA chose the smallest and least expensive option—22,500 square feet. NNSA officials told us that cost was the primary driver of this decision. NNSA’s choice to build a minimally sized facility was questioned in two studies conducted subsequent to NNSA’s decision in 2004. Specifically, a Los Alamos study conducted in 2006 found that increasing CMRR’s size by 9,000 square feet—to a total of 31,500 square feet—would be the best option based on cost, schedule, risk, and the facility’s ability to meet plutonium-related research needs. 16 Furthermore, a separate independent study prepared for NNSA in 2006 determined that adding 9,000 square feet to CMRR would lower risk and increase facility flexibility 15 Weapon types in the nuclear weapons stockpile include the W78 and W87 warheads for intercontinental ballistic missiles used by the Air Force; W76 and W88 warheads for submarine launched ballistic missiles used by the Navy; B61 and B83 bombs used by the Air Force; and the W80 warhead for missiles used by the Navy and Air Force. 16 Los Alamos National Laboratory, Options for Plutonium-Related Missions and Associated Facilities Between 2007 and 2022, LA-CP-06-0957 (Los Alamos, NM: Oct. 10, 2006). Page 15 GAO-12-337 Plutonium Research at Los Alamos but could cost an additional $179 million. 17 Nevertheless, NNSA officials told us that a smaller sized facility had the best chance of minimizing costs. NNSA officials acknowledge that the smaller size option poses more risk because the facility will include no contingency space. This space may be necessary, for example, to respond to potential increases in pit production needs if in the future they unexpectedly approach or exceed 80 pits per year. If this occurs, and no contingency space is available, other plutonium-related research beyond that required for the nuclear weapons stockpile will also likely be affected. According to NNSA and Los Alamos officials, these risks could be mitigated by conducting some nonnuclear weapons plutonium-related research at other facilities, such as Los Alamos’s PF-4 pit production facility. However, PF-4 also has ongoing laboratory and storage limitations and may not be able to accommodate these other nonweapons plutonium activities. Subsequent to its 2004 decision to build CMRR at Los Alamos, NNSA continued to study other locations for consolidating plutonium-related research within the nuclear security enterprise. Specifically, as part of its development of a complexwide strategy to modernize nuclear research, development, and production facilities that support the nuclear weapons stockpile, NNSA studied consolidating the nation’s plutonium-related research capabilities at Los Alamos, the Pantex Plant in Texas, the Nevada National Security Site in Nevada, the Savannah River Site in South Carolina, and the Y-12 National Security Complex in Tennessee. In December 2008, NNSA decided to consolidate plutonium research at Los Alamos and reaffirmed its earlier 2004 decision to locate the new CMRR at Los Alamos. Consolidating plutonium-related research capabilities at Los Alamos presented several advantages, including lower costs and risks when compared to other locations. For example, colocating plutonium analytical capabilities with Los Alamos’s pit manufacturing capabilities reduced the costs and risks of protecting plutonium from potential theft. As part of NNSA’s decision to consolidate plutonium research at Los Alamos, NNSA also decided that the CMRR would be used to support plutonium-related research needs of other non-weapons 17 Techsource Incorporated, Independent Business Case Analysis for Construction of the Chemistry and Metallurgy Research Replacement Nuclear Facility (Washington, D.C.: Dec. 21, 2006). The $179 million cost difference represents the $4.175 billion estimate for the larger 31,500 square foot facility less the $3.996 billion estimate for a smaller 22,500 square foot facility. The study results are based on estimated project costs from fiscal year 2007 through fiscal year 2022. Estimated costs are shown in fiscal year 2006 dollars and are not adjusted to reflect present worth or net residual value. Page 16 GAO-12-337 Plutonium Research at Los Alamos activities, including nuclear nonproliferation activities; homeland security activities, such as nuclear forensics and nuclear counterterrorism; waste management; and material recycle and recovery programs. However, the size of the planned CMRR facility—22,500 square feet—has not changed since NNSA’s initial 2004 decision, which calls into question the facility’s ability to support the needs of these other activities. NNSA’s plans to construct the CMRR focused on meeting changing CMRR May Meet nuclear weapons stockpile requirements. However, CMRR may not be Nuclear Weapons able to accommodate all stockpile and other plutonium-related research needs, particularly as other NNSA facilities reduce or end their plutonium Stockpile research activities as a result of broader NNSA plans to consolidate its Requirements but plutonium activities. May Not Accommodate Other Plutonium-Related Research Needs NNSA’s CMRR Plans Focus NNSA’s plans to construct the CMRR primarily focus on maintaining on Meeting Nuclear plutonium-related research capabilities that are necessary for meeting Weapons Stockpile nuclear weapons stockpile requirements. NNSA designed the CMRR to support the capabilities necessary for maintaining the safety and reliability Requirements, But Some of the nuclear stockpile––namely, the testing, manufacturing, and Plutonium-Related Work certification of the pits––and, in particular, plutonium-related research for the Stockpile May Not capabilities, such as analytical chemistry and materials characterization, Be Accommodated and associated special nuclear materials vault storage. More specifically, in designing the CMRR, NNSA analyzed detailed data on past nuclear weapons activities conducted at Los Alamos, including information on the frequency of plutonium samples analyzed over time and the expected annual requirement for manufacturing new pits to determine the plutonium-related research capabilities the new facility would need to meet NNSA weapons program requirements. For example, NNSA studied the number of plutonium samples that had been processed in 2007 at the old Chemistry and Metallurgy Research facility for analytical chemistry and materials characterization work and used the number as an average representation in assuming future workloads. In addition, NNSA considered the numbers of specific pieces of equipment and the associated square footage of laboratory space needed to conduct specific analytical chemistry and material characterization work. Page 17 GAO-12-337 Plutonium Research at Los Alamos In its planning, NNSA considered how plutonium-related capabilities in the CMRR could meet changing stockpile requirements, including NNSA’s established upper limit of producing 80 pits per year. NNSA designed the facility to ensure that it can meet the pit production requirements regardless of the specific number of pits produced—or, in other words, the number of pits produced each year will not significantly affect the capabilities NNSA will need in the new facility, although capacity limits cap the quantity of new pits at 80 pits per year. For example, NNSA’s 2009 CMRR Program Requirements document states that the new facility will have laboratory spaces designed in a way that is flexible and modular to accommodate changes in the mission and the dynamic conditions associated with normal processing and maintenance activities in a laboratory environment. NNSA officials indicated that they are confident that the CMRR will generally meet nuclear weapons activities needs and accommodate changes in the nuclear weapons stockpile requirements, including the ability to produce up to 80 pits per year. However, some weapons activities capabilities that currently exist at other NNSA sites may no longer be available to the nuclear security enterprise because of broader NNSA modernization plans to consolidate plutonium activities. As part of NNSA’s plan to consolidate plutonium related work at Los Alamos, the CMRR was designed to absorb some plutonium-related research from other facilities as those other facilities reduce or end their weapons activities work. For example, Livermore’s Superblock facility is equipped with the necessary systems to safely work with plutonium and to support extending the life of certain warheads in the nuclear weapons stockpile. Under NNSA’s strategy to consolidate plutonium work at Los Alamos, the majority of Livermore’s plutonium is scheduled to be removed in 2012, and some of this research will be discontinued at Superblock. NNSA plans to have the CMRR take on much of this work; however, Livermore officials told us they believe that NNSA may still lose some plutonium- related capabilities once some research is discontinued at Superblock. For example, NNSA may face a gap in the plutonium-related capabilities necessary to help improve nuclear warhead surety—that is, safety, security, and use control. NNSA has not planned for another facility to take over this work, and NNSA officials told us that the CMRR has not been designed to support this surety research. Furthermore, NNSA and Los Alamos officials told us that NNSA may also lose some pit testing capabilities that only take place in the Superblock at Livermore and are expected to be discontinued there in 2013. Pit testing includes thermal, vibration, and other environmental tests on pits that ensure that the weapon can successfully function from the time it is in the stockpile until it Page 18 GAO-12-337 Plutonium Research at Los Alamos is deployed and reaches a target. Livermore officials told us that CMRR will not accommodate pit environmental testing because the systems used to conduct the environmental tests could cause vibrations through the rest of the facility. This could disrupt other work that requires precision instrumentation. Livermore officials also told us that these pit environmental testing capabilities are necessary to help meet nuclear weapons stockpile requirements. Because the CMRR was not intended to support all of these capabilities, NNSA will need to find another location if this plutonium-related work currently being conducted at Livermore is to be continued. NNSA has begun studying the extent to which the environmental pit testing capabilities will be needed, and if so, where they will be located. However, NNSA currently has no final plans for relocating them elsewhere. 18 CMRR May Not Meet DOE and NNSA conduct important plutonium-related research in other Other Plutonium-Related mission areas outside of nuclear weapons stockpile work, and it is Research and Storage unclear whether the CMRR as designed will be large enough to accommodate these nonweapons activities because they have not Needs As DOE and NNSA comprehensively studied their long-term research and storage needs. A Have Not Fully Analyzed NNSA record of decision states that the CMRR will support other national Programs Outside of security missions involving plutonium-related research, including Nuclear Weapons nonproliferation, nuclear forensics, and nuclear counterterrorism Stockpile Work programs. For example, NNSA plans to use analytical chemistry capabilities in CMRR to perform nuclear forensics work that would be needed to, among other things, identify the source of and individuals responsible for any planned or actual use of a nuclear device. However, DOE and NNSA have not comprehensively studied the long- term plutonium-related research and storage needs of programs outside of NNSA’s nuclear weapons stockpile work and therefore cannot be sure that the CMRR can accommodate them. In particular, DOE does not have important information on departmentwide analytical chemistry and material characterization research and storage needs, which can be helpful in making fully informed planning decisions about its long-term infrastructure and consolidation plans for the nuclear security enterprise. 18 NNSA has initiated a study considering implications of potentially upgrading Livermore’s nuclear facility security and hazard categories for short periods to allow NNSA to continue and maintain needed plutonium-related capabilities. An NNSA official told us that NNSA is confident that the environmental pit testing capabilities will be maintained somewhere. Page 19 GAO-12-337 Plutonium Research at Los Alamos As we have previously reported, conceptual planning for a building—a process by which an organization’s facility needs are identified and understood—is the critical phase of any successful building project development. 19 This conceptual planning results in a building design that should be well defined according to an organization’s needs and include input from all key stakeholders before it is designed. NNSA and Los Alamos officials told us that the programs supporting mission areas outside of the nuclear stockpile work—including NNSA’s Office of National Technical Nuclear Forensics and Office of Fissile Materials Disposition—were generally not involved in planning the CMRR. Los Alamos officials said that they thought that there was too much time before the new facility would be operating for other mission areas to know their specific needs. However, by not including input from all the mission areas during the design of CMRR, NNSA has risked not knowing all of the potential needs and uses for the new facility to complement its important missions outside of the nuclear weapons stockpile work. NNSA and Los Alamos officials told us they are confident that the CMRR will be able to support other missions’ needs for plutonium-related research, but the facility’s design does not include dedicated space for other missions’ research needs and includes little to no contingency space. Los Alamos officials told us that shifting nuclear stockpile requirements and changing pit production rates may impact specific workloads and space capacity issues but that the CMRR is still too far from becoming operational to estimate these impacts. For example, if stockpile requirements are such that the higher boundary of pit production capacity is needed—up to 80 pits per year—then the new facility will have little, if any, space to address other missions’ research. 20 Moreover, in a 2008 analysis of the CMRR’s design, NNSA stated that Los Alamos is uncertain that it will be able to conduct all of NNSA’s plutonium-related research operations within the 22,500 square feet of laboratory space in the facility. NNSA planning documents indicate that CMRR is intended to 19 GAO, Guide to the Building Development Process: The First Phase: Conceptual Planning, GAO-04-859G (Washington, D.C.: July 2004). 20 NNSA and Los Alamos have considered using space in Los Alamos’ PF-4 plutonium facility to handle additional plutonium-related research. However, NNSA officials told us that operating at this high pit production range would also likely use all of PF-4’s capacity. As a result, NNSA would have to consider reducing or eliminating other mission work currently supported in PF-4 or modify CMRR to incorporate additional needed space at additional cost. Page 20 GAO-12-337 Plutonium Research at Los Alamos support nonweapons activity needs only if additional capacity remains after all weapons-related activities are supported. If additional capacity is not available, NNSA may face the prospect of not being able to use the new facility for one of its intended purposes of supporting certain plutonium-related research for missions outside of nuclear weapons stockpile work. A 2004 NNSA study suggested that this could effectively result in national security, nonproliferation, and environmental management programs potentially not performing in a cost-effective, compliant, and timely manner. 21 In addition, the CMRR has been designed to support Los Alamos and NNSA’s mission need to store significant quantities of nuclear material associated with the plutonium operations in a safe and secure manner using vault storage. Specifically, NNSA plans to shift all of Los Alamos’ current vault storage materials from its existing chemistry and metallurgy facility and overflow inventory from the PF-4 facility to the CMRR. 22 However, Los Alamos officials told us that Los Alamos may not have enough storage space even after the CMRR is complete. NNSA plans to first use the newly available vault space in the CMRR for short-term, daily storage of nuclear materials being used for programmatic work and then use any remaining space for long-term storage. NNSA designed the CMRR without much long-term vault storage because these materials were initially planned to be shipped offsite for disposal. However, due to broader departmental challenges with other NNSA sites receiving materials for disposal, Los Alamos may not be able to ship its nuclear material off-site. If this is the case, Los Alamos officials told us that they may have to find additional long-term vault storage. This could also potentially affect Los Alamos’ ability to receive nuclear materials from other sites under NNSA’s consolidation strategy. In addition, Los Alamos officials told us that NNSA is still considering facility layout options that would allow for vault storage space to be configured for other operations and lab space. If this space is used for functional laboratory space rather than storage, less space will be available for short-term vault storage than NNSA originally thought. 21 NNSA, Los Alamos National Laboratory Chemistry and Metallurgy Research Facility Replacement Project Mission Need Statement (Jan. 15, 2004). 22 Los Alamos officials told us that one of the major uses of CMRR storage space will be to relieve vault storage space at its plutonium facility that has already reached its available storage capacity. Page 21 GAO-12-337 Plutonium Research at Los Alamos Once NNSA resumes the CMRR project and constructs the facility, Conclusions CMRR will play an important role in ensuring the continued safety and reliability of the U.S. nuclear weapons stockpile. The CMRR can potentially offer NNSA the opportunity to improve efficiency, save costs, and reduce safety hazards for workers. Because of the facility’s importance to the stockpile, multibillion dollar price tag, the inherent challenges in building facilities that can safely and securely store plutonium, and NNSA’s ongoing difficulties managing large projects, it is critical that NNSA and Congress have accurate estimates of the project’s costs and schedules, particularly when the CMRR project is resumed. After facing a nearly six-fold increase in estimated cost and schedule delays, NNSA’s most recent cost and schedule estimates generally meet industry best practices, but there are important weaknesses that call these estimates’ reliability into question. For example, an independent cost estimate—the most rigorous method to validate major cost elements that is performed by an entity that has no stake in the approval of the project—has not yet been conducted. To its credit, NNSA plans to have an independent cost estimate conducted prior to the completion of CMRR’s project baseline once the project is resumed. With regard to the project’s schedule estimate, however, NNSA cannot yet provide high assurance that all project risks are fully accounted for in the project’s schedule risk analysis that is used for updating the project’s schedule contingency estimates. As a result, NNSA cannot yet be fully confident that, once it decides to resume the CMRR project, the project will meet its estimated completion date, which could lead to further delays and additional costs. However, reliable cost and schedule estimates for CMRR that fully meet industry best practices are of little use if DOE’s and NNSA’s mission needs are not met. Constructing CMRR is an important part of NNSA’s strategy to modernize its nuclear weapons facilities into a smaller and more responsive, efficient, and secure infrastructure to meet the changing requirements of the nuclear weapons stockpile. The CMRR was intended to support the plutonium-related research and storage needs of other DOE and NNSA national security missions and activities outside of the nuclear weapons stockpile work, including homeland security and nuclear nonproliferation activities; but because NNSA decided early in the project to reduce the size of the proposed facility to save money, CMRR may now lack the ability to accommodate these other research needs. In particular, the planned removal of most plutonium from Livermore presents NNSA with a dilemma in that the primary benefit of consolidating plutonium at Los Alamos—lower security costs—may be offset by the need to replace Lawrence Livermore National Laboratory’s plutonium Page 22 GAO-12-337 Plutonium Research at Los Alamos research, storage, and environmental testing capabilities. Importantly, when NNSA decided to consolidate plutonium operations at Los Alamos, it did not fully consider whether planned or existing facilities at Los Alamos would be capable of continuing plutonium work being conducted elsewhere. For example, CMRR was not intended to accommodate the thermal, vibration, and other environmental pit testing that Livermore currently conducts because the vibrations this type of testing creates could disrupt other work at CMRR that requires precision instrumentation. Nevertheless, this type of testing is necessary to meet nuclear weapons stockpile requirements and so must be conducted somewhere. The full extent of the potential shortfall in plutonium research capabilities is not well-understood because DOE and NNSA have not comprehensively assessed their plutonium-related research, storage, and environmental testing needs. Plutonium research for the nuclear weapons stockpile and for other missions may have to compete for limited laboratory and storage space in CMRR and other facilities at Los Alamos, especially if the demand for newly manufactured pits unexpectedly increases. As a result, expansion of CMRR or construction of costly additional plutonium research, storage, and testing facilities at Los Alamos or elsewhere may be needed sometime in the future. To strengthen cost and schedule estimates for the CMRR and ensure Recommendations for needed plutonium research needs are sufficiently accommodated, we Executive Action recommend that the Secretary of Energy take the following three actions: 1. Once NNSA resumes the CMRR project and prior to establishing a new cost and schedule baseline, incorporate all key risks identified by CMRR project officials into the project’s schedule risk analysis, and ensure that this information is then used to update schedule contingency estimates, as appropriate. 2. Conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs for nuclear weapons stockpile activities as well as other missions currently conducted at other NNSA and DOE facilities, with particular emphasis on mitigating the consequences associated with eliminating plutonium research, storage, and environmental testing capabilities from NNSA’s Lawrence Livermore National Laboratory. 3. Using the results of this assessment, report to Congress detailing any modifications to existing or planned facilities or any new facilities that will be needed to support plutonium-related research, storage, and Page 23 GAO-12-337 Plutonium Research at Los Alamos environmental testing needs for nuclear weapons stockpile activities as well as other missions conducted by NNSA and DOE. We provided NNSA with a draft of this report for its review and comment. Agency Comments In its written comments, reproduced in appendix IV, NNSA generally and Our Evaluation agreed with our recommendations to conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs and to report to Congress on any modifications to existing or planned facilities or any new facilities that will be needed to support these needs. However, NNSA disagreed with our recommendation to incorporate all key risks identified by project officials into the project’s schedule risk analysis. Specifically, NNSA stated that, subsequent to receiving our draft report for its comments, the President’s budget request for fiscal year 2013 was released and resulted in several changes to the funding and execution of the CMRR project. In particular, construction of the CMRR is now to be deferred for at least 5 years. Therefore, NNSA stated that it is conducting additional analysis to determine the most effective way to provide analytical chemistry, materials characterization, and storage capabilities that were originally intended for the CMRR through the use of existing infrastructure. As part of this analysis, NNSA stated that it will evaluate options to use existing facilities at other sites. We believe this is consistent with our recommendation that NNSA conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs and that NNSA’s decision to defer construction of the CMRR will give it sufficient time to conduct this assessment. NNSA also commented that it will continue to work with Congress and other stakeholders as it adjusts its plutonium strategy. In our view, this is also consistent with our recommendation to report to Congress on any modifications to existing or planned facilities or any new facilities that will be needed to support plutonium-related research, storage, and environmental testing needs for nuclear weapons stockpile activities as well as other missions conducted by NNSA and DOE. With regard to our recommendation to incorporate all key risks identified by CMRR project officials into the project’s schedule risk analysis, NNSA commented that spending project money to update the CMRR project’s schedule would not be prudent because of the construction delay. Therefore, NNSA disagreed with the recommendation. NNSA stated that its efforts in the near term would be focused on closing out the current Page 24 GAO-12-337 Plutonium Research at Los Alamos design and that any future efforts will require updated cost and schedule estimates. We agree with NNSA that it is not necessary to update the project’s schedule at this time because of the recently announced construction delay; however, we maintain that it is important that all project risks are fully accounted for in the CMRR’s schedule once the project is resumed. Therefore, we clarified our recommendation to specify that NNSA should take action to ensure that the CMRR’s schedule risk analysis is appropriately revised to account for all project risks when NNSA resumes the project and before it establishes a new cost and schedule baseline. We are sending copies of this report to the Secretary of Energy; the Administrator of NNSA; the Director, Office of Management and Budget; the appropriate congressional committees; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staffs have any questions about this report, please contact me at (202) 512-3841 or firstname.lastname@example.org. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. Gene Aloise Director, Natural Resources and Environment Page 25 GAO-12-337 Plutonium Research at Los Alamos Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology Methodology Our objectives were to examine (1) changes in the cost and schedule estimates for the construction of the facility and the extent to which its most recent estimates reflect best practices, (2) options the National Nuclear Security Administration (NNSA) considered to ensure that plutonium-related research activities could continue as needed, and (3) the extent to which NNSA’s plans to construct the Chemistry and Metallurgy Research Replacement Nuclear Facility (CMRR) and its consideration of options reflected changes in nuclear weapons stockpile requirements and other plutonium-related research needs. To examine the project’s cost and schedule estimates and the extent to which its current estimates reflect best practices, we reviewed relevant NNSA documents and met with agency and contractor officials on the changes that have occurred to date and the reasons for them. We compared NNSA’s most recent detailed cost and schedule estimates with industry best practices contained in our cost estimating and assessment guide and discussed them with project officials to give them the opportunity to provide feedback on our assessment. Our review examined specifically those NNSA cost estimates that were prepared in April 2010 and schedule estimates, which at the time of our review were updated as of May 2011 or more recent for some portions of the schedule. As such, the cost and schedule estimates we reviewed do not reflect NNSA’s 5- year construction deferral recently announced in February 2012 and NNSA has not yet determined the potential long-term cost impact of this delay. To examine the options NNSA considered to continue plutonium-related analytical work, we reviewed NNSA and contractor documents on plutonium research needs and the various options available to meet those needs. We also met with NNSA and contractor officials to better understand how these options were analyzed to determine the best approach to fulfill NNSA’s mission. While NNSA evaluated options on how to best meet its mission needs, it may have also evaluated alternatives based on the environmental impact of building the CMRR. As such, our review examined the options NNSA assessed to maintain the capabilities for plutonium-related analytical chemistry, material characterization, and storage and did not address NNSA’s compliance with requirements of the National Environmental Policy Act. We also met with NNSA and contractor officials to gain a better understanding of how these options were analyzed to determine the best approach to fulfill NNSA’s mission. Page 26 GAO-12-337 Plutonium Research at Los Alamos Appendix I: Objectives, Scope, and Methodology To determine the extent to which NNSA’s plans reflect changes in nuclear weapons stockpile requirements, we reviewed NNSA analyses that were used to support CMRR project decisions and met with NNSA officials to determine if these analyses were comprehensive and reflected up-to-date nuclear weapons stockpile requirements. We also visited Los Alamos and Lawrence Livermore National Laboratories. To ensure the data we used were sufficiently reliable, we compared information gathered from a variety of data sources. For example, we interviewed officials from both Los Alamos and Lawrence Livermore National Laboratories to obtain separate and independent perspectives on CMRR project plans. We determined the data were sufficiently reliable for our purposes. We conducted this performance audit from February 2011 through February 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Page 27 GAO-12-337 Plutonium Research at Los Alamos Appendix II: Summary Assessment of Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices CMRR’s Cost Estimate Compared to Industry Best Practices Best practice Overall a a characteristic assessment Detailed best practice Detailed assessment Comprehensive Substantially met The cost estimate includes all life Substantially met. The total project cost for the cycle costs. construction of the Nuclear Facility is $4.2 billion. Government and contractor costs are included. However, operations and retirement costs are not included. These costs were not included because there was no mandate to estimate them. The cost estimate spans from start of construction in June 2010 to completion in 2020 with a schedule contingency through 2022. The cost estimate completely defines Fully met. Technical descriptions were provided in the program, reflects the current multiple documents such as the “CMRR Nuclear schedule, and is technically Facility (NF) Estimate at Complete Forecast–April reasonable. 2010,” the Los Alamos CMRR Mission Need Statement, the Program Requirements Documents, the WBS dictionary, and the “Final Environmental Impact Statement for the Chemistry and Metallurgy Project.” The cost estimate work breakdown Partially met. The work breakdown structure and structure is product-oriented, work breakdown structure dictionary are product traceable to the statement of oriented and the work breakdown structure flows work/objective, and at an appropriate down to level 4 of the program, project, or task. A level of detail to ensure that cost statement of work was provided in the form of a elements are neither omitted nor mission need statement; however, it is not easily double-counted. reconciled with the work breakdown structure dictionary. The estimate documents all cost- Fully met. Cost influencing ground rules and influencing ground rules and assumptions can be found in the CMRR Estimate assumptions. Update Execution Plan. Budget constraints and escalation are addressed. A list of high-level risk drivers along with the handling costs and risk input information was provided. Exclusions to the cost estimate are noted in the documents. Well documented Substantially met The documentation captures the Partially met. The data was analyzed and high-level source data used, the reliability of the cost drivers have been addressed as well as unit data, and how the data were rates and quantities. Source data used to develop normalized. the estimate were found. The cost estimate was based on historical data from other Department of Energy (DOE) sites and the data was normalized. However, the independent review team found inconsistencies and discrepancies of quantities (hours) and costs. In addition, the review team reported that even though the basis of estimate referred to current contract awards or proposals, no reference was made to specific contracts or proposals by date and number. Page 28 GAO-12-337 Plutonium Research at Los Alamos Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices Best practice Overall a a characteristic assessment Detailed best practice Detailed assessment The documentation describes in Substantially met. While not explicitly stating what sufficient detail the calculations methodology was used, the pricing approach performed and the estimating summary indicates that the estimate was developed methodology used to derive each using a combination of the build-up method and element’s cost. extrapolation from pricing information and productivity rates from other DOE sites. However the calculations involved were not clearly shown. The documentation describes, step by Substantially met. The documentation for the step, how the estimate was developed estimate contains a summary narrative about the so that a cost analyst unfamiliar with project as well as high-level cost summaries. The the program could understand what documentation discusses risk and contingency was done and replicate it. reserve. However, it does not address sensitivity although a sensitivity analysis was performed. Narrative on how the sensitivity analysis was conducted was not provided. The documentation discusses the Substantially met. There are technical descriptions technical baseline description and the discussed in the documentation that are consistent data in the baseline is consistent with with the basis of estimate and the work outlined in the estimate. the detail cost estimate spreadsheets. However, we are unable to map specific technical descriptions as outlined in the requirements document to cost elements in the high-level or detailed cost estimates. During the site visit, project officials showed us how the scope of work in the work breakdown structure dictionary was written in a way to illustrate how the scope of work was captured. The documentation provides evidence Partially met. Los Alamos policy states that reviews that the cost estimate was reviewed shall be performed. According to project officials, and accepted by management. these reviews typically include an integrated project team review, functional manager review, directorate review, and in the case of projects of high complexity or risk, an external corporate review and/or DOE Los Alamos Site Office review. A CMRR functional review was held March 12, 2010, and the review of the current estimate was listed on the meeting agenda. However, without further documentation we are unable to determine whether or not a briefing was given to management that clearly explains the detail of the cost estimate— including presentation of lifecycle costs, ground rules and assumptions, estimating methods and data sources as they relate to each work breakdown structure element, results of sensitivity analysis, risk and uncertainty analysis, and if a desired level of confidence was reached. Additionally, it is not clear that an affordability analysis, contingency reserve, conclusions, or recommendations were discussed with management. The documentation also does not show management’s acceptance of the cost estimate. Page 29 GAO-12-337 Plutonium Research at Los Alamos Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices Best practice Overall a a characteristic assessment Detailed best practice Detailed assessment Accurate Substantially met The cost estimate results are Substantially met. Risk and uncertainty analyses unbiased, not overly conservative or were performed providing an 84 percent confidence optimistic, and based on an level. There are three components that contribute to assessment of most likely costs. the total contingency value established for the project—schedule, estimate, and technical and programmatic risk analysis. The estimate has been adjusted Substantially met. The documentation contained properly for inflation. information on escalation rates. However, it is unclear how the cost estimate data were normalized. For example, costs are listed but are not labeled as constant or then-year dollars. Detailed calculations on how escalation was applied to the cost estimate are not documented. The estimate contains few, if any, Substantially met. The numbers shown in the minor mistakes. estimate at complete document and the cost estimate spreadsheet are accurate and the independent review team found only one minor mistake in their review of the estimate. However, we were not provided access to the detailed calculations behind the spreadsheet to check that the estimate was calculated correctly. The cost estimate is regularly updated Substantially met. The CMRR Project Control Plan to reflect significant changes in the outlines a formal change control process that is to program so that it always reflects be executed in accordance with the Los Alamos current status. Project Management and Site Services Directorate as well as the CMRR Baseline Change Control Board. These documents provide an approach to document, communicate, and approve potential changes to scope, cost, and schedule, and they provide the basis for incorporating changes into the project baseline and/or the forecast estimate at completion. These documents also describe the activities and responsibilities for making changes to the baseline. Any variances between planned and Substantially met. Earned value is entered for each actual costs are documented, work package based on the earned value method explained, and reviewed. indicated for that work package. Progress is reported in terms of percent complete by work package and is verified, analyzed, and reported to the project controls team. This information is then analyzed by the project controls team and control account managers and reviewed with CMRR management as the final reports are completed and published. However, there is no evidence of the cost estimate being updated to capture variances from the earned value system. Page 30 GAO-12-337 Plutonium Research at Los Alamos Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices Best practice Overall a a characteristic assessment Detailed best practice Detailed assessment The estimate is based on a historical Substantially met. Part of the estimate was record of cost estimating and actual developed using the engineering build up method experiences from other comparable which includes historical data from other programs. DOE/NNSA sites (Waste Treatment Plant, Mixed Oxide Fuel Fabrication Facility, and two chemical demilitarization facilities). The reliability of the data is documented where confidence levels associated with quantity, productivity, labor, and nonlabor pricing are addressed. However, for some of the data, the sources were not provided and there was no evidence that earned value data was used to develop or update the estimate. Credible Partially met The cost estimate includes a Substantially met. CMRR conducted some sort of sensitivity analysis—a technique that sensitivity analysis. No documentation was given identifies a range of possible costs providing a narrative on how the sensitivity analysis based on varying major assumptions, was conducted—including whether high parameters, and data inputs. percentages of cost were determined and how their parameters and assumptions were examined. Additionally, it cannot be determined whether the outcomes were evaluated for parameters most sensitive to change or how this analysis was applied to the estimate. However, during a site visit, Los Alamos officials provided a copy of a report that shows how a sensitivity analysis was applied to the nuclear facility cost estimate. For this assessment, a high and low range was determined. Some of the factors that were varied included overhead and General and Administrative rates, and escalation. Page 31 GAO-12-337 Plutonium Research at Los Alamos Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices Best practice Overall a a characteristic assessment Detailed best practice Detailed assessment A risk and uncertainty analysis was Substantially met. The cost estimate includes conducted that quantified the contingency costs for schedule ($99 million), cost imperfectly understood risks and estimate ($508 million) and technical and identified the effects of changing key programmatic risks ($404 million). While a schedule cost driver assumptions and factors. risk analysis was performed that identified $99 million in schedule contingency, it is not clear how this analysis was done as no supporting documentation was provided. An independent review team assessed the schedule risk analysis and found that the risk model did not contain enough detail to allow specific risk events to be associated with the schedule activities they affect. Documentation supporting the cost estimate ($508 million) risk and uncertainty analysis was conducted via a Monte Carlo simulation which established an 84 percent confidence level for cost estimate uncertainty. The process by which this analysis was done is well documented and includes the contingency level range results. However, this risk and uncertainty analysis only reviewed classic cost estimate contingency and did not assess technical, programmatic or schedule risks. In addition, the independent review team found that the cost risk uncertainty analysis was done at a summary level so it does not fully reflect the uncertainty of the design costs associated with uncertainty related to quantities or prices listed. Major cost elements were crossed Partially met. Documentation was provided that checked to see whether results were shows comparison of selected CMRR cost similar. elements against cost estimates of other sites. An independent cost estimate was Partially met. An independent cost estimate was not conducted by a group outside the conducted by a group outside of the acquiring acquiring organization to determine organization. However, an independent cost review whether other estimating methods was performed by the U.S. Army Corps of produce similar results. Engineers in conjunction with an experienced contractor. This independent cost review resulted in the identification of key findings which require a Corrective Action Plan. The independent cost review focused on engineering design, and nuclear facility special facility equipment engineering design. The independent cost review team had 24 key findings and recommendations. Source: GAO analysis of CMRR project cost information. a The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no evidence that satisfies any of the practice. “Minimally met” means the CMRR provided evidence that satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that completely satisfies the practice. Page 32 GAO-12-337 Plutonium Research at Los Alamos Appendix III: Summary Assessment of Appendix III: Summary Assessment of CMRR’s Schedule Estimate Compared to Industry Best Practices CMRR’s Schedule Estimate Compared to Industry Best Practices a Best practice Explanation Detailed assessment Capturing all The schedule should reflect all activities as defined Fully met. The schedule integrates all of the effort of activities in the program’s work breakdown structure, to NNSA, its contractor, and its major subcontractors. include activities to be performed by both the government and its contractors. Sequencing all The schedule should be planned so that it can meet Substantially met. While we found that about 16 percent activities critical program dates. To meet this objective, key of the activities were missing predecessors and activities need to be logically sequenced in the successors, or had constraints, lags, and leads, the order that they are to be carried out. In particular, majority (84 percent) of the activities were logically activities that must finish before the start of other sequenced. There are more than 2,400 activities (5 activities (i.e., predecessor activities) as well as percent) with missing or dangling predecessors or activities that cannot begin until other activities are successors. There are summary tasks linked with logic (3 completed (i.e., successor activities) should be percent), but we have determined that they do not affect identified. By doing so, interdependencies among the credibility of the schedule. There are 123 activities activities that collectively lead to the (less than 1 percent) with start-to-finish logic. There are accomplishment of events or milestones can be 460 activities (less than 1 percent) that have 10 established and used as a basis for guiding work predecessors or more. There are 590 activities (1 and measuring progress. percent) scheduled with constraints, in addition to or substituting for complete logic. Assigning resources The schedule should reflect what resources (i.e., Substantially met. Not all activities in the project schedule to all activities labor, material, and overhead) are needed to do the are resource loaded—only 3,757 activities (8 percent) out work, whether all required resources will be of the 45,429 activities with positive remaining duration available when they are needed, and whether any have resources assigned in the schedule we received. funding or time constraints exist. However, there is credible evidence that the program and Los Alamos manage resources in various ways outside the project schedule and that their resource solutions are fed back to the project schedule so that it is feasible given resource limits. Establishing the The schedule should realistically reflect how long Substantially met. There are 1,642 activities (4 percent) duration of all each activity will take to execute. In determining the with durations 44 days or greater, which means that the activities duration of each activity, the same rationale, data, majority of the activities (96 percent) have activities that and assumptions used for cost estimating should are of short duration. Contributing to this is the rolling be used. Further, these durations should be as wave approach to the schedule, where the near-term short as possible and they should have specific activities are detailed while activities further in the future start and end dates. Excessively long periods are left in large planning packages until they become needed to execute an activity should prompt further near-term, at which point they are broken down into their decomposition of the activity so that shorter component activities. execution durations will result. Integrating schedule The schedule should be horizontally integrated, Substantially met. As discussed previously in the activities horizontally meaning that it should link the products and “sequencing all activities,” there are activities missing outcomes associated with already sequenced predecessor and successor logic as well as the presence activities. These links are commonly referred to as of constraints, lags, and leads that call into question the handoffs and serve to verify that activities are adequacy of horizontal traceability. Vertical traceability arranged in the right order to achieve aggregated was confirmed. The schedule hierarchy includes five products or outcomes. The schedule should also be levels, increasing in detail and specificity from top to vertically integrated, meaning that traceability exists bottom. among varying levels of activities and supporting tasks and subtasks. Such mapping or alignment among levels enables different groups to work to the same master schedule. Page 33 GAO-12-337 Plutonium Research at Los Alamos Appendix III: Summary Assessment of CMRR’s Schedule Estimate Compared to Industry Best Practices a Best practice Explanation Detailed assessment Establishing the Using scheduling software, the critical path—the Substantially met. This schedule’s critical path has 5,479 critical path for all longest duration path through the sequenced list of activities with zero or negative total float. There are so activities key activities—should be identified. The many critical activities because of a number of establishment of a program’s critical path is constraints on intermediate milestones which is causing necessary for examining the effects of any activity negative float on paths to those activities. However, these slipping along this path. Potential problems that activities do not all drive the final delivery. Los Alamos may occur on or near the critical path should also officials said that when they baseline the schedule, they be identified and reflected in the scheduling of the plan to remove many of the constraints that are causing time for high-risk activities. negative float. Many of these constraints are there to enable Los Alamos to monitor status of intermediate milestones. Identifying float The schedule should identify float so that schedule Substantially met. Of the remaining activities, 22 percent between activities flexibility can be determined. As a general rule, have unexplained large positive and large negative total activities along the critical path typically have the float values. Even with agency review, these were least amount of float. present in the schedule. The total float values in many cases are several years long. There are 4,611 activities (10 percent) that have total float over 1,000 days or about 3.8 years. These high total float values are likely related to the incomplete logic described in the “sequencing all activities” best practice. Conducting a A schedule risk analysis should be performed using Minimally met. There is no evidence that a risk analysis schedule risk a schedule built using a good critical path method has been conducted on this schedule or any summary analysis and data about project schedule risks, as well as schedule derived from this schedule. Los Alamos officials statistical analysis techniques (such as Monte said that they have conducted a risk analysis using Carlo) to predict the level of confidence in meeting Monte Carlo simulation based on a prior and more a program’s completion date. This analysis focuses concise schedule a full year before the version we not only on critical path activities but also on reviewed was developed. The version we reviewed activities near the critical path, since they can contained 90,000 activities and was developed in the potentially affect program status. Spring of 2010—a full year after Los Alamos conducted its risk analysis and Monte Carlo simulation. Los Alamos did not conduct a risk analysis on this more recent schedule, nor did it prepare and simulate a summary schedule based on this more recent schedule. The summary schedule that Los Alamos simulated was based on critical and near critical paths. This schedule comprised the main, secondary and tertiary critical paths. As a result, we believe that the schedule did not cover the entire work of the project, and therefore may have excluded some activities or paths that have risk sufficient to affect the finish date. Instead, Los Alamos selected about 2,100 activities based on total float, but this practice is risky because they may not have included all of the activities that risks in the risk register may affect. Page 34 GAO-12-337 Plutonium Research at Los Alamos Appendix III: Summary Assessment of CMRR’s Schedule Estimate Compared to Industry Best Practices a Best practice Explanation Detailed assessment Using logic and The schedule should use logic and durations in Fully met. The CMRR schedule is updated at least durations to order to reflect realistic start and completion dates monthly, although much of it is updated weekly. The determine the start for program activities. The schedule should be schedule integrity is checked after each update and and completion dates continually monitored to determine when forecasted metrics are compiled on problems to determine if the completion dates differ from the planned dates, schedule’s integrity is improving with each update. which can be used to determine whether schedule There are no activities in the past that lack the variances will affect downstream work. Maintaining designation of actual start or actual finish. There are the integrity of the schedule logic is not only some activities on or after the data date that have actual necessary to reflect true status, but is also required start or finish designations, but that may be because before conducting a schedule risk analysis. there are 15 schedules combined in the Integrated Master Schedule and some were updated somewhat after May 9, 2011. Source: GAO analysis of CMRR project schedule information. a The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no evidence that satisfies any part of the practice. “Minimally met” means the CMRR provided evidence that satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that completely satisfies the practice. Page 35 GAO-12-337 Plutonium Research at Los Alamos Appendix IV: Comments from the National Appendix IV: Comments from the National Nuclear Security Administration Nuclear Security Administration Page 36 GAO-12-337 Plutonium Research at Los Alamos Appendix IV: Comments from the National Nuclear Security Administration Page 37 GAO-12-337 Plutonium Research at Los Alamos Appendix IV: Comments from the National Nuclear Security Administration Page 38 GAO-12-337 Plutonium Research at Los Alamos Appendix V: GAO Contact and Staff Appendix V: GAO Contact and Staff Acknowledgments Acknowledgments Gene Aloise, (202) 512-3841 or email@example.com GAO Contact In addition to the contact named above, Ryan T. Coles, Assistant Acknowledgments Director; John Bauckman; Jennifer Echard; Eugene Gray; David T. Hulett; Jonathan Kucskar; Alison O’Neill; Christopher Pacheco; Tim Persons; Karen Richey; Stacey Steele; Vasiliki Theodoropoulos; and Mary Welch made key contributions to this report. (361267) Page 39 GAO-12-337 Plutonium Research at Los Alamos GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no Obtaining Copies of cost is through GAO’s website (www.gao.gov). Each weekday afternoon, GAO Reports and GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, Testimony go to www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. Contact: To Report Fraud, Waste, and Abuse in Website: www.gao.gov/fraudnet/fraudnet.htm E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Katherine Siggerud, Managing Director, email@example.com, (202) 512- Congressional 4400, U.S. Government Accountability Office, 441 G Street NW, Room Relations 7125, Washington, DC 20548 Chuck Young, Managing Director, firstname.lastname@example.org, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Please Print on Recycled Paper.
Modernizing the Nuclear Security Enterprise: New Plutonium Research Facility at Los Alamos May Not Meet All Mission Needs
Published by the Government Accountability Office on 2012-03-26.
Below is a raw (and likely hideous) rendition of the original report. (PDF)