United States General Accounting Office GAO Testimony Before the Subcommittee on Basic Research, Committee on Science, House of Representatives To Be Released on Delivery Expected at DEPARTMENT OF 2 p.m. EDT Wednesday ENERGY July 14, 1999 Problems in the Management and Use of Supercomputers Statement For the Record by Susan D. Kladiva, Associate Director, Energy, Resources, and Science Issues, Resources, Community, and Economic Development Division GAO/T-RCED-99-257 Mr. Chairman and Members of the Subcommittee: We appreciate the opportunity to provide a statement for the record on the Department of Energy’s (DOE) use, acquisition, and management of its supercomputing resources. The Department spent about $826 million between 1994 and 1997 acquiring and operating supercomputers. These computers support a variety of uses, and DOE’s largest supercomputer effort—the Accelerated Strategic Computing Initiative—is a critical part of DOE’s attempt to build the capability for “virtual testing” as a replacement for the actual testing of nuclear weapons. DOE estimates that it will spend about $5.2 billion on the strategic computing initiative between fiscal years 1996 and 2004. Our testimony covers how effectively DOE has managed the use and acquisition of its supercomputers, and DOE’s management of its strategic computing initiative. Our testimony is based on two reports we issued in 1998 and 1999.1 In summary, we found that DOE had not effectively overseen the use and acquisition of supercomputers and that oversight of the $5.2 billion strategic computing initiative is hampered by weaknesses in management and information processes. In July 1998, we reported that DOE’s national laboratories used only about 59 percent of their available supercomputing capacity and were missing opportunities to share these resources. At that time, DOE had about 17 percent of the world’s supercomputing capacity and was planning to almost triple its capacity over the next 3 years. In terms of managing acquisitions of supercomputers, we concluded that DOE has not effectively overseen the process. Furthermore, the strategic computing initiative’s strategic plan is out of date, annual plans have been prepared only sporadically, and milestones are not well defined. Currently, little information exists to track the program’s progress or to compare the program’s accomplishments with its milestones. Consequently, it is difficult to determine which of the hundreds of milestones have been met, which are behind schedule, or even which are still relevant, given changes in the program. In July 1998, we reported that nine DOE laboratories have supercomputers. Background DOE’s program offices fund supercomputer purchases (or leases), and the laboratories’ management and operating contractors acquire and operate the systems. According to DOE, it acquires new supercomputers relatively frequently because of rapidly changing technology. Since 1993, when 1 Information Technology: Department of Energy Does Not Effectively Manage Its Supercomputers (GAO/RCED-98-208, July 17, 1998) and Nuclear Weapons: DOE Needs to Improve Oversight of the $5 Billion Strategic Computing Initiative (GAO/RCED-99-195, June 28, 1999). Page 1 GAO/T-RCED-99-257 statistics were first collected, DOE has consistently had several supercomputers that have ranked among the world’s most powerful as measured by a list of the top 500 supercomputers in the world. DOE uses supercomputers to support two major research missions. First, DOE uses them in the strategic computing initiative to ensure the safety and reliability of nuclear weapons. The initiative is part of DOE’s Office of Defense Programs’ nuclear stockpile stewardship program. DOE created the initiative as a substitute for the physical testing of nuclear weapons. The initiative is intended to provide the unprecedented simulation capabilities needed to help verify the safety and reliability of U.S. nuclear weapons without nuclear testing. The estimated cost of the initiative is about $5.2 billion for fiscal years 1996 through 2004. Second, DOE’s Office of Science funds non-defense computational research projects, including specific science and engineering problems that require large-scale supercomputing capability. In 1996, the Congress passed the Clinger-Cohen Act, which requires federal agencies to adopt a comprehensive approach to acquiring and managing information technology (including supercomputers), and charged the Office of Management and Budget with oversight responsibility. We found that DOE had not effectively overseen the use and acquisition of DOE Does Not supercomputers. During 1997, DOE’s national laboratories used only about Effectively Manage Its 59 percent of their available supercomputing capacity and were missing Supercomputers opportunities to share these resources. Utilization rates varied among the laboratories from about 31 percent to about 75 percent. At that time, DOE had about 17 percent of the world’s supercomputing capacity and was planning to almost triple its capacity during fiscal years 1998 through 2000. Sharing of supercomputers among DOE laboratories and with DOE-funded off-site users was not generally considered by the Department as a way to better use existing resources and/or to forgo the need to acquire more supercomputers. In addition, the largest supercomputers were not being used to run the very large-scale programs that were used to justify their acquisition. In 1997, for example, less than 5 percent of the jobs run on the largest supercomputers used more than one-half of the machines’ capabilities. Because DOE does not manage supercomputers centrally, no single person or office within the Department knows at a given time how many Page 2 GAO/T-RCED-99-257 supercomputers the Department has agency-wide, what they cost, or how they are being utilized. The Department lacks an investment strategy and a defined process to ensure that supercomputer acquisitions are fully justified and represent the best use of funds among competing priorities. Instead, individual program offices (e.g., Office of Defense Programs, Office of Science) independently procure and operate supercomputers and the Department’s chief information officer does not oversee the acquisition of these supercomputers. As a result, new supercomputers are planned and acquired with little departmental oversight, while underutilized capacity already exists within the Department. We also reported that DOE’s proposed implementation of the Clinger-Cohen Act would not improve departmental oversight. In April 1998, the Department outlined its plan to implement a new investment planning and oversight process for information technology in response to the Clinger-Cohen Act. DOE’s process separately manages administrative and scientific computers, leaving the responsibility for scientific computers—including supercomputers—to individual program offices. This proposed approach reflects the view of the Department’s program offices that supercomputers are research “tools” rather than information technology investments. This approach may also allow DOE’s program offices to continue acquiring supercomputers outside the Department’s normal process for complying with the Clinger-Cohen Act. Contrary to what is envisioned in the Clinger-Cohen Act, this approach effectively places the vast majority of DOE’s information technology resources outside the purview of the chief information officer. In June 1999, we reported that significant weaknesses in DOE’s DOE Needs to management and information processes hamper the oversight of the Improve Oversight of strategic computing initiative. Although program managers report that the Strategic many milestones have been met, the lack of a comprehensive planning and progress tracking system makes assessment of the program’s progress Computing Initiative difficult and subjective. Currently, the program’s strategic plan is out of date, annual plans have been prepared only sporadically, and milestones are not well defined. Furthermore, little information exists to track the program’s progress or to compare the program’s accomplishments with its milestones. Consequently, it is difficult to determine which of the hundreds of milestones have been met, which are behind schedule, or even which are still relevant, given changes in the program. Page 3 GAO/T-RCED-99-257 Program cost estimates have also increased substantially. In 1995, DOE estimated that costs for the first 5 years of the program (fiscal years 1996 through 2001) would be $1.7 billion. By 1999, estimated costs for that same 5-year period increased to $2.9 billion. DOE currently estimates that the program will cost about $5.2 billion for fiscal years 1996 through 2004. Some of the cost increases result from the shift to computer-based simulations, while some reflect weaknesses in DOE’s cost estimation. For future contacts regarding this testimony, please contact Susan D. Contact and Kladiva at (202) 512-3841. Individuals making contributions to this Acknowledgment testimony included Linda Chu, Daniel Feehan, Anne McCaffrey, and Edward Zadjura. (141364) Page 4 GAO/T-RCED-99-257 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Department of Energy: Problems in the Management and Use of Supercomputers
Published by the Government Accountability Office on 1999-07-14.
Below is a raw (and likely hideous) rendition of the original report. (PDF)