National Security and International Affairs Division B-220298 January 31,199O The Honorable Edward Kennedy Chairman, Subcommittee on Projection Forces and Regional Defense Committee on Armed Services United States Senate Dear Mr. Chairman: In response to your November 2, 1987, request, we assessed the ability of the Navy’s new advanced combat system programs-AN/BSY-1 and AN/BSY-2-to meet cost, schedule, and performance goals. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 7 days from the date of this letter. At that time, we will send copies to the Chairmen, Senate Committee on Government Affairs, House Committee on Government Operations, Senate and House Committees on Armed Services, and Senate and House Committees on Appropriations; the Director, Office of Management and Budget; and the Secretaries of Defense and the Navy. We also will send copies to interested parties and make copies available to others upon request. This report was prepared under the direction of Martin M. Ferber, Director, Navy Issues, who may be reached on 275-6504 if you or your staff have any questions. Other major contributors are listed in appendix III. Sincerely yours, Frank C. Conahan Assistant Comptroller General Executive Summary To meet new Soviet threats and ensure continued U.S. submarine Purpose superiority, the U.S. Navy has initiated development of two new advanced combat systems. These systems-the AN/BSY-1 and the AN/ BSY-2-are to be installed in improved Los Angeles (~~~-688) and new Seawolf (SSN-8 1) class nuclear attack submarines, respectively. The life- cycle costs for the two systems have been estimated at over $26 billion. The Chairman, Subcommittee on Projection Forces and Regional Defense, Senate Committee on Armed Services, requested that GAO examine the status of the Navy’s submarine combat system develop- ment programs. Specifically, GAO determined whether these two combat systems will meet cost, schedule, and performance goals and whether the combat system being developed for ~~~-21s can avoid developmental problems experienced with the ~~~-688 combat system. In 1980 the Navy began developing an advanced combat system for Background improved ~~~-688s authorized in fiscal year 1989 and beyond. Origi- nally, it planned a single-phased program. However, in October 1983 the Secretary of Defense accelerated the program and approved a three- phased plan to apply to ~~~-688s authorized, starting in fiscal year 1983-a B-year acceleration. Because of ambitious program objectives and schedules, cost, schedule, and technical problems surfaced causing the Navy to restructure the program into two separate development efforts-AN/BSY-1 for use on improved ~~~-688s and AN/BSY-2 for SSN-21s. These combat systems are designed to improve data processing and management capabilities. With the use of new and more capable computers, new data displays, and additional software, certain tasks, such as searching for, detecting, and tracking targets, will be more automated. System operators can thus perform multiple tasks, address multiple targets concurrently, and proc- ess tactical data faster and more accurately than they can with prior systems. Collectively, these capabilities are designed to reduce the response time between initially detecting a target and launching a weapon. The Navy’s submarine combat system development programs are Results in Brief experiencing problems. AN/BSY-1 program problems raise questions as to when the improved ~~~-688s will be fully mission capable. Because of continued ambitious development objectives and schedules for the com- bat system development program, the Navy allowed insufficient time in Page2 GAO/NSIAD-90-72SubmarineCombatSystems l . 1 Executive Summary the development schedule to resolve technical problems. As a result, the AN/BSY-1 systems will provide the ~~~-688s improved performance capabilities in the acoustics and weapons launch areas, but the systems will be less capable in other areas. Also, the capabilities will be delivered later and cost more than originally planned under the earlier program. The Navy has taken steps to reduce risks in the AN/BSY-2 program. However, it appears that potential problems in the AN/BSY-2 are similar to those experienced in developing prior submarine advanced combat systems, including the AN/BX-1. In order to meet the SSN-2l’s construc- tion schedule, the Navy also has established ambitious objectives and schedules for the AN/BEY-2 development program. As a result, the first combat system will not have full capabilities when delivered to the ship- builder. In addition, combat system development problems could adversely affect the planned cost, schedule, and performance of the first SW-2 1. The Navy continues to establish ambitious program objectives and Principal Findings schedules in its development of complex submarine combat systems. As a result, the Navy must accept less than fully capable combat systems in order to meet the shipbuilders’ schedule. AN/BSY-1 Has The estimated life-cycle costs for the AN/BSY-1 have increased from $5.4 billion to $12.1 billion for 19 and 24 systems, respectively. The first Experienced Problems four systems will not have full AN/BSY-1 offensive capabilities and will be upgraded during the submarines’ post shakedown availability period. Therefore, these submarines will not be able to perform a full range of missions. In addition, the AN/BSY-1 will provide less capabilities than originally planned under the original submarine advanced combat sys- tem program. AN/BSY-1 design changes were the major cause of several improved ~~~-688s under construction being modified. These changes also resulted in one shipyard being awarded almost $82 million for changes to five submarines and another requesting a $150 million contract adjustment for modifications for nine submarines. The first nine AN/B%-l- equipped submarines will be delivered an average 17 months late to the Navy. Page 3 GAO/NSIAD-90-72Submarine Combat Systems Executive Summary AN/BSY-2 Will Not Have Like the improved ~~~-688 program, the need to meet the SSN-21ship construction schedule also is affecting the AN/BSY-2 development pro- Full Capabilities When gram. As a result, the prime contractor does not have sufficient time to Delivered deliver the first combat system with full capabilities to the Navy. Remaining capabilities are scheduled to be delivered to the Navy in November 1994. AN/BSY-2 Development As of November 1989, AN/BSY-2 development was about 3 months behind the program’s current schedule and further delays are expected. Problems Could Adversely Delays have resulted in deferring two Navy critical system design Delay - . . Further System reviews and some critical item testing. Until the Navy completes these lJ lelivery reviews, the prime contractor is unable to begin developing most hard- ware and writing most system software code. GAO is not making recommendations in this report. Recommendations The Department of Defense (DOD) generally agreed with GAO'Sreport and Agency Comments with the facts as presented on the AN/BSY-1 but only partially con- curred with GAO'Sfindings on the AN/BE%-2. In those cases where it par- tially agreed with the report, DODprovided further elaboration. (See app. II.) DODagreed that AN/BSY-1 combat system design changes were a major contributor to submarine delivery delays and cost increases but added that other design changes also contributed to the submarines’ delays and cost increases. Regarding the AN/BSY-2, DODstated that the slippage of the preliminary design review and the critical design review had no impact on the scheduled delivery of the system to the Navy. Although DODagreed that some critical item tests have been delayed, it stated that many have been satisfactorily completed. Also, DODagreed slowness in subcontract definitization increased program risks but added that no subcontractor design effort is being delayed. Page 4 GAO/NSIAD-90-72Submarine Combat Systems ;. -1 .i ;’ .’ -’ Page 5 GAO/NSIAIMO-72 Submarine Combat Systems .’ Contents Executive Summary 2 Chapter 1 8 Introduction Background 8 Objectives, Scope, and Methodology 10 Chapter 2 12 SSNISSSCombat Program Costs 12 System Capabilities Not Provided on Schedule 13 System Problems Performance 14 Affect the Readiness Conflicting Combat System and Submarine Schedules 17 Increase Risks of Some Submarines Conclusions 19 Agency Comments 19 Chapter 3 20 Early AN/BSY-2 Program Cost Program Schedule 20 21 Combat System First AN/BSY-2 Will Not Be Delivered With Full 26 Problems Could Performance Similarities Between Submarine Combat System Programs 29 Increase Costs and Conclusions 30 Delay SSN-21 Delivery Agency Comments 31 Appendixes Appendix I: Comparison of Past and Present Combat 32 System Program Features Appendix II: Comments From the Department of Defense 33 Appendix III: Major Contributors to This Report 46 Tables Table 2.1: Estimated AN/BSY-1 Life-Cycle Costs Table 2.2: Delivery Schedules for First Five Combat Systems Table 2.3: Costs and Schedule Delays Due to SUBACS 18 Basic Redesign Table 3.1: Estimated Life-Cycle Costs for AN/BSY-2 20 Systems Page 6 GAO/NSIAD-90-72Submarine Combat Systems Contents Abbreviations DOD Department of Defense OPTEVFOROperational Test and Evaluation Force PSA post shakedown availability SSN nuclear attack submarine SUBACS Submarine Advanced Combat System Page 7 GAO/NSIAD-90-72Submarine Combat Systems .’ L Chapter 1 / Introduction The nuclear attack submarine (SSN) is one of the nation’s most important antisubmarine warfare assets. To enhance the performance and ensure the continued superiority of U.S. nuclear attack submarines, the Navy will equip both the improved Los Angeles class (~~~-688) and the new Seawolf class (~~~421) nuclear attack submarines with new and improved combat systems-AN/BSY-1 and AN/BSY-2, respectively. The two computer-based combat systems are designed to detect, classify, track, and launch weapons at enemy subsurface, surface, and land targets; locate enemy targets faster than previous systems; allow opera- tors to perform multiple tasks and address multiple targets concur- rently; and reduce the time between detecting a target and launching weapons. Fully capable combat systems, which are successfully devel- oped, fully tested, and delivered on schedule, are critical to the improved ~~~-688 and the SSN-2 1 achieving full combat capability. Advanced submarine combat system development began in 1980 with Background the Submarine Advanced Combat System (SUBACS)~program. SUBACSwas started to meet the improved ~~~-688’s expanded mission and to counter the Soviet antisubmarine warfare threat through the 1990s. SUBACSwas to be a single-phase program and was to upgrade improved ~~~-688s authorized in fiscal years 1989 and beyond. In October 1983, however, the Secretary of Defense accelerated the program and approved a three- phased plan (SUBACS Basic, SUBACS A, and SUBACSB) for improved ~~~-688s authorized in fiscal years 1983 and beyond-6 years earlier than originally planned. SUBACSBasic was intended to be a new combat system to add new sonar subsystems and to process and integrate acoustic and combat control information by using a new fiber optic technology called a distributive system data bus. SUBACSA was to use new and upgraded software to integrate SUBACSBasic’s acoustic and combat control processing and was to be used in improved ~~~-688s authorized in fiscal years 1986 through 1988. SUBACSB was to introduce sonar improvements into the integrated combat system on improved ~~~-688s authorized in fiscal years 1989 and beyond. Once completed, a modified SUBACSB system was to serve as the baseline design for the SSN-21's combat system. The three-phased plan, however, was not carried to full implementation. ‘For further information on the SUBACS program, see our report entitled SUBACS Problems May Adversely Affect Navy Attack Submarine Programs (GAO/NSIAD-86-12, Nov. 4, 1985). Page 8 GAO/NSIAD-90-72Submarine Combat Systems Chapter 1 Introduction In December 1983 the Navy awarded the International Business Machines Corporation a $772 million contract for concurrent full-scale development and production of five SUBACSBasic systems and for an engineering development model. In 1985, because of program cost, schedule, and technical problems, the Navy restructured SUBACSinto two separate development efforts-AN/BSY-1 and AN/B%‘-2. Emergence of AN/BSY-1 In redesigning the SUBACSBasic effort, the Navy named the program AN/BSY-1, deleted the fiber optic distributive system data bus, and rein- troduced the heavier and bulkier copper cable and some previously developed hardware to accomplish data distribution. These actions reduced the system’s planned performance capabilities. The SUBACTS Basic contract was renegotiated in February 1986 to establish AN/BSY-1 requirements. The renegotiation also established maximum liability of the government at $1.03 billion for development and production of the first five systems. Because there was insufficient time to correct all design and development problems and meet the improved ~~~-688 con- struction schedule, two versions of the AN/BSY-1 combat system were delivered. The first version, preliminary product baseline system, was installed on four submarines (SSN-751 through SSN-754). The preliminary systems provide those submarines with limited self-defense capabilities (acous- tic, safety, and weapon firing functions) necessary to operate until the systems are upgraded to include all offensive capabilities. These four systems will be upgraded’ to full performance capabilities during each submarine’s post shakedown availability (PSA)’ and before the subma- rine’s initial operational date. A second version, product baseline sys- tem, is to have full performance capability and will be installed on the 20 remaining improved ~~~-688s starting with SSN-755. The AN/BSY-1 replaces the AN/BQQ-5 sonar and the CCS MK-1 fire con- trol systems and is designed to address shortfalls in these systems. The ‘The upgrade will include improved software, replacement of the old AN/UYK-7 computer with the new Navy standard AN/UYK-43 computer that was not available during the early design phase of SUBACS, and addition of hardware for signal processing and under ice operations. ‘An availability is an asstgnment of a shop to a repair facility for repairs or maintenance. A PSA occurs after a newly built. activated, or converted ship has completed its shakedown cruise. The heavy estimates the PSA for the SSX-751 through SSN-754 will be 9 to 10 months each. Page 9 GAO/NSIAD-90-72Submarine Combat Systems Chapter 1 Introduction Navy expects to buy 244 AN/BSY-1 combat systems and associated equipment, at a total cost of about $4.7 billion. The AN/BSY-1 is nearing completion of full-scale development. Emergence of AN/BSY-2 The Navy combined SUBACS A and SUBACS B performance requirements and renamed this effort the AN/BSY-2. This multibillion dollar combat system is currently in full-scale development and is to be installed on 29 new SSN-21s. The AN/BSY-2 will be highly automated, will use existing technology, and will include larger and more sophisticated sensors, greater data processing capabilities, and more new computer software than the AN/BSY-1. To reduce AN/B%!-2 development risks, the Navy undertook a 2-year development program to design a combat system and implemented a leader/follower arrangement. In early 1986, the Navy awarded fixed- price system design development contracts to the General Electric Com- pany and the International Business Machines. In December 1987 the Navy selected General Electric as the prime contractor and International Business Machines as the “follower” contractor. On March 31, 1988, the Navy awarded General Electric a $1.84 billion fixed-price-incentive fee contract to develop and produce three AN/B%‘-:! combat systems and related items. General Electric will develop and qualify International Business Machines as a competitor for fiscal year 1992 production systems. The Chairman, Subcommittee on Projection Forces and Regional Objectives, Scope, and Defense, Senate Committee on Armed Services, asked us to assess the Methodology status of the Navy’s AN/BSY-2 combat system development program. Specifically, we were asked to determine whether the AN/BSY-2 pro- gram would meet cost, schedule, and performance goals, including soft- ware development and system test and integration requirements. We also were asked to assess the Navy’s AN/BSY-1 combat system program in terms of cost, schedule, and performance goals. To evaluate AN/BSY-1 and AN/BSY-2 program progress, we discussed cost, schedule, and performance issues with officials in Washington, D.C., at the Office of the Under Secretary of Defense for Research and ‘The total number of systems and associated costs reflects a 24 AN/BSY-1 combat system program. However, the program’s size has been reduced to 23 as a result of fiscal year 1990 appropriation decisions made for the SSN-688 construction program. Page 10 GAO/NSIAD-90-72Submarine Combat Systems Chapter 1 Introduction Engineering, the Office of the Under Secretary of Defense for Test and Evaluation, the Office of the Secretary of Defense for Program Analysis and Evaluation (Cost Analysis Division), the Naval Sea Systems Com- mand program offices responsible for these two programs, and the Kavy’s Board of Inspection and Survey. We also discussed these issues with officials at the l International Business Machines Corporation, Manassas, Virginia; l General Electric Company, Syracuse, New York; l Naval Underwater Systems Center, Newport, Rhode Island, and its on- site office at International Business Machines Corporation, Manassas, Virginia; l Supervisors of Shipbuilding, Conversion, and Repair, Groton, Connecti- cut, and Newport News, Virginia; l Operational Test and Evaluation Force, Norfolk, Virginia; l Newport News Shipbuilding and Drydock Company, Newport News, Virginia; . Electric Boat Division of General Dynamics Corporation, Groton, Con- necticut; and . Charles Stark Draper Laboratory, Cambridge, Massachusetts. We also reviewed and analyzed documents that describe and track sys- tem requirements, development, and acquisition. We discussed internal control system compliance with officials in Wash- ington, D.C., at the Department of Defense’s (DOD) Office of the Inspector General; the Naval Sea Systems Command’s, the Naval Space and War- fare Command’s, and the Naval Underwater System Center’s Offices of Internal Review; and the Naval Audit Service. We also discussed General Electric’s Underwater Systems Division’s internal control compliance with officials at the Defense Contract Audit Agency and the Defense Contract Services Administration in Syracuse, New York. We reviewed the Defense Contract Audit Agency’s audit of General Electric’s AN/BSY-2 full-scale engineering development proposal. We conducted our review between April 1988 and November 1989 in accordance with generally accepted government auditing standards. DOD provided written comments on this report. These comments are pre- sented and evaluated in chapters 2 and 3 and are included in appendix II. Page 11 GAO/NSIAD-90-72Submarine Combat Systems .’ Chapter 2 , SSN-688 Combat System Problems Affect the Readiness of Some Submarines Ambitious development objectives and schedules have caused the AN/ BSY-1 combat system program to experience cost increases and schedule delays. AN/BSY-1 life-cycle cost’ estimates more than doubled between February 1986 and November 1988. In addition, AN/BSY-1 design changes were the major contributor to several submarines being deliv- ered later and costing more than originally planned. As a result, AN/ BSY-1 delivery dates were revised to coincide with shipyard require- ments. Revised AN/BSY-1 delivery dates have been met and, as of November 1989, two of the four system upgrades have been delivered. However, only one of the two upgrades has been installed on improved ~~~-688s. Thus, these four submarines are not fully mission capable or able to counter the evolving threat until the upgrades are completed. The fully compliant AN/BSY-1 combat system has not been operation- ally tested under realistic at sea conditions. However, land-based assess- ments show the system has the potential to be operationally effective and suitable, although it will provide less performance capabilities than originally planned under the SUBACSBasic program to alleviate program schedule and cost risks. Total estimated AN/BSY-1 program costs have increased 125 percent Program Costs since February 1986 when the Navy estimated the life-cycle costs at about $5.4 billion for 19 systems, trainers, and spares. In November 1988, the Navy estimated these costs at $12.1 billion for 24 systems and associated equipment. (See table 2.1.) Table 2.1: Estimated AN/BSY-1 Life- Cycle Costs Escalated dollars In millIons Feb. 1986 Nov. 1988 Difference Development $1,227.7 $1,211 0 $-16.7 Productton 2.588 3 3.513 8 925 5 ODeratIon and support 1,578.l 7,424 7 5.826 6 Total $5,394.1 $12,149.5 $6,755.4 ‘Life-cycle costs refer to total costs to the government to develop, produce, construct, operate, sup- port, and own a system over a specified life span. Page 12 GAO/NSIAD-90-72Submarine Combat Systems Chapter 2 SSN-688Combat System Problems Affect the Readinessof Some Submarines Production costs increased primarily from the addition of five systems, six new wide aperture arrays,? spares, and repair parts. Operation and support cost estimates increased about $6 billion, from $83 million per system in February 1986 to $309 million per system in November 1988. The substantial increase was attributed to a revised cost estimating model that corrected assumptions made in calculating the 1986 operat- ing and support cost estimate. For example, according to an AN/BSY-1 program official, the February 1986 estimate did not include all operat- ing expenses and reflected a significant error in calculating the life expectancy of the systems. Timely delivery of the combat systems to the shipyards is essential to System Capabilities ensure that improved ~~~-688 delivery schedules are met. However, Not Provided on early combat system development, design problems with AN/BSY-1 , Schedule design changes to the ~~~-688s to accommodate AN/BSY-1, and other ship design changes led to ship construction delays and shipbuilder claims against the Navy. Further, a labor dispute at one shipyard led to an additional 6-month construction delay. Original and revised AN/BSY- 1 delivery dates based on shipyard need for the first five combat sys- tems with initial and full capabilities are shown in table 2.2. Table 2.2: Delivery Schedules for First Five Combat Systems Initial capabilities Required delivery to Full capabilities SSN Original shipbuildera Original Currentb 751 5187 II/87 9/88 11189” 752 8187 4188 12188 4/90 753 11187 l/89 3189 6191 754 l/88 7188 4189 12190 755 418% 5189” aThese dates are also the actual delwery dates bSchedule IS as of June 1989 but, accordmg to a Navy official, these dates are tentatwe and further delays may occur if shlpyard requirements change In accordance with the revised delivery schedule, the Navy accepted delivery of 5 of the 24 required AN/BSY-1 systems. However, because of ship construction delays, the shipbuilders were not prepared to install ‘A wide aperture array is a passive sonar that will be mounted on SSN-688 and SSN-21 hulls. It will provide enhanced capabilities over previous systems because it will locate targets faster and provide more accurate range and target motion analysis. It was originally planned for implementation in the SUBACS B tune frame. Page 13 GAO/NSLAD-90-72Submarine Combat Systems Chapter 2 SSN-688Combat System Problems Affect the Readinessof SomeSubmarines the systems so the AN/BSY-1 contractor retained them. Later, the Navy delivered the systems to the shipbuilders as government-furnished equipment. The four systems with initial capabilities were installed on the SSN-751 through SSN-754. However, upgrades to make the four sys- tems fully capable will be installed an average 19 months later than originally scheduled. Upgrades for the SSN-751 and SSN-752 systems were delivered to the shipbuilder, but only the ~~~-751 upgrade was installed. In its comments on this report, DODstated that the two system upgrades delivered through November 1989 were delivered on time. However, DOD’S milestone assessment is based on revised delivery to the ship- builder, whereas our late delivery assessment is based on the program’s original installation schedule or the PSAending date. According to the Navy, the remaining upgrades will be delivered in time for installation during each submarine’s PSA.The fifth system, with full performance capabilities, was delivered to the shipbuilder in May 1989 and installed in September 1989. The Navy does not expect delivery of the remaining 19 systems to be a problem. Although the Navy has taken delivery of five systems, the first four Performance were delivered without full performance capabilities. Until the systems are upgraded, the improved ~~~-688s will not be able to accomplish their expanded mission or to counter the evolving threat. According to Navy officials, these submarines would not normally be operated outside U.S. waters. i The AN/BSY-1 systems have not yet been subjected to operational test- ing under realistic, at sea conditions. Instead, the Navy has conducted developmental and preliminary tests on some AN/BSY-1 subsystems4 and a land-based operational assessment. On the basis of these tests, the Navy is confident that a fully capable AN/BSY-1 will improve the SSN- 688’s performance. Even though DODrequires that a production repre- sentative system undergo operational testing before proceeding beyond low rate production, the first full system testing under realistic, at sea ‘Although this is a normal sequence for newly constructed submarines between delivery and PSA, PSAs for the improved SSN-688s have been continuously delayed due to ship construction slippage. ‘Accordmg to the Navy, the SSN-751 fired 12 torpedoes with 9 hits during a weapon system accuracy test and tactlcal weapons training and certification firings. Three torpedo failures occurred. Accord- mg to the AN/B%-1 program manager, this performance is above the fleet average. The passive sonar also tracked MK-30 targets at ranges greater than any previous submarine. Page 14 GAO/NSIAD-90-72Submarine Combat Systems Chapter 2 SSN-688Combat System Problems Affect the Readinessof SomeSubmarines conditions will not be performed until after commitment is made to pro- cure all systems. Whether the AN/BSY-1 is more effective and is an improvement over older systems, however, cannot be fully demon- strated until a complete system is subjected to operational and technical testing under these conditions. This testing is planned to occur in fiscal year 1990 when the first improved ~~~-688 with a complete AN/B%?-1 system is subjected to technical and operational evaluation. System Deficiencies In June 1988 the Navy’s Board of Inspection and Surveys conducted an underway acceptance sea trial of the first AN/BSY-l-equipped subma- Preclude the Use of Some rine, the SSN-751. The Board’s July 15, 1988, report identified 20 defi- SSN-688s II. . for Intended ciencies of such significance that the submarine’s ability to perform its Missions missions was degraded and cited additional deficiencies relating to com- bat system reliability and maintainability. Most of the 20 significant deficiencies involved the AN/BSY-1 and existed because the upgrade to full performance had not been installed. Therefore, the first four subma- rines had limited acoustic and combat control capabilities. The Board recommended that the submarine be accepted and that a retrial be conducted after the system upgrades are installed. However, according to a Chief of Naval Operations official, a retrial will not be necessary because the Navy will rely on operational and technical evalu- ation results to determine the SSN-75l’s suitability for deployment. In an October 11, 1989, letter, the Board’s President agreed that a retrial of the first AN/BSY-l-equipped submarine was not required. In its com- ments on this report, DOD noted that all deficiencies the Board identified and attributed to the AN/BSY-1 had been corrected. Operational Testing DODregulations require that a major defense acquisition program undergo operational testing on a production representative system before proceeding beyond low rate production to confirm that the items or components actually tested are effective and suitable for combat. However, operational testing has not been conducted on a fully compli- ant AN/BSY-1 under realistic, at sea conditions, even though 20 of the 24 systems are under contract and the contractor has delivered five sys- tems to the Navy. According to DOD,this is a consequence of having a program enter low rate production concurrent with development. The Naval Underwater Systems Center has conducted a comparative analysis of the AN/BSi’-1 and its predecessor systems-the AN/BQQ-5 sonar and the CCS MK-1 fire control systems. The analysis shows the Page 16 GAO/NSIAD-90-72Submarine Combat Systems ’ Chapter 2 SSN-688Combat System Problems Affect the Readiness of Some Submarines AN/BSY-1 has the potential to provide significantly improved effective- ness over these systems in the acoustics and weapon launch areas, but the AN/BSY-l’s performance capabilities are less than originally planned under SUBACSBasic. In September 1987 the Navy’s Operational Test and Evaluation Force (OPTEVFOR)provided an observation of the AN/BSY-1 , and in September 1988 it recommended continuing the devel- opment process. However, it emphasized that the findings must be viewed in the context of extreme limitations. For example, OPTEVFORoffi- cials cautioned that, because of the lack of suitable data for projecting the system’s potential effectiveness and suitability, the observation was based on OPTEVFOR'S review of such items as development specifications, personnel, training concepts, and resource documents. The first AN/BSY-1 operational assessment, a land-based computer sim- ulation, was conducted in July 1989. In August 1989 OPTEVFORreleased an assessment of that simulation but again stated that the assessment was preliminary and the testing scope limited. On the basis of an incom- plete analysis of the test data and within the constraints of the testing scope, OPTEVFORprojected that the AN/BSY-1 had the potential to be operationally effective and suitable and again recommended continued system development. Although no major deficiencies were noted in most areas, additional testing to resolve critical operational issues and to complete evaluation of operational effectiveness and operational suitability is required due to limitations in the land-based test facility and in the testing scope. These limitations precluded the evaluation of certain effectiveness thresholds. For example, the land-based test facility was not representa- tive of the shipboard environment with respect to heat, noise, vibration, ship motion, electrical power, accessibility, electromagnetic compatibil- ity and interference, and interfacing systems. Additionally, mine laying, mine field navigation and avoidance, under ice operations, and underwa- ter communications could not be simulated. OPTEVFORstated that the assessment may be modified as a result of additional analysis. According to DOD,the final assessment of operational effectiveness and suitability will occur with technical and operational evaluation. How- ever, a fully compliant AN/BSY-1 system was not available for actual at sea operational testing until November 1989-when the SSN-751 com- pleted PSA.DODalso stated that the start of the evaluation is contingent upon the ~~751’s schedule following PSA.The Navy plans to start tech- nical and operational testing on the ~~~-751 in March and August 1990, respectively. According to AN/BSY-1 officials, the testing delay is due to Page 16 GAO/NSIALI-90-72Submarine Combat Systems Chapter 2 SSN-688Combat System Problems Affect the Readinessof Some Submarines the limited number of systems being procured, concurrent system pro- duction, and program schedule. However, by the time testing is com- pleted and results are reported, all 24 systems will be under contract and several additional systems will have been delivered. Thus, problems identified during testing will have to be corrected on all systems deliv- ered as well as those in production. The Navy will have to pay to correct any existing technical problems on systems. The next Defense Acquisition Board review for the AK/BSY-1 is the fis- cal year 1991 full production review. Because all systems will be under contract before the review, DODwill consider eliminating the AN/BSY-1 review based on several factors, including the land-based test results. However, such test data are normally used to support approved limited production, while more comprehensive testing is used to support approval for full production. When SUBACSBasic was changed to AN/BSY-1, nine improved SSN-688s- Conflicting Combat five by Electric Boat and four by Newport News Shipbuilding-were System and Submarine under construction. With the approval of the AK/BSY-1 design, portions Schedules Increase of these submarines had to be redesigned to accommodate the AN/BSY-1 rather than the SI:BACSBasic design. Thus, the Kavy was required to pro- Risks vide various design and configuration data to the ship design agent in time to meet the ~~~-688 construction schedule if it was to avoid govern- ment liability for delay. However, untimely and defective data caused Electric Boat to incur significant delays and additional construction costs, resulting in Electric Boat submitting a request for equitable adjustment. Newport Kews Shipbuilding also has submitted a similar request. Between September 1986 and October 1986, Electric Boat alerted the Navy on several occasions that late and/or faulty AN/BSY-1 design data were affecting the ~3~688 construction schedule. During September 1986, Electric Boat notified the Navy that work on the SSN-751 was sig- nificantly behind schedule, that major design problems, primarily with Ah’/BSY-1, were causing the delay, and that the design must be finalized immediately to meet the submarine delivery date. On October 7, 1986, Electric Boat again cautioned the Navy that these major design problems were causing rework, delays, and stoppages to the construction of the ss1+751. Electric Boat again stated that the AN/BSY-1 must be delivered on schedule to support the improved ~~~688 construction schedule. Page 17 GAO/NSIALI-90-72Submarine Combat Systems _’ Chapter 2 SSN-688Combat System ProblemsAffect the Readinessof SomeSubmarines On July 17, 1987, Electric Boat submitted a request for equitable adjust- ment for about $97.1 m illion and schedule adjustments for five improved ~~~-688s.The request for equitable adjustment is based on the impact of formal and constructive changes (untimely and defective data) for the AN/BSY-1 design, retractable bow planes, and the place- ments of lead ballast, The Navy awarded Electric Boat a $824 m illion cost adjustment, including about $27 m illion for delivery delays. (See table 2.3 for cost and delivery information for specific submarines.) Table 2.3: Costs and Schedule Delays Due to SUBACS Basic Redesign Dollars in millions Total Adjustment Contract Revised Delay SSN adjustment for delay date date months 751 $353 $6.1 Nov. 1987a June 1988” 7 752 20.5 5.5 Mar. 1988” Jan. 198gb 10 754 12.9 6.3 July 1988 Sept. 1 98gb 14 755 89 6.1 Dee 1988 Mav 1990 17 757 4.8 3.0 June 1989 Julv 1991 25 Total $82.4 $27.0 “Because the Navy accelerated the delrvery dates for the SSN-7.51and SSN-752,we used thusdate rather than the contract dellvery date to determine the schedule vanance ‘Submarrne was delrvered on thusdate According to a Navy official, the total $82.4 m illion adjustment, with the exception of $650,000,”was exclusively for AN/BSY-1 design changes and delivery delays. As a result of changing the AN/BSY-1 program and a labor strike,‘l Electric Boat will deliver five submarines to the Navy an average of about 14 months late. In addition, Newport News has submit- ted a similar request for its first nine submarines. As of June 1989, Newport News had submitted three of eight incre- ments to its request for equitable adjustment. These three increments represent an increase to the ceiling price; of about $198 m illion, includ- ing about $150 m illion for AN/BSY-1 design changes. It will deliver its first four submarines an average of 25 months late. ‘The $650,000 is for material for the retractable bow planes. The bow planes resulted in no delay. “In June 1988 Electric Boat shipyard workers went on strike over wage increases An agreement was reached in October 1988; a phased comeback of employees was initiated and continued through December 1988. As an aftermath of the strike, Electric Boat was unable to rehn-esufficrent quahfred workers. ‘The contract ceiling is the total dollar amount for which the government is hable. Page 18 GAO/NSIAD-90-72SubmarineCombat Systems Chapter 2 SSN-688Combat System Problems Affect the Readinessof Some Submarines In an attempt to resolve cost, schedule, and performance problems expe- Conclusions rienced with SUBACSBasic, the Navy established the AN/BSY-1 combat system program. However, ambitious program objectives and schedules also have caused problems within the AN/BSY-1 program. AN/BSY-1 has experienced significant increases in total program costs and system design problems. In addition, because the time to correct all AN/BSX-1 design and development problems was insufficient, the AN/B8Y-1 became the major contributor to delays in the improved ~~~-688 con- struction program. This resulted in increased ~~-688 submarine costs and delayed the introduction of new and fully mission capable subma- rines into the fleet. These construction delays, in turn, caused a rippling effect on AN/BSY-1 schedules. To meet the improved ~~~-688 construction schedule, the Navy accepted delivery of the first four systems without full performance capabilities. These four systems are to be upgraded to full performance during each ship’s PSA.However, since PSAcannot occur until after the improved SSN- 688 construction is completed, only the SSN-751'Ssystem has been upgraded to full performance. The Navy has accepted three AN/BSY-l-equipped improved ~~~-688s. However, with the exception of the SSN-751, these submarines have only limited performance capabilities and cannot accomplish their mission of conducting prompt and sustained operations at sea until the systems are upgraded during PSA.In addition, because PSAfor the first AN/BSX-l- equipped submarine was only recently completed, a fully compliant AN/ BSY-1 has not been field tested under realistic, at sea conditions. Never- theless, AN/BSY-1 assessments indicate the systems have the potential to improve performance significantly and to be operationally effective and suitable. However, it is not known how much improvement the AN/ BSY-1 will achieve over existing systems or if operational thresholds will be obtained until the tests are conducted. DODgenerally agreed with the facts on the AN/BSY-1 system as pre- Agency Comments sented. It partially agreed with our statement that AN/BSX-1 design problems led to submarine construction delays and cost increases. It stated that other ship design changes also contributed to schedule delays and cost increases. However, during subsequent discussions, DOD officials agreed that AN/BSY-1 design changes were the major contribu- tor to ship construction delays and cost increases. Page 19 GAO/N&W-90-72 Submarine Combat Systems .- Chapter 3 Early AN/B%2 Combat System Problems Could Inerease Costs and Delay SSN-21 Delivery The successful development of a fully capable AN/B?%-2 combat system is critical to the ~~21 achieving its mission requirements. Dedicated to the SSN-21, the Navy has no backup combat system planned should the AK/B%‘-2 be unavailable when needed. The AN/BSY-1 cannot be substi- tuted because its configuration does not permit its installation on the ~~-21 without modifying the ship design and the combat system hard- ware and software. The AK/BSY-2 is intended to take advantage of sig- nificant noise reductions expected of the ss1+21; however, if the SSN-21 does not meet its noise quieting requirements, the AN/BSY-2 will be less effective. The AN/BSY-2 development program is in the first year of full-scale development and indications are that ambitious goals and development schedules may cause problems. Total program costs have decreased, from an estimated $16 billion to about $14 billion, but the program has experienced some schedule slippage and performance problems. Further schedule slippage and performance problems are possible and the poten- tial also exists for costs to increase. The greatest risk is the development of Ah’/BSY-2 software. An analysis of combat system program features shows many similarities between the AN/BSY-2 and its predecessors- SI'BACSand AT\;/BSY-1. Problems or delays in the development and pro- duction of the AN/BSY-2 could affect the SSN-21program. Estimated life-cycle costs for 29 AN/BSY-2 systems have decreased. As Program Cost shown in table 3.1, total life-cycle costs decreased from about $15.6 bil- lion to about $13.9 billion because the Navy eliminated one shore facility from which ~~~-21s were to be deployed. As a result, the resources (per- sonnel, equipment, training, spares, etc.) committed for the facility over the program’s life span are no longer required. Table 3.1: Estimated Life-Cycle Costs for AN/&Y-P Systems Dollars In mllllons Phase May 1986 June 1989” Difference Development $1,596 $1,737 $141 Procurement - 5,698 5.711 13 Operatton and Support 8,291 6,542 -1,749 Total $15,585 $13,990 $-1,595 These are prellmlnary cost estimates not yet approved by DOD Page 20 GAO/NSIAD-90-72Submarine Combat Systems . 1 Chapter3 Early A N /B S Y - 2C o m b a tSystemP r o b l e m s C o u l d IncreaseCostsa n d DelayS S N - 2 1 Delivery T h e A N /B S Y -2 p r o g r a m ’s a m b i tio u s d e v e l o p m e n ts c h e d u l e is b e g i n n i n g P r o g r a mS c h e d u le to slip, a n d th e N a v y h a s n o b a c k u p system p l a n n e d for u s e in th e S S N - 2 1 s h o u l d th e A N /B S Y -2 n o t b e available w h e n n e e d e d .T h e N a v y states th a t a b a c k u p c o m b a t system is n o t r e q u i r e d b e c a u s eth e A N /B S Y -2 h a r d w a r e a n d software a r e b e i n g d e s i g n e dto b e built a n d written m o d u - larly. T h e N a v y e x p e c ts th e software to b e delivered incrementally, with fin a l delivery ta k i n g p l a c e b e fo r e technical a n d o p e r a tio n a l testing. S h o u l d software delays h a p p e n , th e N a v y believes th a t th e m o d u l a r d e v e l o p m e n t a n d th e incremental delivery a p p r o a c h e swill give th e sys- te m a n d th e s s 1 + 21 basic warfighting capabilities while p r o b l e m s a r e resolved. T o m e e t th e ~ ~ a - 2 1 construction s c h e d u l e a n d to r e d u c e th e software d e v e l o p m e n t risk, th e N a v y p l a n s to h a v e A N /B S Y -2 p e r f o r m a n c e c a p a - bilities for th e first S S N - 2 1delivered in two p h a s e s .D u r i n g th e first p h a s e , all h a r d w a r e a n d m o s t software a r e to b e p r o v i d e d to th e ship- builder in N o v e m b e r 1 9 9 3 . D u r i n g th e s e c o n d p h a s e ,th e r e m a i n i n g soft- w a r e is to b e delivered to th e N a v y in N o v e m b e r 1 9 9 4 .T h e N a v y will c o n tin u e to evaluate installation p l a n s to incorporate this software a t th e earliest feasible d a te consistent with th e ship’s s c h e d u l eb u t b e fo r e c o m p l e tio n o f P S A te, n ta tively s c h e d u l e d d u r i n g fiscal year 1 9 9 6 . H o w - ever, th e A N /B S Y -2 p r o g r a m is b e h i n d its d e v e l o p m e n tm ilestones a n d schedules.A s a result, two critical N a v y system d e s i g n reviews h a v e b e e n d e l a y e d . In a d d i tio n , late s u b c o n tractor deliveries h a v e c a u s e d s o m e critical ite m testing to b e d e l a y e d . Unless th e s e p r o b l e m s a r e resolved, A N /B S Y -2 delivery s c h e d u l e c o u l d b e adversely a ffected. Navy P r o g r a m N a v y p r o g r a m a s s e s s m e n tsexpress c o n c e r n r e g a r d i n g A N /B S Y -2 s c h e d - A s s e s s m e n ts S h o w u l e delays. B e t w e e n M a r c h 1 9 8 8 a n d N o v e m b e r 1 9 8 9 ,th e A N /B ? % -2 d e v e l o p m e n t p r o g r a m h a d fallen 3 m o n th s b e h i n d schedule. S e v e r a l fac- C o n c e r n A b o u t S c h e d u le tors c o n tributed to th e delay, including initial u n d e r s ta ffin g , late a n d D e lays incomplete c h a n g e sin system r e q u i r e m e n ts, c h a n g e sin a c o u s tic a n d w e a p o n s designs, a n d th e n e e d to perform a d d i tio n a l w i d e a p e r tu r e array testing. A J u n e 1 9 8 9 N a v y a s s e s s m e n talso s h o w s u n d e r s ta ffin g c o n tin u e s to b e a serious p r o b l e m . A c c o r d i n g to th e a s s e s s m e n t,G e n e r a l E lectric’s overall staffing level, e x c e p t a t o n e s u b c o n tractor location, is c o n s i d e r e dseriously short o f p l a n n e d levels in certain h a r d w a r e a n d software critical skill areas, w h i c h c o u l d seriously a ffect fu tu r e costs a n d schedule. A M a r c h 1 9 8 9 a s s e s s m e n tindicates th a t b e c a u s eo f th e a m o u n t o f work required, th e lack o f sufficient available resources, a n d th e current P a g e2 1 G A O / N S L A D - 9 0 -S7u2b m a r i n eC o m b a tSystems Chapter 3 Early AN/BSY-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery schedule, architecture’ development was considered a high risk area. The assessment also indicates that history with other programs shows that early schedule problems become future cost growth and the AN/ BSY-2 program is no different. Critical Design Reviews Two important Navy design reviews that will determine the extent to which General Electric’s design meets system specification requirements Delayed have been delayed. The first is a preliminary design review to assess the Company’s hardware and software design approach. The second is a critical design review to assess whether detailed design specifications meet performance requirements before producing hardware and writing software. In March 1989 the Navy and General Electric agreed to delay the com- pletion of the preliminary design review from April 1989 to October 1989 and the critical design review from August 1989 to January 1990. Delaying these reviews will result in compressing the hardware and software development for at least 5 months to meet the unchanged sys- tem integration test schedule. In addition, the Navy’s February 1989 program assessment indicates that delays in completing the preliminary design review could prevent the ship design agent from receiving updated AN/BSY-2 data in a timely manner. Further delays will only exacerbate the problem. According to an AN/BSY-2 program official, two factors contributed to the delays. One, General Electric’s contract was awarded later than planned. Two, system specifications and designs were not detailed enough for the Navy to conduct the reviews on schedule. This occurred because General Electric needed more time to establish a firm system design before preparing detailed specifications and writing software. According to the official, the additional time should result in a better system design baseline and less problems as the system progresses through development and testing. The official also believes these sched- ule delays will not adversely affect the planned delivery of the first sys- tem. However, a June 1989 program review shows that General Electric’s performance to meet the critical design review has not improved, bringing into question the Navy’s ability to complete this review by January 1990. ‘According to the Navy, system architecture is the structural framework for all hardware and soft- ware components that are combined and integrated to implement the total system’s functions. Page 22 GAO/NSIAD-90-72Submarine Combat Systems Chapter 3 Early AN/BSY-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery According to DOD, the action taken concerning the design review delays will result in both cost and schedule savings and has not affected the scheduled delivery of the systems to the Navy. The preliminary design review, rescheduled to October 1989, was completed on schedule, and all required contractual documents were provided. Whether the actions taken will provide the Navy with cost and schedule savings, with no affect on delivery schedules, will not be known for sometime. However, the 5-month delay in conducting the reviews is a fact and results in 5 months less time to correct any system design defi- ciencies Because of the tight time frame, little flexibility is provided to deal with developmental problems. Therefore, we continue to believe that compressing AN/BSY-2 hardware and software development for 5 months makes it difficult to achieve the overall schedule and that the potential exist for delays in the scheduled delivery of the systems to the Navy. Some Critical Item Testing During the system design definition phase of the development program, Delayed according to the Navy, General Electric and the Navy identified several critical system items that could present significant cost, schedule, or performance risks during full-scale development. To mitigate these risks early and allow contingency plans or approaches to be developed in time to ensure that the overall development program would be successfully implemented, General Electric developed a critical item demonstration test schedule. However, Navy program assessments show that some crit- ical item tests have been delayed. Should problems occur when these tests are conducted, additional time will be needed to correct and retest the items. One subcontractor’s inability to deliver a cathode ray tube for AN/BSY-2 display consoles and another’s lag in designing the console hardware have resulted in delaying a critical shock test of this tube from Decem- ber 1988 to August 1989. In another instance, critical item tests to demonstrate system response times for the database management sys- tem, operator display consoles, and submarine switch network’ have been delayed from October 1990 to March 1991. General Electric had planned to perform demonstration tests on prototype hardware and then on tactical hardware. Now such tests will be conducted only on tactical hardware. The assessment indicates that, although the tests ‘The submarine switch network transfers data between various computers and links all functional processors for the workstations, supervisory displays and combat control. Page 23 GAO/NSIAD-90-72Submarine Combat Systems Chapter 3 Early AN/BSY-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery have been delayed, contingency plans to mitigate risks have not been submitted. Subcontractors Not Under As of September 1, 1989, General Electric had not awarded final con- tracts for the development and limited production phase of the program Contract to two subcontractors-International Business Machines and Librascope Corporation-that have important roles in developing hardware and software and assuring that the first system is delivered as planned. The central issue between General Electric and the subcontractors is deter- mination of a mutually agreeable contract price. It is important that final contracts be awarded as soon as possible after the beginning of the development phase to ensure that subcontractors “buy into” work they are responsible for performing and that a viable, overall system test schedule is developed. The International Business Machines is to perform work amounting to about 15 percent of the total value of General Electric’s contract. This work will include developing and producing a wide aperture array, a land-based engineering system, and other related AN/BSY-2 compo- nents. Also, General Electric will qualify International Business Machines as a competitor for fiscal year 1992 production systems. Librascope is to develop and produce the weapons launch group and the operator display consoles. These subcontractors were performing work under sustaining engineering or letter contracts. According to the Navy, delays in awarding these subcontracts have raised questions as to the subcontractors’ commitment to the overall development and testing program, introduced potential delays for test- ing and delivering critical hardware and software, and decreased confi- dence in the validity of the current test schedule. For example, a June 1989 Navy program review indicated that the absence of a contract with Librascope was posing a great risk in the completion of a General Elec- tric system integration test. In the long term, delays in awarding Inter- national Business Machines a contract may affect not only the leader/ follower acquisition strategy but also its ability to be a qualified compet- itor for the fiscal year 1992 production systems. According to DOD,slowness in subcontract definitization added to pro- gram risk. However, General Electric’s full-scale development contract with Librascope was definitized on October 6, 1989. Although the sub- contract with International Business Machines is not definitized, General Electric and International Business Machines have agreed on both scope Page 24 GAO/NSLAD-90-72Submarine Combat Systems Chapter 3 Early AN/M-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery and price. As of December 3 1,1989, the contract had not been signed. The Navy states that no subcontractor design efforts are being delayed and all are being performed to the correct baseline. Combat System and Conflicts with the AN/BSY-2 and the SSN-21design schedule have resulted in additional costs to the SSN-21 construction program. To SSN-2 1 Schedules Conflict ensure that the submarines’ design and subsequent construction will accommodate the AN/BSY-2, the ship designer (Newport News) needs system configuration data and information before construction begins. The Navy has taken actions to assure that the ship designer is aware of system changes and has provided General Electric data to reflect those changes. However, General Electric did not provide the ship designer with combat system data in a timely manner nor has it provided final design data. As a result, portions of the submarines had to be redesigned to accommodate late combat system data, increasing SSN-21design cost an estimated $5 million. If final AN/BSY-2 design data result in further changes to the SSN-21design, additional costs and delays could occur. In June 1988 General Electric provided the ship designer with prelimi- nary combat system design data that resulted in design changes to the submarine’s interior. The ship designer estimated that this would increase submarine design costs by $3.4 million. Subsequently, General Electric, among other actions, changed the AN/BSY-2 design by replac- ing a Navy standard computer with a commercially available processor. The Navy Supervisor of Shipbuilding, Conversion and Repair, Newport News, estimates that the redesign will cost an additional $1.6 million. The changes and data provided do not fully represent the final AN/BSY- 2 design. A General Electric official stated that the ship designer was urging General Electric to provide it with final AN/BSY-2 space and weight requirements to ensure the submarine design is compatible with the system. However, General Electric will not be able to provide all final ship design data and information until the Navy completes its criti- cal design review in January 1990. DOD,in its comments on this report, stated that attributing the estimated $5 million increase in ss~21 design cost to late delivery of combat sys- tem design data reflects a misunderstanding of the sequence of events. Before the AN/B,‘%-2 contract was awarded, the ship design agent was provided with a “notional” system baseline, which was the Navy’s best estimate of the system configuration prior to having a definitized con- tract. After contract award, the winner’s system baseline was disclosed Page 25 GAO/NSIAD-90-72Submarine Combat Systems Chapter 3 Early AN/B.‘%-2Combat System Problems Could Increase Costs and Delay SSN-21 Delivery to the agent. According to DOD, the fact that the cost of this change was less than $5 million illustrates the accuracy of the Navy’s initial estimates. DOD also stated that although AN/BSY-2 design changes have affected the ship’s design, many AN/BSY-2 changes have been made to accommo- date the ship design agent’s requests. Potential changes in both ship and combat system designs continue to be discussed and evaluated in open forums as to impact, necessity, and desirability among all affected parties. An AiY/BSY-2 program official stated that because of the standard risks of concurrent development efforts, changes to the AN/BSY-2 design resulting from the Navy’s design review could increase the submarine’s cost and delay submarine delivery. Thus, the Navy has taken precau- tionary measures, even though significant changes are not expected. For example, the Navy and the ship designer have biweekly meetings to address combat system issues and concerns. Also, the ship designer has been provided current AN/BSY-2 changes and has prioritized the data it needs from General Electric, and General Electric has scheduled its work to meet the ship designer’s needs. According to DOD, concurrent development programs for the ship and the combat system pose risks for both contractors and the Navy’s pro- gram offices; these risks have been mitigated to the maximum possible extent by the close coordination of the contractors and the Navy pro- gram offices. These risks were well understood from the Seawolf pro- gram inception. The Navy has provided a continuous stream of data to the agent and has prioritized the generation of design data to accommo- date the agent’s needs. According to DOD, as of November 21,1989, no unresolved interface issues between the ship and combat system designs existed. The first AN/BSY-2 will not be fully capable or meet mission capabilities First AN/BE&Z Will when delivered to the shipbuilder in November 1993. According to a Not Be Delivered With G eneral Electric official, General Electric is unable to develop, test, pro- Full Performance duce, and deliver a fully capable system in time for the first delivery. Therefore, in November 1987 General Electric proposed-and the Navy agreed-that the first AN/B.‘%-2 would be delivered to the shipbuilder in h’ovember 1993 complete with hardware but not all software. An AN/ BSY-2 program official stated that software would be delivered in two phases to mitigate software development schedule risks. The official Page 26 GAO/NSLAD-90-72Submarine Combat Systems . Chapter 3 Early AN/BSY-2Combat System Problems Could Increase Costs and Delay SSN-21 Delivery stated that delivering all software in November 1993 would have signifi- cantly increased software development risks and General Electric’s con- tract price. During the first phase, General Electric plans to deliver about 2.7 million lines of the required software (about 86 percent) in November 1993. According to an OPTEVFORofficial, the planned capabilities to be deliv- ered at that time will be sufficient to meet the expected threat. The sec- ond phase, which includes the remaining 426,000 lines of software (14 percent), will primarily add acoustic functions. This software is to be provided to the Navy in November 1994, before technical evaluation begins in July 1995. The extent to which system performance capabilities will be delivered depends on the nature and the seriousness of problems General Electric encounters as the program progresses through development and testing. Although it is too early in AN/B=-2 development to determine how the AN/BSY-2 will function under realistic operational conditions, software development risks could jeopardize performance capabilities. In its comments on this report, DOD stated the decision to install the final portion of software on the system after ship delivery is a factor of ship construction, integration and testing schedules, and system availability. As both the ship and the combat system are being developed, the Navy will continue to evaluate the installation schedule of the final portion of software. Technical Performance The Navy is developing a new computer, the enhanced modular signal Development Problems processor, for the AN/BSY-2 combat system. The original model selected (SIM B) had problems meeting its monitoring and fault localization and AN/BSY-2 initial program loading requirements. In addition, the esti- mated cost of providing the model increased $65 million because several planned users switched to another model (SIM E). In June 1989 the Sec- retary of the Navy approved the SIM E model for the AN/BSY-2 pro- gram. Program officials expect the conversion to occur by fiscal year 1992. Because the SIM E model will be more widely used, the Navy believes the cost will be reduced and savings will occur over the pro- gram’s life. However, there will be nonrecurring and replacement costs. In November 1989 the Navy estimated it would cost about $50 million to modify the SIM E model to meet AN/B,!%-2 requirements and $52 million to retrofit three ~~~-21s one maintenance trainer, and one land-based engineering system with the SIM E model. Page 27 GAO/NSIAD-90-72Submarine Combat Systems . Chapter 3 Early AN/BSY-2Combat System Problems Could Increase Costs and Delay SSN-21 Delivery General Electric also is experiencing other system architecture prob- lems. The Navy requires all key AN/BSY-2 components to have redun- dant capabilities. However, redundant capabilities do not exist for the system receiver and beamformer cabinets. Also, General Electric is resolving a technical problem that could result in a significant loss of display data and make target detection nearly impossible over a large area. Software Development Under the AN/BSY-2 program, the Navy will be significantly challenged to meet performance requirements within tight time frames and bud- Risks gets. For example, our March 1989 report’ on the combat system identi- fied several areas where the Navy’s early and continued management attention is desirable if it is to meet this challenge. Among them are the Navy’s ability to maintain the software development schedule due to massive software requirements and its use of Ada computer language. The AN/BSY-2 requires designing, developing, and testing about 2.8 mil- lion lines of new tactical software, the largest new computer software development effort ever undertaken for a submarine combat system. During system design definition, it appeared that this system would retain a significant amount of software from previous systems that, according to a technical direction agent official, would have greatly reduced risks associated with this effort. However, recent software esti- mates show that none will be retained from prior systems and that about 83 percent, or 2.3 million lines, of the AN/BSY-2 new design soft- ware will be accomplished using Ada language. Although DOD believes Ada has matured and over 100 DOD programs either use or plan to use Ada, our recent report’ shows that most experts agree that the Ada lan- guage had not yet matured. The Navy’s June 1989 AN/BSY-2 program review document shows the shortage of Ada qualified (experienced) personnel in software was par- ticularly a serious concern. Staffing, in certain functional areas, was 93 percent below the software plan for May 1989. Additionally, it shows General Electric’s ability to recruit qualified Ada personnel could severely affect both cost and schedule should General Electric have to contract out portions of its tasks. The AN/BSY-2 contractor’s software ‘Submarine Combat System: Technical Challenges Confronting Navy’s Seawolf AN/B%-2 Develop- ment, (GAO/IMTEC-89-35, Mar. 13, 1989). ‘Computer Language Standardization: Status, Costs, and Issues Associated With Defense’s Implemen- tatlon of Ada (GAO/IMTEC-89-9, Mar. 24, 1989). Page 28 GAO/NSLAD-90-72Submarine Combat Systems Chapter 3 Early AN/BSf-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery development plan estimates that up to 900 software personnel will be required to develop and integrate all software. - According to DOD,whether retaining software from prior systems decreases or increases risk is a matter of opinion and depends on many technical factors: architecture/partitioning, depth and quality of docu- ments, interface definition, standards and conventions used, and lan- guage and operating system compatibility. The Navy, on the basis of problems with the SUBACSand AN/BSY-1 com- Similarities Between bat system programs, has adjusted its strategy for the AN/BSY-2. These Submarine Combat measures include establishing a 2-year system definition phase, develop- System Programs ing and testing software as separate modules called partitioning, and procuring and providing General Electric with hardware early enough to allow an additional year of development and testing within the overall schedule. Many similarities, however, still exist between the AN/BSY-2 program and its predecessors, SUBACSand AN/BSY-1 programs, that indi- cate the program could encounter future problems. An analysis of vari- ous program features (see app. I) shows similarities exist between schedules, software developments, concurrency, combat system capabil- ities, and at sea testing. Of particular concern is the similarities in pro- gram schedule and software development features. To meet the SW:! 1 lead ship construction schedule, the Navy established ambitious development objectives and schedules for the AN/BSY-2 pro- gram. Initially, the first AN/BSY-2 system delivered was to be fully capable; however, to maintain the required delivery schedule, the Navy is having it delivered in phases. Other schedule delays-overall, the pro- gram is 3 months behind and two important system design reviews have been delayed 5 months-have surfaced that indicate the potential for further slippages. Essentially, the same events (ambitious schedules and phased delivery) were experienced within the SUBACSand AN/BSY-1 programs. The AN/BSY-2 system requires twice as much new tactical software as the AI\;/BSY-1 system required. To attempt to design, develop, test, and integrate 2.8 million lines of new software, including 2.3 million lines written in Ada language, without adequately trained, experienced pro- grammers could affect the development schedule. Because of the soft- ware volume and the ambitious schedules, AK/BSY-2 software development will lag behind the hardware development and will be delivered in phases. Final software installation is planned to take place Page 29 GAO/NSIAD-90-72Submarine Combat Systems , Chapter 3 Early AN/B%-2 Combat System Problems Could Increase Costs and Delay SSN-21 Delivery in January 1995, more than 13 months after the hardware is scheduled to be delivered. AN/B%‘-1 program software development, although not written in the Ada language, also lagged behind the hardware develop- ment. As a result, the first four systems were delivered without a full software package. In its comments on this report, DODstated the lessons learned in develop- ing earlier combat control and acoustics systems, including AN/BSY-1, were incorporated into the planning for the AN/BSY-2 development effort. It is the first submarine system, for instance, to use the software development requirement of DOD-STD-2167.This, combined with (1) the longer development time for AN/BSY-2, (2) the benefits derived from the two-contractor, 2- year system design definition phase, and (3) the intensive early oversight, should allow AN/BSY-2 to profit from the sim- ilarities to earlier programs. In addition, the AN/BSY-2 phased software delivery is different from the AN/BSY-l’s in that the AN/BSY-2’s approach was taken before award of the development contract rather than as a mid-development mitigation plan. DODfurther stated that, although more new code development is required for AN/BSY-2, the integration job will be easier since all code is written to the same standards, conventions and style guides; code parti- tioning was done from scratch; computer software configuration item interfaces were defined before detailed design; and code is written in the same language, developed with a common tool set and written for the same operating system. Also, DODbelieves that on the basis of the training programs in use at General Electric and its subcontractors and the nature of the standards, tools, conventions and guides being used in the program, AN/BSY-2 soft- ware programmers will be adequately trained. The AN/BSY-2 combat system program, like its predecessor SUBACSand Conclusions AK/BSY-1 programs, must meet ambitious objectives and schedules in order to meet the needs of the SSK-21lead ship construction schedule. Critical to the SSN-21 meeting its performance goals and missions requirements, the AN/BSY-2 program has little flexibility for delays. In developing the AN/B.‘%-2 combat system, the Navy took measures to mitigate the problems encountered within the SUBACSand AN/BSY-1 pro- grams. Despite these measures, the AN/BSY-2 program is experiencing technical and performance problems and is behind schedule, and the Page 30 GAO/NSLAD-90-72Submarine Combat Systems . Chapter 3 Early AN/BSY-2Combat System Problems Could Increase Costs and Delay SSN-21 Delivery potential exists for program cost increases and further schedule slip- pages. In the software development and testing areas, the Navy needs to take additional steps to maximize the possibility of delivering the first system as planned and without jeopardizing the cost, schedule, and per- formance of the SSN-21program. The h’avy’s attempt to design, develop, test, and integrate the large amount of new tactical software in the Ada language without adequately trained and experienced programmers could severely affect program cost and schedule. Moreover, two impor- tant system design reviews, which will establish the system’s final con- figuration, have been delayed at least 5 months, and could affect program cost and schedule. This delay could prevent General Electric from providing the ship designer with final AN/BSY-2 design data. Should further changes be required, the SSN-2I program’s cost could increase and its schedule could incur delays. DODpartially agreed with the facts presented on the AN/BSY-2 combat Agency Comments system program. In those instances where DODpartially agreed with our presentation or provided further clarification, we modified the report accordingly. DODstated that although some critical item tests have been delayed, many have been satisfactorily completed. The wide aperture array suc- cessfully passed its flow noise test in July 1989, the beamformer chip met requirements with significant margin on the first pass, the cathode ray tube successfully completed both shock and magnetic susceptibility tests in August 1989, and the database manager and executive were suc- cessfully ported to switchnet, meeting all timing requirements. We note, however, that the tests conducted were not as comprehensive as the scheduled critical item tests and did not include all hardware. For example, the tests did not include operator display consoles that were to have been a part of the original August 1989 tests. Page 31 GAO/NSIAD-90-72Submarine Combat Systems , Appendix I Comparison of Past and Present Combat System Program Features Program SUBACS Basic and features AN/BSY-1 AN/BSY-2 Cost cap Set under AN/BSY-I after cost, Set before full-scale englneerlng schedule, and performance development and lImIted problems surfaced lsroductlon Development Ambitious development objectIves Ambitious development objectIves objectIves and and schedules established to meet and schedules to meet SSN-21 schedules ship constructlon schedule. construction schedule InsuffIcIent lnsufftcient time to address time to deliver fully tested and problems and deliver fully tested capable system and fullv caDable svstems. Software Originally required 1.4 milllon lines Requires 2 8 mIllIon lines of new intensiveness of new software tact’ical software In addition, most new software wntten In Ada programminq lanquaoe Concurrency Program delivery schedule required Program dellvery schedule requires five systems to be developed and three systems be developed and produced concurrently Droduced concurrentlv Combat Software and hardware not fully Not all software to be developed capabIlItIes developed when four systems were and tested when first system delivered to shipbuilder. First delivered to shIpbuIlder First system was to be fully capable system to be fully capable wlthln 2 about 17 months after the months after submarine delivered submarine was delivered to Navy. to Navy At sea testing Limited testing before 14 Limited testing to be completed production combat systems were before two production systems are to be delivered. to be delivered Management Sufficient visiblllty and Sufficient visibility to witness and management authority. Navy observe development and testing report, however, identified several No authority under fixed-priced program management incentive contract to direct weaknesses. changes to ensure goals are met unless the contract IS modlfled. This could potentially Increase cost and delay the schedule OversIght Program manager was responsible Program manager primarily for informing the Navy, DOD, and responsible for Informing the Navy the Congress of program goals and and the Congress of program goals problems and problems DOD to review program status annually, with emphasis on software development. Page 32 GAO/NSIAD-90-72Submarine Combat Systems Appendix II Comments From the Department of Defense DIRECTOR OF DEFENSE RESEARCH AND ENGINEERING WASHINGTON, DC 20301-3010 Mr. Frank Conahan Assistant Comptroller General, National Security and International Affairs Division U.S. General Accounting Office Washington, DC 20548 Dear Mr. Conahan: This is the Department of Defense (DOD) response to the General Accounting Office (GAO) draft report, "NAVY ACQUISITION: Progress and Problems in Implementing Navy Submarine Combat System Programs," dated October 2, 1989 (GAO Code 394236/OSD Case 8135). The DOD generally concurs with most of the report findings. The detailed DOD comments on the report findings are provided in the enclosure. The opportunity to comment on the draft report is appreciated. Sincerely, Robert C. Duncan Enclosure Page 33 GAO/NSIAD-90-72Submarine Combat Systems l AppendivII Comments From theDepartment ofDefense GAO DRAFT REPORT- DATED OCTOBER2, 1989 (GAO CODE 394236) OSD CASE 8135 "NAVY ACQUISITION: PRXRESSANDPRO~INIMPLEMEZJTING NAVYSUBMARINECOMBATSYSTEMPRWRAMS" DOD RESPONSETO THE GAO DRAFT REPORT ***** FINDINGS . FINDINGA: Backaround: Improved Combat Svstems. The GAO reported that, to enhance the performance and ensure the contin- ued superiority of U.S. nuclear submarines, the Navy will equip both the Los Angeles class (SSN-688) and the new Seawolf class (SSN-21) nuclear attack submarines with new and improved combat systems--the AN/BSY-1 and AN/BSY-2, respectively. The GAO explained that these systems are designed to reduce the time between detecting a target and launching weapons. The GAO observed that advanced submarine combat development began in 1980 with the Submarine Advanced Combat System (SDBACS) program. The GAO noted that, initially, this was to be a single-phase program to upgrade improved nuclear attack submarines authorized in FY 1989 and beyond. The GAO reported , however, that in October 1983 the Secretary of Defense approved a three-phased plan (SUBACSBasic, SUBACSA and SUBACSB) for the SSN-688s authorized in FY 1983 and beyond--six years earlier than origi- nally planned. The GAO also reported that a modified SUBACSB system was to serve as the baseline for the SSN-21 combat system. The GAO found that, in 1985, because of program cost, schedule and technical problems, the Navy restructured SUBACS into two separate development efforts--the AN/BSY-1 and AN/BSY-2. The GAO reported that the SUBACSBasic contract was renegotiated to establish AN/BSY-1 requirements. The GAO found that, because there was insufficient time to correct all design and develop- ment problems and meet the improved SSN-688 construction sched- ule--two versions of the AN/BSY-1 combat system have been delivered. The GAO noted that the first version (the prelimi- nary product baseline system) was installed on SSN-751 through SSN-754--providing those four submarines with limited self-de- fense capabilities necessary to operate until the systems are upgraded to include offensive capabilities. The GAO found that Page34 GAO/NSIAD-90-72SubmarineCombatSystems Appendix11 CommentsFrom theDepartment of Defense the second version will be installed on the remaining 20 SSN-688s--starting with SSN-755. Finally, the GAO reported that the Navy combined SUBACSA and SUBACSB performance requirements and renamed this effort the AN/BSY-2, which is currently in full-scale development and is to Now on pp 8-11 be installed on 29 new SSN-21s. (pp. 7-13/GAO Draft Report) . DOD RESPONSE: Concur. . FINDING 8: AN/BSY-1 Cost Increases and Schedule Delavs. The GAO reported that, between February 1986 and November 1988, the AN/BSY-1 life cycle cost estimate more than doubled--from about $5.4 billion to $12.1 billion. The GAO noted that the substan- tial increase (shown in table 2.1) is attributed to a revised cost estimating model that corrected assumptions on operating and support costs. The GAO also reported that the program has experienced schedule delays. The GAO compared original and revised AN/BSY-1 delivery dates for initial and full capabilities. The GAO noted that, according to a Navy official, the current dates for full capa- bilities are still tentative and further delays may occur. The GAO found that, while four systems have been installed, upgrades will be delivered--on average-- 19 months later than originally scheduled. The GAO observed, however, the Navy nevertheless maintains that the upgrades will be delivered in time for each submarine's post shakedown availability. The GAO further observed the Navy also takes the position that the delivery of the remaining systems (including one already delivered) will not be a problem. The GAO concluded that ambitious program objec- tives and schedules have caused problems with the AN/BSY-1 program. The GAO further concluded that the SSN-688 combat system problems affect the readiness of some submarines. (PP. Now on pp 12-14 and 14-17, P. 25-26/GAO Draft Report) 17-18 l DOD RESPONSE: Concur. It should be noted that each of the five systems delivered to date has met shipyard construction sched- ules. The two system upgrades delivered to date (for SSN 751 and SSN 752) were delivered on time. The research, development, test and evaluation costs have not changed. Only the production costs increased to reflect an increase in the number of systems. Overall, cost growth was basically a result of changes in assumption of ship's operating years. l FINDINGC: AN/BSY-1 Perfomance. The GAO reported that, until the first four systems already delivered are upgraded, the improved SSN-688s will not be able to accomplish their expanded Page35 GAO/NSIAD-90-72SubmarineCombatSystems Appendix11 CommentsFromtheDepartmentofDefense mission or to counter the evolving threat. The GAO noted that, 1 according to Navy officials, these submarines normally would not be operated outside U.S. waters. The GAO found that, in June 1988, the Navy's Board of Inspection and Surveys conducted an underway acceptance sea trial of the first AN/BSY-l-equipped submarine, and listed 20 significant deficiencies (most involv- ing the AN/BSY-1) that degraded the submarine's ability to perform its missions. The GAO noted that, although the Board recommended a retrial after the system upgrades are installed, a Chief of Naval Operations official stated that a retrial will not be necessary because the Navy will rely on operational and technical evaluation results to determine the submarine's Now on pp 14-17 suitability for deployment. (pp. 17-19, pp. 25-26/GAO Draft Report) DOD RESPONSE: Concur. The President, Board of Inspection and Survey, in a letter dated October 11, 1989, stated that a retrial of the first AN/BSY-1 equipped submarine is not required. (m: All deficiencies attributable to AN/BSY-1 program noted by the Navy's Board of Inspection and Surveys have been corrected.) FINDING D: Operational Testinu Not CoWded on AN/BSY-1. The GAO reported that, despite the fact Federal and DOD regulations require that a major defense acquisition program undergo opera- tional testing on a production representative system before going beyond low-rate production --a fully compliant AN/BSY-1 has not been operationally tested under realistic, at-sea, condi- tions. The GAO did note that the Naval Underwater Systems Center conducted a comparative analysis that showed the AN/BSY-1 has the potential to provide significantly improved effective- ness over its predecessor systems. The GAO also noted that in, September 1987, the Commander, Naval Operational Test and Evaluation Force, provided observations on the system and recommended continuing development--but at the same time cau- tioned on the lack of suitable data for projecting the system's potential effectiveness and suitability. The GAO further noted that, in August 1989, the Naval Operational Test and Evaluation Force assessed a land-based computer simulation of the AN/BSY-l--but again cautioned that the testing scopewas limited. The GAO reported that the Naval Operational Test and Evaluation Force stated that, within the described constraints, the system had the potential to be operationally effective and suitable. The GAO observed that, although no major deficiencies were noted in most areas, additional testing is nonetheless required to resolve critical operational issues and to complete the evalua- tion--due to the land-based test facility limitations and to the limitations in the testing scope. Page36 GAO/NSIAD-90s72SubmarineCombatSystems Appendix11 Comments Fromthe DepartmentofDefense The GAO found that a fully compliant AN/BSY-1 system will not be available for actual at-sea operational testing until November 1989, when the SSN-751 completes post shakedown availability. The GAO observed that the Navy plans to start technical and operational testing in March and August 1990, respectively. The GAO reported that, according to Navy officials, the delay in testing is due to (1) the limited number of systems being procured, (2) the concurrent system production, and (3) the program schedule. The GAO concluded that, by the time opera- tional testing is completed and the results are reported, all 23 systems will be under contract and several additional systems delivered. The GAO observed that those problems identified during testing will have to be corrected on systems already delivered. The GAO also observed that, once post shakedown availability is completed, the Navy will have to pay to correct any existing technical problems on the systems--if the warran- ties have expired. The GAO also noted that the next Defense Acquisition Board review of the AN/BSY-1 is the FY 1991 full production review, but because all systems will be under con- tract the DOD is considering eliminating this review. The GAO concluded that it will not be known how much improvement the AN/BSY-1 will achieve over existing systems or if operational thresholds will be obtained until the tests are conducted. Now on pp 15-17 and 19 (pp. 18-22, p. 26/GAO Draft Report) . DOD RESPONSE: Concur. A land based assessment was accom- plished. The preliminary assessment had generally favorable results. The Acquisition Strategy of concurrent development and low level production included the provision for the Commander, Operational Test and Evaluation Force, periodically to assess the AN/BSY-1 capabilities. These assessments occurred in 1988 and 1989, with the conclusion that the system is potentially operationally effective and operationally suitable for its mission. The final assessment of operational effectiveness and suitability will occur with Technical and Operational Evaluation. The start of Technical and Operational Evaluation is contingent on the SSN 751 post-Post Shakedown Availability schedule. . FINDING E: Conflicting AN/BSY-1 and Submarine Schedules Increase Risks. The GAO reported that, when SUBACSBasic was changed to AN/BSY-1, nine submarines were under construction. The GAO found that, with the approval of the AN/BSY-1 design, portions of the improved SSN-688s had to be redesigned to accommodate the AN/BSY-1 and the Navy was required to provide various design and configuration data to the ship design agent in time to meet the ship construction schedules. The GAO noted Page37 GAO/NSL%D-90-72 SubmarineCombatSystems , Appendix II Comments From the Department of Defense that, on several occasions between September and October 1986, due to non-receipt of sufficient ship design data from the ship design agent (Newport News Shipbuilding), the Electric Boat Company alerted the Navy that late or faulty AN/BSY-1 design data were affecting the construction schedule. The GAO also noted that, on July 17, 1987, the Electric Boat Company submit- ted a request for equitable adjustment in the amount of $97.1 million and, subsequently, the Navy awarded the company a $84.4 million cost adjustment. (The GAO listed these costs and the related schedule delays in table 2.3.) The GAO also reported that Newport News Shipbuilding has submitted a similar request for its first nine submarines. The GAO noted that the first three of eight increments of this request include about $150 million for AN/BSY-1 design changes. The GAO concluded that the amount of time required to revise the SUBACSdesign, produce AN/BSY-1 ship design data, and develop changes to ship construc- tion/installation drawings was insufficient to support the shipbuilder's construction schedule. This resulted in increased SSN-688 submarine costs associated with incorporating corrective ship design efforts. Further the GAO noted that even though the phased approach minimized the length of time operational capa- bility was lost in the early improved SSN-688's (SSNs 751-754), the delay in delivery of fully capable submarine resulted from restructuring SUBACS. The GAO further concluded that the construction delays have caused a rippling effect with new Now on pp 17-19 construction SSN-6881 schedules. (pp. 22-25/GAO Draft Report) . DOD RESPONSE: Partially concur. A number of submarine design changes, including those attributable to AN/BSY-1, resulted in the request for equitable adjustment. l FINDING F: AN/BSY-2 Cost and Schedule. The GAO reportedthat the successful development of a fully capable AN/BSY-2 combat system is critical to the SSN-21 achieving its mission require- ments, as the Navy has no backup combat system planned. The GAO noted that the Navy maintains that a backup combat system is not required because the AN/BSY-2 hardware and software were being designed to be built and written modularly. The GAO reported that the AN/BSY-2 system development program is in the first year of full-scale development and indications are that the ambitious goals and development schedule may cause problems. The GAO noted that (as shown in table 3.1) total life cycle cost decreased from about $15.6 billion to a preliminary cost estimate of about $13.9 billion, because the Navy elimi- nated one shore facility from which the SSN-21s were to be deployed. The GAO also noted that, to meet the SSN-21 construc- tion schedule and to reduce the software development risk, the Page 38 GAO/NSIAD-90-72SubmarineCombatSystems , . Appendix II Comments From the Department of Defense Navy plans to have AN/BSY-2 performance capabilities for the first SSN-21 delivered in two phases--(l) during the first phase, all hardware and most software are to be provided to the shipbuilder in November 1993, and (2) during the second phase, the remaining software is to be installed within 2 months after the submarine is delivered to the Navy in May 1995. -Schedule SlipPacre. The GAO also reported that, between March 1988 and March 1989, the AN/BSY-2 development program has fallen three months behind schedule. The GAO observed that, accord- ing to a Navy assessment one of the causative factors--understaffing--continues to be a seri- ous problem except at one contractor location. The GAO also noted other areas of concern iden- tified by the Navy--such as architecture devel- opment and early program schedule slippage . --sign F&views Delayed. The GAO reported that, in addition, two important Navy design reviews have been delayed-- reviews designed to determine the extent to which the contractor meets system specifications. The GAO found that, in March 1989, the Navy and the contractor agreed to delay the completion of the preliminary design review from April to October 1989, and the crit- ical design review from August 1989 to January 1990. The GAO noted that, according to an AN/BSY-2 program official, two factors contrib- uted to the delays--(l) the prime contract was awarded later than planned and (2) the system specifications and designs were not detailed enough for the Navy to conduct the reviews on schedule. The GAO noted that the same Navy official also maintained these schedule delays will not adversely impact the planned delivery of the first system. The GAO observed, however, that preliminary design review completion could be delayed further, since program assessments show delivery of key documentation has been delayed and documentation quality may not be adequate for Navy review purposes. The GAO pointed out the assessment shows that the con- tractor's performance to meet critical design review has not improved--bringing into question the Navy's ability to complete that review by January 1990. The GAO also observed that delay- ing the reviews will compress the hardware Page 39 GAO/NSIAD-90-72Submarine Combat Systems c Appendix11 CommentsFrom the DepartmentofDefense development and software writing by at least five months in order to meet the system integra- tion schedule. In addition, that the GAO noted the Navy's February 1989 assessment indicates that delay in the preliminary design review could delay the ship design agent's receipt of updated AN/BSY-2 design data. -Critical Tests Delaved. Finally, the GAO reported that the Navy had identified several items for critical tests that could present significant cost, schedule or performance risks. The GAO noted that Navy assessments indicated that some of these tests have been delayed. (The GAO cited some examples, such as shock testing of a cathode ray tube for the display consoles, as well as tests of various system response times.) The GAO concluded that these delays (of at least five months) could affect Now on pp 20-23 and program cost and schedule. (PP. 27-34, pp. 30-31 42-43/GAO Draft Report) . DOD RESPONSE: Partially concur. Although most of the facts in this finding, coming from internal Navy documents, are correct, the conclusions inferred from the facts are not always correct. Slipping both Preliminary Design Review and Critical Design Review to allow for more maturity in the design specification has been assessed to have no impact in the scheduled delivery of the system to the Navy. In fact, modeling, using the Pugh Roberts Association Program Management Model, indicates that taking these actions will result in both cost and schedule savings. The Preliminary Design Review Executive Session, rescheduled to October 3, was conducted on schedule and was fully supported by all of the documentation required from the contractor. Although some critical item tests have been delayed, many have been satisfactorily completed. The Wide Aperture Array success- fully passed its flow noise test in July; the beamformer chip met requirements with significant margin on the first pass; the cathode ray tube successfully completed both shock and magnetic susceptibility tests in August; and the database manager and executive were successfully ported to switchnet, meeting all timing requirements. . FINDINGG: Problems With Contractor Interfaces. TheGAO reported that, as of September 1, 1989, the contractor had not awarded final contracts to two important subcontractors--the Page40 GAO/NSIAD-90-72SubmarineCombatSystems Appendix II CommentsFromtheDepartment ofDefense central issue being a mutually agreeable price. The GAO noted that, according to the Navy, problems in awarding these con- tracts raises questions as to the subcontractor's commitment to the overall development and testing program, introducing poten- tial delays. The GAO concluded that, in the longer term, delays in awarding one of these contracts may also affect the leader- follower acquisition strategy. The GAO also reported that the submarine designer needs firm system configuration data on the AN/BSY-2 before construction begins. The GAO noted that the Navy has taken actions to assure that the ship designer is aware of system changes and has provided the AN/BSY-2 contractor preliminary data--but the AN/BSY-2 contractor has provided the ship designer neither timely data nor final design data. The GAO found that, as a result, portions of the submarines have had to be redesigned, increasing SSN-21 design cost an estimated $5 million. The GAO observed that, if final design data requires further changes to the submarine design, additional costs and delays could occur. The GAO found, however, that the AN/BSY-2 contractor cannot provide all final ship design data until the Navy completes its critical design review in January 1990. The GAO noted that, according to a AN/BSY-2 program office official, changes result- ing from the Navy design review could increase cost and delay delivery of the submarine. The GAO recognized that the Navy has taken precautionary measures, such as biweekly meetings with the ship designer. The GAO reported that, in addition, the ship designer has prioritized the data it needs and the AN/BSY-2 contractor has scheduled its work accordingly. The GAO con- cluded, however, that the AN/BSY-2 design schedule is not compatible with the SSN-21 design schedule--resulting in addi- tional costs to the SSN-21 construction program. The GAO also concluded that, should further changes be required, this could Now on pp 24-26 and affect SSN-21 program cost and schedule. (pp. 27-37, pp. 30-31 42-43/GAO Draft Report) . DOD RESPONSE: Partially concur. As noted, slowness in definitization of subcontracts added to program risk. However, the General Electric Full Scale Development contract with Librascope was definitized on October 6. Although the subcon- tract with International Business Machines is not yet definitized, General Electric and International Business Machines have agreed on both scope and price. The contract is expected to be signed by the end of the year. The Navy contin- ues to press this issue, but it should be noted that no subcon- tractor design efforts are being delayed and all are being performed to the correct baseline. Page 41 GAO/NSIAD-90-72Submarine Combat Systems c Appendix11 CommentsFrom the DepartmentofDefense Concurrent development programs for the ship and the combat system pose risks for both the contractor and Navy program offices, but these risks have been mitigated to the maximum possible extent by the close coordination between the contrac- tors, as well as with the Navy program offices. These risks were well understood from the SEAWOLFprogram inception. It is not true that "the ship designer . ..needs firm system configura- tion data and information before construction begins." The Navy has provided a continuous stream of data to the ship design agent and has prioritized the generation of design data to accommodate the ship design agent's needs. As of this date, there are no unresolved interface issues between the ship and combat system designs. The "late combat system data, increasing SSN-21 design cost an estimated $5.0 million" statement reflects a misunderstanding of the sequence of events. Prior to award of the AN/BSY-2 con- tract, the ship design agent was provided with a "notional" system baseline, which was the Navy's best estimate of the system configuration prior to having a definitized contract. After contract award, the winner's system baseline was disclosed to the ship design agent. The fact that the cost of this change was less than $5 million illustrates the accuracy of the Navy's initial Navy estimates. Although AN/BSY-2 design changes have impacted the ship design, it is also true that many AN/BSY-2 changes have been made to accommodate ship design agent requests. Potential changes in both ship and combat system designs continue to be discussed and evaluated in open forums as to impact, necessity and desirabil- ity in the presence of all affected parties. . FINDINGH: First AN/BSY-2 W ill Not Be Deliver& W ith Full Performance. The GAO reported that the first AN/BSY-2 will not be fully capable when delivered to the shipbuilder in November 1993--because (according to a contractor official) the contrac- tor is unable to deliver a fully capable system in time. The GAO noted, therefore, that with the agreement of the Navy, the first system delivery will include all hardware but not all software. The GAO also noted that this first phase of the software, comprising 2.7 million lines, is planned for November 1993, and according to a Naval Operational Test and Evaluation Force official, will be sufficient to meet the threat. The GAO found that the remaining 426,000 lines, primarily for acoustic Now on pp 26-27 functions, are scheduled for January 1995. (pp. 37-38/GAO Draft Report) Page42 GAO/NSIAD-90-72SubmarIneCombatSystems , Appendix11 CommentsFromthe DepartmentofDefense . DOD RESPONSE: Concur. The decision to install the final portion of software after ship delivery is a factor of ship construction and the integration and testing schedules, as much as system availability. As both the ship and combat system are being developed, the Navy will continue to evaluate the instal- lation schedule of the final portion of software. . FINDING I: Software Development Risks. The GAO reported that, under the AN/BSY-2 program, the Navy will be challenged to meet performance requirements within tight time frames and budgets. The GAO noted that its March 13, 1989, report "Technical Chal- lenges Confronting Navy's Seawolf AN/BSY Development" (OSD Case 7944) identified the software development schedule as one of these challenges. The GAO observed that the AN/BSY-2 soft- ware represents the largest new software development effort ever undertaken for a submarine combat system. The GAO further observed that, while it appeared initially that the system could retain a significant amount of software from existing systems, a recent software estimate shows that it will all be new, includ- ing 2.3 million lines in the Ada language. The GAO also referred to its March 24, 1989, report "Status, Costs, and Issues Associated With Defense's Implementation of Ada" (OSD Case 7832), which indicated most experts agree that the Ada language has not yet matured. The GAO alsonoted (1) that the June 1989 Navy program review documentation shows a shortage of Ada qualified personnel as a serious concern and (2) that the contractor's ability to recruit such personnel could seriously affect both cost and schedule. The GAO concluded that, in the software development and testing areas, additional steps are needed to maximize the possibility of delivering the first system as planned and without jeopardizing the cost, schedule, and performance of the SSN-21 program. The GAO also concluded that the Navy's attempt to design, develop, test, and integrate the large amount of new tactical software in the Ada language-- without adequately trained and experienced programmers--could Now on pp 28-29 and 31 severely affect program cost and schedule. (pp. 39-40, p. 43/GAO Draft report) . DOD RESPONSE: Partially concur. Whether or not retaining code from prior systems decreases or increases risk is a matter of opinion and depends on many technical factors, (1) architec- ture/partitioning, (2) depth and quality of documentation, (3) interface definition, (4) standards and conventions used, and (5) language and operating system compatibiltiy. . FINDINGJ: Similarities Between Submarine Combat Svstem Pro- arams. The GAO reported that, based upon its experience with the SUBACSand AN/BSY-1 combat system programs, the Navy has Page43 GAO/NSLAD-90-72SubmarineCombatSystems Appendix II CommentsFromtheDepartment of Defense adjusted its strategy for the AN/BSY-2. The GAO noted that measures taken include-- (1) establishing a 2-year system defini- tion phase, (2) developing and testing software as separate modules, and (3) procuring and providing the contractor with software early enough to allow an additional year of development and testing. The GAO observed, however, that many similarities still exist between the AN/BSY-2 program and its predecessors-- which tend to indicate the program could encounter future problems. The GAO pointed to an analysis of various program features (Appendix l), which showed similarities between the systems' schedules, software development, concurrency, combat system capabilities, and at-sea testing. In the areas of particular concern, i.e., program schedule and software develop- ment, the GAO noted the ambitious development objectives to meet the ship construction schedule and the attempt to write 2.8 million lines of new software, without adequately trained programmers. The GAO concluded that, despite the Navy measures to mitigate problems encountered in earlier programs, the AN/BSY program is nonetheless experiencing problems and delays and the potential exists for cost increases and further slippages. (PP. Now on pp 29 and 30-31 4+44/GAO Draft Report) . DOD RESPONSE: Partially concur. The lessons learned in devel- oping earlier combat control and acoustics systems, including AN/BSY-1, were incorporated into the planning for the AN/BSY-2 development effort. It is, for instance, the first submarine system to employ the software development requirements of DOD-STD-2167. This, combined with (1) the longer development time for AN/BSY-2, (2) the benefits derived from the two-con- tractor, two year System Design Definition phase, and (3) the intensive early oversight, should allow AN/BSY-2 to profit from the similarities to earlier programs. In addition, the AN/BSY-2 phased software delivery is different from AN/BSY-1 in that this approach was taken prior to award of the development contract for AN/BSY-2, rather than as a mid-development mitigation plan. Although more new code development is required for AN/BSY-2, the integration job will be easier since (1) all code is written to the same standards, conventions and style guides, (2) code partitioning was done from scratch, (3) Computer Software Configuration Interfaces have been defined prior to detailed design, and (4) code is written in the same language, developed with a common tool set and written for the same operating system. Page 44 GAO/NSIAD-90-72Submarine Combat Systems * . Appendix II Comments From the Department of Defense Based on the training programs in use at General Electric and its subcontractors and on the nature of the standards, tools, conventions and guides being used in the program, the AN/BSY-2 software programmers will be adequately trained. RJZCO-TIONS NONE. Page 45 GAO/NSIAD-90-72Submarine Combat Systems Appendix III . Major Contributors to This Report Brad H. Hathaway, Associate Director, Navy Issues National Security and Frederick A. Bigden, Assistant Director, Navy Issues International Affairs Frances W. Scott, Evaluator Division, Washington, D.C. Richard E. Silveira, Regional Manager Representative Boston Regional Office Ralph L. Tavares, Evaluator-in-Charge Edmund L. Kelley, Jr., Evaluator Edward W. States, Regional Manager Representative Norfolk Regional Lester L. Ward, Evaluator Office (3Y423fq Page 46 GAO/NSIAD-90-72Submarine Combat Systems
Navy Acquisition: Cost, Schedule, and Performance of New Submarine Combat Systems
Published by the Government Accountability Office on 1990-01-31.
Below is a raw (and likely hideous) rendition of the original report. (PDF)