Acquisition Of Major Weapon Systems E-763058 Department of Defense BY THE COMPTROLLER GENERAL, I OF THE UNITED STATES -- COMPTROLLER GENERAL OF THE UNITED STATES WASHINGTON. D.C. 2O!X8 Bm i63058 To the President of the Senate and the Speaker of the House of Representatives This is our report on the acquisition of major weapon systems by the Department of Defense. Our review was made pursuant to the Budget and Accounting Act, 1921 (31 U.S.C. 53), and the Accounting and Auditing Act of 1950 (31 U.S.C. 67). In addition, you will receive a classified supplement con- taining summaries of our evaluations of the individual weapon systems covered by our study. More detailed classified studies have been prepared on each weapon system. Copies of these studies will be provided on request. Copies of this report are being sent to the Director, Office of Management and Budget; the Secretary of Defense; and the Secretaries of the Army, Navy, and Air Force. Comptroller General of the United States . 50 TH ANNIVERSARY 1921- 1971 Contents Page DIGEST 1 CHAPTER 1 INTRODUCTION 4 The Development Process for a Major Weapon System 5 Concepts of This Study 11 2 ASSESSMENTOF CRITICAL MANAGEMENTACTIONS 13 Requirements 13 Identification of need for a system 13 Definition of performance character- istics 19 Obtaining assurance of feasibility of performance requirements 24 Cost-effectiveness determinations 28 Stability of the program and its re- lationship to other programs 33 Subsystem development phasing and interfacing 36 Continuous trade-off between cost and performance 41 Technical Assessment 43 Organization for Program Management 48 Organizational "layering" 51 DOD-proposed action on acquisition management problems 56 3 SYSTEMCOST EXPERIENCE 57 Analysis of Cost Changes as of June 30, 1970 61 Quantity Changes 62 Engineering Changes 63 Support Changes 63 Schedule Changes 63 . Economic Changes 64 Estimate Changes 64 Unidentified Causes for Cost Change 65 SAR System 66 . CHAPTER Page 4 SYSTEMSCHEDULEEXPERIENCE 67 5 GENERAL OBSERVATIONS, CONCLUSIONS, AND REC- OMMENDATIONS 70 APPENDIX I Schedule of program cost data as of June 30, 1970 and arranged by acquisition phase and military service 81 II Letter from Director of Defense Research and Engineering of January 22, 1971, commenting on this report 83 ABBREVIATIONS Army Materiel Command CDC Combat Development Command CONARC Continental Army Command DA Department of the Army DOD Department of Defense GAO General Accounting Office OSD Office of the Secretary of Defense SAR Selected Acquisition Reporting SOR Specific Operational Requirement . ” , COMPTROLLERGENERAL'S ACQUISITIONOF MAJORWEAPON SYSTEMS REPORTTO THECONGRESS Department of Defense B-163058 DIGEST ------ WHYTHEREVIEWWASMADE The large investment required in recent years for acquisition of major weapons has impacted heavily on the resources available for other na- tional goals and priorities. Acquiring these major weapons involves substantial long-range commitment of future expenditures. Because of deep concern in the Congress on these matters and because of evidence that the weapon systems acquisition pro- cess has serious weaknesses, the General Accounting Office (GAO) has un- dertaken to provide the Congress and the Department of Defense (DOD) with a continuing series of appraisals of those factors most closely re- lated to effective performance in procuring major weapons. This report represents GAO's first such appraisal. FINDINGSAND CONCLUSIONS . 1 I . Concurrent with GAO's studies, over the last several months the Of- fice of the Secretary of Defense (OSD) and the military services have been engaged in a substantial effort to identify and solve prob- lems that have adversely affected the acquisition of major weapon systems in terms of compromised performance, delayed availability, and increased costs. GAO has found that generally the newer weapon procurements are following a slower development pace and procurement practices are more conservative than those of earlier periods. Be- cause many of the current programs are in early states of acquisition, evidence of the results of the changed concepts is not yet available to adequately assess them, but the outlook is brighter. 2. The identification of need for a weapon system and the relative priority to be assigned its development is a fundamental problem in acquisition of weapon systems. Initial decisions as to which weapon system will be developed and the priority of its development is made by any one of the mili- tary services, but DOD has no organized method by which such pro- posals can be measured against its total needs. Such a method is _ now under development but it is in its infancy. s-r . 3. In recent months, the Office of the Secretary of Defense and the mil- itary services have paid extensive attention to the persistent prob- lems of defining performance characteristics of weapon systems and Tear Sheet . of determining the technical feasibility of achieving that perfor- mance. There are many encouraging signs that these problems are being abated. Extensive efforts are being applied--early in the weapon development process --to identifying areas with high design risks and to con- structing and testing the hardware itself to demonstrate the feasi- bility of high-risk components before proceeding with further devel- opment. 4. In the preparation of and attention given to cost-effectiveness determinations, there was a wide range of quality. This variation has lessened the value of these studies to the entire acquisition process. 5. One of the most important unresolved problems in the management of major acquisitions is the problem of organization. The essence of the problem appears to be attempts to combine the specialized roles of major weapon systems acquisition management into more or less traditional military cotnnand structures. Because of this, there usually are a large number of organizations not directly involved which can only negatively influence the project. It occurs to GAO that ideally there should be a direct relationship * ; between the missions for which weapon systems requirements are deter- I mined; e.g., strategic deterrent, land warfare, ocean control, etc., * , ; and the organizational structure needed to acquire them. Such an I arrangement would facilitate grouping related weapon systems in pack- I ages of common mission and would permit putting together an acquisi- I tion organization of appropriate size and stature to handle these matters. Eventually, GAO believes, program management and organiza- tion will evolve along mission lines. There are other alternatives involved, but whichever is chosen must clearly provide for someone to be in charge, to have authority to I I make decisions and to have full responsibility for the results. The I Deputy Secretary of Defense has recognized that the correction of I this problem is fundamental to any real improvement and has stated i that he plans to pursue it aggressively. I I I 6. GAO found that, on 61 weapon systems where complete cost data were available, estimates to develop and produce the weapon system had i I increased some $33.4 billion. About one third of this increase, or I $9.5 billion, represented the difference between the estimate pre- I pared when the system was first approved for development (the plan- . 1 ning estimate) and an updated estimate prepared when the system was I about to be placed under a development contract. The remaining * I $23.9 billion increase was due to changes in quantities to be acquired. , and to a combination of such things as engineering changes, revisions I I to estimates, and provisions for increased cost due to economic in- I flation. (See p. 58.) I I I I I I 2 ; I I , I I RECOI'@dENLlATIONS ORSUGGESTIONS The Secretary of Defense should: I I 1. Make every effort to develop and perfect a Department-wide method-- now in its early stages of development--to be followed by all mili- tary services for determining two things: first, what weapon systems are needed in relation to the Department's missions; second, what the priority of each should be in relation to other systems and their missions. 2. Establish guidelines and standards for the preparation and utiliza- tion of cost-effectiveness studies. These guidelines should require that studies be updated and reviewed as part of the decision process when major changes in cost and/or performance require revised sched- ules for funding commitments. 3. Place greater decisionmaking authority for each major acquisition in a single organization within the service concerned, with more direct control over the operations of weapon systems programs and with sufficient status to overcome organizational conflict between weapon system managers and the traditional functional organization. I I ’ 4. Ensure that each selected acquisition report (a) contain a summary statement regarding the overall acceptability of the weapon for its mission, (b) recognize the relationships of other weapon systems complementary to the subject systems, and (c) reflect the current , status of program accomplishment. I I I AGENCY ACTIONSANDUNRESOLlrED ISSUES L DOD has been actively pursuing a program to improve the management of the acquisition of major weapons. The Deputy Secretary of Defense has assumed a significant role in this improvement program. It is too early to say how effective many of these actions will be; but, if ef- fectively pursued, they should result in better management. As GAO has noted previously, beneficial results of some of these actions have be- come apparent. I I The comments by DOD on this report express only a general reaction due II to the limited amount of time GAOwas able to allow for DOD review. 1I Because of the nature and importance of this subject, DODwants to ex- I amine the final report further. I , I I 1. iWi"TE&I.S FORCONSIDERATION BY THE CONGRESS This report provides the Congress with an independent appraisal of the complex problems associated with weapon systems development and procure- ment by DOD--a matter of serious concern in the Congress., I I Tear Sheet I 3 COMPTROLLER GENERAL'S ACQUISITION OF MAJORWEAPONSYSTEMS REPORT TO THECONGRESS Department of Defense B-163058 DIGEST ------ WHYTHEREVIEWWASMADE The large investment required in recent years for acquisition of major weapons has impacted heavily on the resources available for other na- tional goals and priorities. Acquiring these major weapons involves substantial long-range commitment of future expenditures. Because of deep concern in the Congress on these matters and because of evidence that the weapon systems acquisition pro- cess has serious weaknesses, the General Accounting Office (GAO) has un- dertaken to provide the Congress and the Department of Defense (DOD) with a continuing series of appraisals of those factors most closely re- lated to effective performance in procuring major weapons. This report represents GAO's first such appraisal. r s FINDINGSANDCONCLUSIONS 1. Concurrent with GAO's studies, over the last several months the Of- fice of the Secretary of Defense (OSD) and the military services have been engaged in a substantial effort to identify and solve prob- lems that have adversely affected the acquisition of major weapon systems in terms of compromised performance, delayed availability, and increased costs. GAO has found that generally the newer weapon procurements are following a slower development pace and procurement practices are more conservative than those of earlier periods. Be- cause many of the current programs are in early states of acquisition, evidence of the results of the changed concepts is not yet available to adequately assess them, but the outlook is brighter. 2. The identification of need for a weapon system and the relative priority to be assigned its development is a fundamental problem in acquisition of weapon systems. Initial decisions as to which weapon system will be developed and the priority of its development is made by any one of the mili- tary services, but DOD has no organized method by which such pro- posals can be measured against its total needs. Such a method is now under development but it is in its infancy. 3. In recent months, the Office of the Secretary of Defense and the mil- itary services have paid extensive attention to the persistent prob- lems of defining performance characteristics of weapon systems and of determining the technical feasibility of achieving that perfor- mance. There are many encouraging signs that these problems are being abated. Extensive efforts are being applied--early in the weapon development process--to identifying areas with high design risks and to con- structing and testing the hardware itself to demonstrate the feasi- bility of high-risk components before proceeding with further devel- opment. 4. In the preparation of and attention given to cost-effectiveness determinations, there was a wide range of quality. This variation has lessened the value of these studies to the entire acquisition process. 5. One of the most important unresolved problems in the management of major acquisitions is the problem of organization. The essence of the problem appears to be attempts to combine the specialized roles of major weapon systems acquisition management into more or less traditional military command structures. Because of this, there usually are a large number of organizations not directly involved which can only negatively influence the project. It occurs to GAO that ideally there should be a direct relationship between the missions for which weapon systems requirements are deter- mined; e.g., strategic deterrent, land warfare, ocean control, etc., and the organizational structure needed to acquire them. Such an arrangement would facilitate grouping related weapon systems in pack- ages of common mission and would permit putting together an acquisi- tion organization of appropriate size and stature to handle these matters. Eventually, GAO believes, program management and organiza- tion will evolve along mission lines. There are other alternatives involved, but whichever is chosen must clearly provide for someone to be in charge, to have authority to make decisions and to have full responsibility for the results. The Deputy Secretary of Defense has recognized that the correction of this problem is fundamental to any real improvement and has stated that he plans to pursue it aggressively. 6. GAO found that, on 61 weapon systems where complete cost data were available, estimates to develop and produce the weapon system had increased some $33.4 billion. About one third of this increase, or $9.5 billion, represented the difference between the estimate pre- pared when the system was first approved for development (the plan- ning estimate) and an updated estimate prepared when the system was about to be placed under a development contract. The remaining $23.9 billion increase was due to changes in quantities to be acquired and to a combination of such things as engineering changes, revisions to estimates, and provisions for increased cost due to economic in- flation. (See p- 58.) 2 ' RECO~NDATIONSORSUGGESTIONS The Secretary of Defense should: 1. Make every effort to develop and perfect a Department-wide method-- now in its early stages of development--to be followed by all mili- tary services for determining two things: first, what weapon systems are needed in relation to the Department's missions; second, what the priority of each should be in relation to other systems and their missions. 2. Establish guidelines and standards for the preparation and utiliza- tion of cost-effectiveness studies. These guidelines should require that studies be updated and reviewed as part of the decision process when major changes in cost and/or performance require revised sched- ules for funding commitments. 3. Place greater decisionmaking authority for each major acquisition in a single organization within the service concerned, with more direct control over the operations of weapon systems programs and with'sufficient status to overcome organizational conflict between weapon system managers and the traditional functional organization. 4. Ensure that each selected acquisition report (a) contain a summary statement regarding the overall acceptability of the weapon for its mission, (b) recognize the relationships of other weapon systems . complementary to the subject systems, and (c) reflect the current status of program accomplishment. AGENCY ACTIOK?ANDUNRli'SOLVED ISSUES DOD has been actively pursuing a program to improve the management of the acquisition of major weapons. The Deputy Secretary of Defense has assumed a significant role in this improvement program. It is too early to say how effective many of these actions will be; but, if ef- fectively pursued, they should result in better management. As GAO has noted previously, beneficial results of some of these actions have be- come apparent. The comments by DOD on this report express only a general reaction due to the limited amount of time GAOwas able to allow for DOD review. Because of the nature and importance of this subject, DOD wants to ex- amine the final report further. MATTERS FORCONSIDERATION BY THE CONGRESS . This report provides the Congress with an independent appraisal of the complex problems associated with weapon systems development and procure- ment by DOD--a matter of serious concern in the Congress. 3 CHAPTER1 INTRODUCTION The investment to acquire major Department of Defense (DOD) weapons impacts heavily on allocation of the Nation's resources. Acquiring these major weapons involves substan- tial long-range commitment of futureexpenditures. Because of this and because evidence exists that the weapon systems acquisition process has not been conducted in an efficient manner, there has been considerable congressional and public attention focused upon improving the process for acquiring major weapon systems. In the past year, several studies of the acquisition process for major weapon systems have been completed. These include studies by the Department of Defense Blue Ribbon Panel, the National Security Industrial Association, and the Defense Science Board Task Force on Research and Development Management. All these studies were critical of the systems acquisition process to some degree. More recently the Gov- ernment Operations Committee, House of Representatives, held hearings on policy changes in weapon systems acquisition. The Committee report on this subject, dated December 10, 1970, contained recommendations for several improvements and the Commission on Government Procurement is including major acquisitions as one of the subjects in its study. Recently, the Congress has called upon GAO to report periodically on the progress of various acquisition programs and to provide other forms of assistance that would make available to its committees and members more reliable infor- mation on which to base judgments concerning issues that in- volve its oversight, as well as its legislative function. In order to effectively respond to the interest and needs of the Congress to obtain more timely and comprehen- sive data on which to base an evaluation of the management of ongoing procurements, the General Accounting Office has initiated a long-term program which will help provide data for continuing appraisal. This report presents the basic format which GAO intends to use in its long-term evaluation. The GAO program is an effort to establish an approach conducive to nurturing greater agreement among the Congress, GAO, and DOD which will clarify facts and issues and result in improved man- agement of the acquistion process. Our intent is to de- velop an orderly process which will lead to a constantly improving body of basic data to assist all participants in $he making of critical weapon systems decisions. Another objective of this GAO program is to provide a recurring series of evaluations of the weapon systems ac- quistion process. In these reiterations, GAO will (1) re- examine overall acquisition process efficiency and (2) make detailed and comprehensive examinations of the process fol- lowed in most, if not all, of the individual major acquisi- tion programs. The consistency of format and the recurring nature of the evaluation program should aid in the annual review of these acquisitions by the Congress, as well as provide DOD with an independent assessment of the weapon . acquisition process. Finally, the GAO program is structured for recognition and appraisal of any improvement programs that DOD initiates for its acquisition process. It is not the intention of GAO to judge the propriety of technical decisions made by DOD but rather to evaluate the efficiency of the management and decisionmaking pro- cesses applied. THE DEVELOPMENTPROCESSFOR A MAJOR WEAPONSYSTEM Developing major weapon systems is a primary function of DOD. The development process is highly structured and complex. The combined process involves close interactions between needs of the user and the ability of the developer to fulfill them. A substantial portion of personnel of OSD and the mili- tary services are involved in the acquisition process. Costs of weapon development consume a large portion of the military budget each year. Large segments of industry are engaged in producing the needed weapons. More than 5 $150 billion is estimated to be necessary to acquire the weapon systems currently under development. Some $95 bil- lion of that amount is yet to be appropriated by the Con- gress. An oversimplified representation of the manner in which weapon systems evolve from an idea to production is shown in the following chart. (See figure I.) Conceptual phase--This is the initial phase in weapon acquisition. In this phase, need for new military capabil- ities is established, concepts are developed for a weapon system which will provide those capabilities,, and technical feasibility is explored and determined. The objective of this phase is to provide the technical, economic, and mili- tary bases for initiating full-scale development of the weapon system, Advancement to the next phase, validation, is dependent upon satisfying criteria designed to measure achievement of the conceptual phase's objective. There are six objectives which should be accomplished in the conceptual phase. First, mission and performance envelopes should be defined. Second, a thorough trade-off analysis must be made among the elements of cost, schedule, and performance to ensure that the most effective product is obtained when it is needed and at the most reasonable cost. Third, a military service must ensure that the best technical approaches have been selected for the new weapon system. Fourth, the service must provide assurance that engineering rather than experimental effort remains upper- most in the program and that the needed technology is avail- able. Fifth, that the cost effectiveness of the proposed weapon must have been determined to be favorable in relation to the cost effectiveness of competing systems on a DOD-wide basis. Sixth and last, the service must ensure that, inso- far as it can, the cost and schedule estimates are both credible and acceptable. When these prerequisite criteria have been fulfilled, the weapon program is ready to go into the validation phase. Secretary of Defense approval is re- quired to authorize the program to move into the validation phase. Validation phase--In this phase, the preliminary de- signs and engineering for the weapon system are verified or accomplished; management plans are made; proposals for en- gineering development are solicited and evaluated; and the 6 H development contractor selected. The objective of this phase is to verify that the technical and economic bases for initiating full-scale development of the weapon system are valid. Advancement to the next phase, full-scale de- velopment, depends upon establishment of achievable perfor- mance specifications for the weapon system that are sup- ported by an acceptable proposal from the development con- tractor selected. Secretary of Defense approval is re- quired for the program to move into the development phase. Pull-scale development--In this phase, the design and engineering of the weapon system is accomplished. The de- velopment contract is negotiated and awarded; the prototype of the weapon system is developed, produced, and tested; and the detailed specifications for manufacturing the weapon system are prepared. The objective of this phase is to develop a weapon system acceptable for production, Ad- vancement to the production phase must be authorized by the Secretary of Defense. The development phase overlaps the production phase since development is not considered complete until adequacy .. of the production model of the weapon system has been vali- dated by a series of production acceptance tests. Production-- In this phase, the weapon system is pro- duced in quantity for deployment. It begins when the pro- duction contract is negotiated and awarded. Production ac- ceptance tests are conducted to validate the adequacy of the production model of the weapon system. Quantity pro- duction is initiated and the first operational unit is equipped with the weapon system and trained in its use. Advancement to the operational phase occurs when the first operational unit equipped with the weapon system is de- ployed. Production continues, however, until all required quantities of the weapon system are produced. The produc- tion phase includes production tests, service acceptance tests, and user acceptance tests. Many potential weapon systems never progress beyond the early stages of consideration, e.g., conceptual phase. . - There are many reasons for this: unavailability of neces- sary technology, realization that a potential system may 8 become too costly for its intended purpose, anticipated obsolescence in terms of threat that the system is intended to counter, or another system concept subsequently may com- pete more effectively. As a system passes through valida- tion, however, the Government's commitment to it becomes firmer. By the time the system reaches full-scale develop- ment, the Government's commitment has become so great and the structure of the program so definite that major adjust- ments to the program are difficult because they almost al- ways delay critical delivery dates and are costly. Few re- ally acceptable options are available to the Government once the design is approved and a decision is made to begin pro- duction. The pattern of deeper involvement and decreasing op- tions is shown in the following chart (figure II). The greatest opportunity for broad decisions occurs during the early stages of acquisition. 9 H . P ‘O-6 H , - CONCEPTSOF THIS STUDY It was clear to GAO, when this study began, that the underlying management difficulties, as well as the problems of actually executing sound day-to-day actions at all levels, were probably deep seated and could best be evaluated by a systematic review of the entire process by using specific systems and phases as a basis for case studies. At the outset, critical major weapon acquisition man- agement actions and decisions, which would occur in every acquisition, were outlined. In determining these critical actions, DOD's own criteria and objectives were used, The critical management activities examined pertain to --requirements for systems, --assessment of technical progress, and --organization and procedures. We selected specific weapon systems now being acquired on which to conduct reviews on the basis of the criteria which had been developed. Several factors influenced our selection of specific weapon systems, First, we selected some of the systems where the Congress or DOD would have future options regarding a further course of action. Sec- ond, we selected a number of weapon systems which recently proceeded into the early phase of the acquisition process. This factor is most important, because problems occurring in the earlier phases may plague the system for years and adversely affect the cost, schedule, and performance of the system at a point when adjustments are difficult to make. As was noted earlier, it is also the point in time when the greatest number of options are available to both DOD and the Congress, Relatively small sums of money are committed at this stage, and therefore it is easier to change the direction of a program. As the program progresses, how- ever, choices will decrease and the responsible officials will tend to become committed to a particular course of ac- tion, until no options are left, Although there is little to be gained by dwelling on problems which have occurred in weapon systems where options were low, we have included a few such systems in our study since they provide the best means of assessing the full import of sound as well as un- sound past actions. 11 To fulfill our task, 45 systems (14 Air Force, 14 Navy, 17 Army) were reviewed. In addition, we reviewed cost and schedule data from a number of other systems. Still other systems have been reviewed at the request of congressional committees. In all, the data in this report are distilled from studies of some aspect of 70 weapon systems. In chapter 2, several of the management actions criti- cal to weapon systems acquisition are described in some de- tail and are followed by examples of good and poor perfor- mance. In chapters 3 and 4, information collected in this study on the costs and schedules of programs studied is presented in summary form to provide a useful basis for further analysis. Chapter 5 contains our observations, conclusions, and recommendations. ,Scope In order to review current policies and practices, we examined weapon systems which were in various phases of acquisition --conceptual, validation, full-scale development, or production. Information on these programs was obtained by review- ing plans, reports, correspcxldence, and other records and by interviewing officials at the system program office, intermediate and higher commands throughout the military departments, and the Office of the Secretary of Defense, We evaluated management policies and the procedures and controls related to the decisionmaking process, but we did not make detailed analyses or audits of the basic data sup- porting program documents. We made no attempts to (1) as- sess the military threat or the technology, (2) develop technological approaches, or (3) involve ourselves in de- cisions while they were being made. 12 CHAPTER 2 ASSESSMENTOF CRITICAL MANAGEMENTACTIONS In this chapter, several of the management actions crit- ical to weapon systems acquisition are described and are fol- lowed by examples of adequate and inadequate application of criteria. Although each example is based on an evaluation of the management of the particular weapon system procure- ment which is cited, these examples are mainly illustrative. It is not the purpose of this report to focus on any partic- ular acquisition. REQUIREMENTS Establishing requirements for weapon systems is an in- volved process. It is the basis for getting the system off on the right track and for controlling the development pro- cess. The process begins with identification of need for a specific capability and proceeds through such steps as de- fining performance characteristics, assessing the feasibil- ity of achieving them, establishing some relative priority of need, and selecting the system that promises to be most cost effective. Once requirements have been firmly estab- lished, a basis for important actions during the development process exists. Such actions are controlling changes, con- tinually making trade-offs between performance and cost, and controlling system phasing and interfaces. Requirements provide a yardstick against which action can be measured. The following section outlines our understanding of gen- eral criteria to be followed in principal acquisition process steps and some illustrative examples where the performance in meeting the criteria has been both good and bad. Identification of need for a system The first step in weapon system acquisition includes (1) evaluating the products of documented military department threat studies, (2) projected enemy force structures, and (3) operating command statements of requirements and trans- lating them into specific mission requirements and technology assessments. 13 The justification for selecting a particular major weapon system to fulfill the need includes analysis of con- cepts of existing and alternate capabilities, as well as the establishment of a relative priority of need. The clear identification of new mission requirements establishes a firm basis for initial and subsequent weapon systems and production. Key considerations for establishing needs are: Threat studies--Prepare future military risk positions and provide a justification of future needs. Mission requirements--Define system capabilities in terms of specific objectives and tasks required of the potential system or systems, including operational and logistics concepts. Current capabilities--Review abilities of existing sys- tems or modifications to them in relation to defined mission requirements, and identify areas of required technological advances. Technological advances required--Analyze alternate tech- nical approaches and generally identify technical risk areas in relation to mission objectives. Tactical concept of employment--Construct a detailed plan for the use of the weapon which sets the opera- tional limits, We have not attempted to pass judgment on military threat assessment; but we have examined methods that mili- tary services have followed to estimate current capability, to assess the potential for technological advancement,and to apply priority to a program relative to other weapon ac- quisitions. Following are some examples of instances where criteria were adequately applied and some where they were not ade- quately applied in determining the need for the system. The success which some of the weapon programs achieve in meeting their objectives for performance, schedule, and 14 cost confirms the usefulness of DOD's own criteria for se- lecting the specific weapon system to be acquired. A. Adequate application of criteria S-3A The S-3A need was identified and the decision to develop this weapon was made by a comprehensive analysis of future military requirements for carrier- based airborne antisubmarine warfare capability. In the analysis, the Navy and DODconsidered mission re- quirements, technological advancements required to develop the S-3A, and possible alternative.ways of satisfying the need. For example, the system that the S-3A is to replace was examined to determine whether it could meet the military requirement and whether it could possibly be modified to meet the re- quirement. Also considered was the feasibility of achieving technological advances needed to meet the performance planned for the S-3A program. (In addi- tion to this early attention to ensure technical success, actual commencementof the development cycle of the S-3A aircraft was slowed considerably because of continuing reassessment of program priority with the-land-based antisubmarine weapon system.) B. Inadequate application of criteria 1. LAMPS As early as 1957, the Navy stated a need to extend the weapon delivery range of destroyers to take ad- vantage of improved submarine detection capability. The Drone Anti-Submarine Helicopter program was first developed to fill this need. (This program was can- celed later due to limited capability and unreliabil- ity.) The Navy then considered filling the need for an antisubmarine warfare (ASW) capability on destroyers with a manned helicopter, the Light Airborne ASWVe- hicle (LAAV). Shortly after LAAV entered the con- ceptual phase, however, it was canceled, and effort 15 was directed to development of the Light Airborne Multi-Purpose System (LAMPS). This was done because it was felt within the Navy that OSD support for a strictly ASW system would be withheld. In order to %ellt' the system, the mission profile was expanded to include an Anti-Ship Missile Defense (ASMD) capability. At this point ASMD was added to the LAMPS capabilities and was given prior- ity over the ASW mission, Thus, although need for an ASW helicopter had been clearly demonstrated for a number of years, the Navy decided to develop a multipurpose helicopter. This decision led to 2-l/2 years of debate on how these mission requirements-were to be met within the weight restrictions that had been imposed on the he- licopter because of anticipated interface problems with the ship. 2. Mechanized Infantry Combat Vehicle (MICV) The MICV project began because of forecasts of threat to U.S. Forces and, in 1964, a change in Army mechanized infantry doctrine. From 1964 to 1966, the Army began a program to acquire an MICV for the 1960's--on an urgent basis. This effort was dis- continued because the vehicles were too heavy, were not mobile enough, and were not cost effective. In the meantime, the Army embarked upon a second pro- gram, to acquire an MICV for the 1970's. This pro- gram had high priority, since the vehicle was sched- uled for early deployment. The schedule has not been met, The vehicle is still in the conceptual phase, and deployment is ex- pected to be 5 years later than originally scheduled. The program has been drawn out for various reasons, including the priority for available fundsd The first major delay came in defining the vehi- cle's mission and characteristics. Approval of the definition was scheduled for March 1967 but was not made until October 1968. The delay occurred, in 16 part, because certain studies were considered inade- quate and additional work was required. An impor- tant factor was the complexity and resultant slow- ness of the Army's decisionmaking process. The October 1968 vehicle definition assigned the MICV "Priority I" and called for development on an "'urgent basis." In mid-1969, the project manager and higher commands sought, without success, Depart- ment of the Army (DA) approval to move the MICV out of the concept formulation phase. DA reviewed the program in the light of several factors including anticipated budget cuts, an increase in the MUX's estimated cost, and possible use of alternative ve- hicles which did not exist at the time of earlier studies. It considered several program alternatives, including (1) deferring the program a year,(2) de- ferring it a year and testing additional alterna- ' tive vehicles,and (3) terminating the program and developing one of the alternative vehicles. DA de- cided to authorize a review of all feasible compet- ing vehicle systems before deciding to move the MICV out of concept formulation. During the new review, completed in April 1970, the project manager established the concept of an "austere" MICV, which would have a lower cost be- cause of deletion of features that the Army earlier had termed "essential." In July 1970, he and higher commands recommended its adoption. At the time of our review, the DA decision was still pending. 3. SAM-D Development of the SAM-D system began although there was uncertainty over the utility of the system, the character of the threat which was to be coun- tered, and the capabilities of companion weapons with which the system would operate. Because of these uncertainties, in May 1967 the Secretary of Defense delayed the system's entry into full-scale development. Instead, the system was placed in an advanced development program to be conducted over a 3-year period. After 2-years in the advanced 17 development phases the system was studied in March 1969 to determine whether it should enter full- scale development. The Deputy Secretary of Defense directed that the system be continued in the ad- vanced development phase through fiscal year 1970 and that the decision to place the system into full- scale development be deferred until fiscal year 1971. His position was that the system would not be needed until sometime later, the number'of batteries needed and how the system would be deployed in the field was unknown, and the system was neither fully de- fined nor justified, In March 1970, the Army subjected the system to review by the Air Defense Evaluation Board. The Board was directed to again analyze the threat that the system had to meet,to identify the air defense capabilities required to defend against this threat, and to identify existing air defense capabilities and deficiencies to meet the threat, The Board's report was approved by the Chief of Staff on Novem- ber 19, 1970, and, in essences confirmed the Army's position on the need for the SAM-D, As of December 1970, no action had been taken by OSD. 18 Definition of performance characteristics Determination of weapon system operational requirements and performance characteristics (speed, range, accuracy, etc.) depends on well-defined mission statements. Perfor- mance characteristics are used to determine parameters of trade-off studies, performance feasibility studies, and phased system acquisition projection. Performance specifica- tions prepared from these characteristics are the basis for initial design feasibility studies and validation efforts. System design studies and development test programs derive from performance specifications. Absence of well- defined system specifications can cause underdesign or over- design. Completion of the entire development process, with- out actual satisfaction of system mission requirements can result from this absence. Conversely, it is more likely that an explicit definition of systm performance character- istics will result in an improved product. This is not to say, however, that system performance characteristics, once defined, must never be changed. This is an iterative pro- cess which becomes more firm as one approaches final design for production. Program management, to be effective, should allow for trade-offs as system development progresses and for unanticipated technical unknowns which are surfaced by detailed engineering design. Following aresomeexamples of instances where criteria for defining performance characteristics were adequately applied and some where they were not. A. Adequate application of criteria 1. A-X The definition of performance characteristics for the A-X weapon system flows from a clear, pre- cise statement of the mission this weapon system is intended to perform in support of the Army mission. The A-X mission is defined as close supporting fire for ground forces, armed escort, and armed re- connaissance in battle areas. It was determined, by contractor studies, that an aircraft with twin 19 engines, capable of takeoff and landing at forw&d operating bases, surviving hits by light antiair- craft artillery projectiles, having a rapid-fire gun and carrying bombs or rockets; having high subsonic speeds and a range sufficient for effective close air support, is required. A minimum of avionics for visual control is to be included initially with the weapon delivery. The aircraft, however, is to be designed with extra space and power so that more sophisticated avionics could bring its capability up to all-weather use, which the Army considers essential. The Air Force awarded competitive prototype development contracts for the A-X close air support aircraft on December 30, 1970. This will give the Air Force actual hardware upon which to base a deci- sion for further full-scale development and should . provide a sound basis uponwhich to establish firm 1 performance specifications. 2. Heavy Lift Helicopter (HLH) The basic military mission for HLJJ was articu- lated by representatives of the operating command. Some objectives of the mission were revised, how- ever, during early attempts to gain approval. One revision changed the mission emphasis from tactical to logistical. Another revision resulted from an Army/Navy compromise initiated by congressional interest in developing an HLH that would satisfy both the Army and Navy. Basic mission requirements established by the representatives of operational commands were in clear and concise terms. The change in mission ' emphasis caused appropriate changes in the mission statements and subsequently in performance character- istics. Performance characteristics, as well as all changes for the first two9 were developed by study groups from various Army organizations. These study 20 groups included representatives from the field, the project office, and engineers with various func- tional capabilities. The characteristics finally selected were considered the most desirable to achieve the established mission. Our review was completed before the Army/Navy compromise was ap- proved, but preliminary studies were conducted in- dividually by Army and Navy engineers who examined the compromise position. Their results indicated that a compromise on performance characteristics would limit some of the mission requirements of both the Army and Navy. In September 1970, the Secretary of Defense ap- proved a program to develop high-risk critical com- ponents for the HLH before full-scale development is approved, on the basis of performance require- ments agreed to by the ArmyandNavy. This approach assumes that advanced technolog- ical development of the critical components is nec- essary to (1) determine whether technology is avail- able to build such an aircraft system and to iden- tify the best technical approach offered by the he- licopter industry and (2) establish realistic cost estimates. Related studies concerning further re- finement of the mission, technology, and economy of the HLH will be made but will be subject to even further refinement on the basis of results of the component development program. If the critical component development is a success, this should per- mit a decision to be made whether or not to proceed with full-scale development. 3. HARPOON The HARPOONmissile is a good example of the Navy's thoroughness in defining the performance characteristics required of a system. The potential enemy threat and the mission profile of a new missile to meet thisthreatwere defined by the Chief of Naval Operations in June 1969. He specified certain restricting design characteristics. For example, the missile range required and the range desired were 21 specified. The maximum weight of the missile to be launched from a ship and from an aircraft was given. The requirement also specified that the mis- sile have an all-weather capability, i.e., be able to hit a target under specified adverse weather conditions. For a year, the Navy conducted numerous studies to determine how best to meet the requirement. The aerodynamic qualities of various,.missiPe designs combined with different kinds of propulsion systems were studied to ascertain whether the desired range could be obtained. Additionally, the reasonableness of the weight limitations was verified and studies were conducted of subsystems to find out if an effec- tive missile could be made within the limitation. The adequacy of the size of the warhead, which is one of the factors having a direct bearing on the weight, was tested by blowing up a number of obso- lete ships. Problems in selection of a seeker with all- weather capability were anticipated. Different kinds of seekers were tested in flight before the kind of seeker desired was identified. These studies and tests provide reasonable assurance that the re- quirements can be met before proposals are solicited from contractors. B. Inadequate ap$ication of criteria LAMPS Although performance characteristics of the LAMPS have been under study since early in its de- velopment, the Navy has had difficulty in agreeing on the gross takeoff weight of the helicopter, The significant factor contributing to this difficulty is the fact that the program has been managed by various committees within the Office of the Chief of Naval Operations and had no consistent sponsor to guide and control it. Naval committee representatives have varying vested interests in 22 the program and, as a result, agreement has been delayed on major questions such as the gross take- off weight of the LAMPShelicopter. Committee representatives from one organization within the Navy were pressing for a heavier helicop- ter. In their opinion, a light aircraft would not have the desired mission capability. Representatives from another organization wanted a light helicopter because it would fit on the DE-1052 class ship and would be available to the fleet sooner. Controversy centered around the question of whether the deck of the DE-1052 was strong enough td support the helicop- ter. Although the Deputy Chief of Naval Operations (Air) requested data on the maximumdeck strength of the DE-1052 early in the LAMPSprogram, testing of the deck for maximumallowable landing weight was not'accomplished until 2% years after the LAMPS program was started. The tests occurred in Novem- ber 1970, shortly after the Deputy Chief of Naval Operations (Fleet Operations and Readiness) succeeded the Deputy Chief of Naval Operations (Air) as the official primarily responsible for the LAMPS. The Navy's long delay in specifying the weight of the helicopter will result in a significant de- lay in delivery of the UMPS to the fleet. 23 Obtaining assurance of feasibility of performance requirements The probability of a technically successful develop- ment depends upon an assessment of the availability of proven technical knowledge required to build the item(s), by identifying design risk areas and assessing the likeli- hood of resolving them early in the development process. Feasibility of performance requirements is usually assessed as a part of conceptual studies and confirmed during the validation phase, Entering into full-scale development without establishing design feasibility can result in at- tempts to achieve unrealistic technical progress within a specific test and schedule plan. Positive identification of these design risk areas will permit the program manager to facilitate the system development process by bringing his resources of men and money to bear upon critical ele- ments and streamline the development schedule. Following are some examples of instances where criteria . , for obtaining assurance of feasibility of performance re- quirements were adequately applied and some where they were . not. A. Adequate application of criteria 1. AEGIS missile system In the cases of the AEGIS missile system, a group of highly qualified people from the Navy and industry performed a risk analysis as part of a comprehensive missile system study. In evaluating results, the Navy directed a laboratory model to be constructed and tested to demonstrate the feasibil- ity of high-risk components before proceeding fur- ther with development. Added assurance that the system was technically feasible was obtained through an independent evaluation. The successful demonstration of the highest technical risk component has been established as . . the first critical milestone in the current engi- neering development contract. 24 2. Heavy Lift Helicopter (HLH) The HLH is planned to lift heavy loads over short distances in support of combat missions and peacetime operations. The HLH is planned as an im- provement in lift capability over present transport and flying crane aircraft, and has no counterpart in the current Department of Defense inventroy. The development approach for the HLH differs from many major system acquisitions in that the early phase of the acquisition process includes de- velopment of critical hardware in contrast to paper studies. The Army has identified high-risk components for immediate development effort. If the critical component development is a success, full-scale de- velopment can proceed. 3. F-14 Aircraft The F-14 aircraft is composed of three basic subsystems; namely, avionics, propulsion, and air- frame. The potential risks in developing each sub- system were studied and analyzed by the Navy before proposals were received from interested bidders. The Navy analysis indicated that the risks associ- ated with the avionics and propulsion subsystems were low because these subsystems had been devel- oped for use on another aircraft. For example, the engine to be used on the F-14 was available and was tested on the ground in a simulated F-14. The airframe was considered a normal develop- ment risk although various potential problems were identified. Plans were developed to resolve poten- tial problems including identification of possible backup items which could be used to provide an in- terim capability, if required. The Navy also used risk analysis in consider- ing the reasonableness of the contractors' propos- als. Therefore, when the Navy entered the 25 development and production contract, risk had been minimized. Identification of program risk also en- abled the project manager to more adequately moni- tor development of the airframe. 4, Airborne Warning and Control System (AWACS) In the AWACS,the high-risk area was identified as the overland radar, the extent of which will be determined by building and testing actual hardware. Actual demonstration that the AWACS,including the overland radar subsystem, will work as intended is stipulated as a condition of continuing develop- ment. Two competing overland radar systems will be developed, and a fly-off <ompetition with AWACS configured aircraft held, to demonstrate their re- spective merits and detect shortcomings. If a suc- cessful system is demonstrated, the AWACSprogram . will be allowed to begin the remainder of the full- scale development program. B. Inadequate application of criteria 1. DRAGON The DRAGON weapon system was approved for full- scale development before essential technology was available to correct major system limitations. At that-time, the development of a required night sight was not believed to be within the state of the art. Other technical risks (i,e., friendly electronic interference and enemy countermeasures> had not been assessed as thoroughly as was pos- sible. Several DODreview groups, while acknowledging _ major system difficulties and performance limita- tions, recommended an accelerated development schedule for limited production of the system. . - Operational need was stated as a basis for these recommendations. 26 Demonstrated DRAGON performance has not met requirements contained in the Army’s Qualitative Materiel Requirement. This problem comes from failure to properly assess high technical risks, and from granting approval to proceed prior to the resolution of risks. This is contrary to DOD rules. 2. Short Range Attack Missile (SRAM) The feasibility of developing the motor re- quired for SlUM was not adequately considered be- fore commencementof full-scale development. The missile motor represented a high-risk area that neither the Air Force nor its contractors ad,e- quately evaluated during the contract definition phase. After award of the full-scale development con- tract, the contractors concluded that the rocket motor required to meet design and performance con- tract specifications was beyond the state of the art. The Air Force now estimates that the rocket motor planned for production will have a total im- pulse less than expected at the time of the devel- opment contract award. Performance was thus com- promised. Development of the rocket motor delayed completion of system development several years and raised costs as well. 3. C-5A aircraft Similar to the SRAMprogram experience, the C-5A program encountered technical difficulties which were appraised but which may not have been adequately recognized at higher levels during the validation process. These technical problems proved difficult and costly to resolve and caused cost growth and schedule slippages. In the case of the C-5A, the development schedule also was unrealistic. At the outset it was overly optimistic with no allowance for set- backs in the development program. 27 Cost-effectiveness determinations Cost-effectiveness studies are one accepted means of selecting a system. They are particularly useful during concept formulation of a weapon program. Systems selected for consideration should include equipment already in inven- tory and should specify the degree to which such systems provide needed mission capability. A cost-effectiveness study considers the need that a system is supposed to fill, the alternative technical solu- tions that are available to meet that need, technical per- formance characteristics of each alternative, cost associ- ated with each possible solution, and criteria for choosing among alternatives. The overall study should emphasize sig- nificant issues to clarify merits of alternative systems. Also, the analysis should be updated when changes in basic assumptions occur. Updating ensures continuing cost effec- tiveness of the system selected by allowing for changes in threat, technological advancement, or desired level of de- fense. GAO's examination was limited to the questions whether (1) the military service had fulfilled a requirement that cost-effectiveness studies be performed,(2) studies had been made of competing equipment systems, (3) each study was evaluated, used, and became part of the competing weapon program records, and (4) realistic equipment operating envi- ronments and personnel training levels were included as conditions for performance of the equipment end-item. Cost-effectiveness studies provide a measure for eval- uating changes as the program proceeds and for making con- tinuing trade-offs between cost and performance. With such studies, we have a technique by which balance can be main- tained between cost and performance. Without such studies, , ill-advised program decisions affecting performance and schedules can seriously jeopardize program cost estimates. Following are some examples of instances where criteria for performing cost-effectiveness studies were adequately applied and some where they were not. 28 A. Adequate application of criteria 1. DD-963 Ihe program for development and production of new destroyers (DD-963) to replace World War II ships was initiated in August 1966. In September 1967, the Office of the Chief of Naval Operations completed a study comparing the cost and antisub- marine warfare effectiveness of the DD-963 class destroyer with alternatives. Formal approval to enter contract definition for the DD-963 was granted in February 1968. The scope of the DD-963 cost-effectiveness study included a comparison of existing, modernized, and new design destroyers. Results of the DD-963 study showed that the DD-963 could provide antisub- marine warfare effectiveness with substantially. fewer ships at a lower life-cycle cost, and at ap- proximately the same total investment cost as any alternative ship. The DD-963 study relied heavily upon previous studies for such things as postulation of threat and estimate of differences in effectiveness among the various antisubmarine warfare components used on the competing ships. Since our review did not encompass earlier studies, we did not determine rea- sonableness of the assumptions used in the DD-963 study regarding these or other significant aspects of the cost-effectiveness question. Nevertheless, our limited review has shown that the Navy (1) pre- pared the DD-963, cost-effectiveness study early in the acquisition process,(Z) considered a number of alternative systems in DOD's inventory,and (3) ap- parently selected the most cost-effective alter- native. 2. AEGIS A cost-effectiveness study of the AEGIS Ad- vanced Surface Missile System was made in early 1965. The principal characteristics of the missile system reconrmendedfor development at that time were es- 29 sentially the same as those currently approved for development of what is now called the AEGIS system. In that study9 comparisons were made of the perfor- mance of various individual systems in a wide vari- ety of tactical situations and of alternative com- binations of systems providing for the defense of specific naval forces. Alternative systems eval- uated included existing Navy missile systems, air- craft equipped with air-to-air missiles, and several versions of the AEGIS system,, Costs of alternatives considered included devel- opment, investment, and annual operating costs, The cost and effectiveness of alternatives were compared, The conclusion showed the AEGIS system to be supe- rior in cost effectiveness to the alternatives. Although a formal updated cost-effectiveness study was not prepared, the Development Concept Paper sub- mitted to the Defense Systems Acquisition Review Council following receipt and evaluation of contrac- tors? proposals, compared the cost and effectiveness of alternate systems against various threats. Although we did not question validity of basic assumptions, we believe methods used in this study conform to acceptable cost-effectiveness-study prac- tices. 3. Armored Reconnaissance Scout Vehicle (SCaVT> A cost-effectiveness determination was made by comparing threat, mission, and effectiveness analyses with schedule, cost, and feasibility studies. By providing seven different firms with such data as scope of work, description of the system, vehicle design, etc. 9 various concept designs were sub2 mitted. These designs were consolidated with in- house effort, and the results were furnished to a research firm. In addition, Army research organi- zations supplied auxiliary data on threat analysis. In the assessment of design, cost, and combat effec- tiveness (parametric design/cost-effectiveness study), the research firm analyzed (1) effectiveness evaluations of nine concept vehicles and eight 30 reference vehicles in computer simulations of repre- sentative missions and (2) life-cycle costs of each of the 17 candidate systems. The comparisons of effectiveness required evaluation of the candidate vehicles in different threat and geographical envi- ronments as well as in the performance of two dif- ferent types of mission--security and reconnais- sance. B. Inadequate application of criteria- 1. A-X aircraft The Air Force cost-effectiveness studies for the A-X aircraft considered only the A-W, A-7D, A-37B, F-4C/D, and the improved OV-10. These are all Air Force fixed-wing type aircraft. Such pos- sible candidates as the Army's AH-56 Cheyenne he- licopter, the Marine's AH-LJ Cobra helicopter, and the Marine's Harrier-- a vertical/short takeoff and landing aircraft-- -were not covered in the studies. Also excluded is a more expensive version of the A-X which incorporates an all-weather capability. 2. A-7D aircraft The Air Force recommended, on the basis of cost-effectiveness studies of existing DOD-wide com- peting systems, the procurement of a slightly mod- ified version of the Navy's A-7 in-production air- craft to fulfill its need for close ground support missions and interdiction in future years. Subsequent to these studies and DOD's approval of the procurement, major configuration changes in avionics invalidated initial plans and contributed to the Air Force procurement of a more sophisticated and expensive aircraft. The average unit weapon system cost increased about 110 percent between DOD's approval in November 1965 and June 30, 1970. These changes also contributed to delay in estab- lishing firm detail specificationsand attaining delivery schedules. We were informed by Air Force personnel that the cost effectiveness of the A-7D 31 was not revalidated to determine if the aircraft was still cost effective after these changes were made when compared with competing existing systems. The principal management weakness in adminis- tering this program was failure to give formal rec- I ognition in the management process to the effect of these changes. Such recognition would have subjected the revised A-7D plan to the same basic decision- making process as the initial plan, including a val- uable cost-effectiveness comparison of the changed A-7D configuration with other DOD systems. 3. A-7E aircraft Similarly, the Navy A-7 aircraft program began with the Navy A-7A version, which was developed to I fulfill the requirement for light attack aircraft with increased range and load carrying capability to replace the all-weather A-4E. This A-7A was sub- jected'to the full-scale development cycle. The next Navy version, the A-7B, was basically the same asthe A-7A except for a different engine which pro- .vided increased acceleration and decreased takeoff y digtance. In developing the "E" version, the Navy started with its existing "B" version and developed a sig- nificantly improved light attack aircraft. The Specific Operational Requirement &OR) for this air- craft has not been changed, However, the improved avionics system and engine of the A-7E represent significant advances in the military capabilities over the A-?B, one being increased bombing accuracy. Like the A-7D above, the cost effectiveness of the ultimate A-7E configuration was not validated. 32 Stability of the program and its relationship to other programs Effective pursuit of program objectives requires stabit- ity of priorit 2nd of aZZo,cation of all. critical resources in combination with eZarity and consistency of program diree- tion. The discipline imposed by OSD and the service secre- taries upon the military services' weapon acquisition organi- zations has helped to bring about a more orderly management process. The rather long period of time required for acqui- sition has been broken down into logical stages. Comprehen- sive criteria have been established for an acceptable pro- posal for a program to advance from one stage such as the conceptual phase to the next. Detailed OSD direction has been given to the military services on all aspects of pro- curement, such as 'bake-or-buy," national priorities and de- fense materials systems, and small business set-aside. The rigorous structuring and close management control mentioned above do not address the question of need for, and priority of, a specific weapon acquisition program relative to others. That process of determination and execution of the relative need/priority is accomplished principally through the formulation of budget and is reconsidered annu- ally for each weapon program, in each appropriation in- volved, with consequent instability permeating all program direction. The impact of instability is illustrated by the reduc- tion in capability experienced by the Defense Satellite Com- munications System and its associated earth terminals. In another USAF mission area, the bomber air defense system, in which provision was made to systematically develop, pro- cure, and deploy the system components in a preplanned, well- organized manner, the principal components are the OTH-B radar, AWACS, F-15 air superiority fighter, or an as yet un- determined interceptor fighter. For two system components (OTH-B & AWACS), the initial operational capability (IOC) dates do not coincide and the full operational capability dates bear no relation to one another. An overa aequisi- tion management plan, with provisions for integration and coordination of mutuaZZy interdependent weapons required for mission performance, and an interrelated air defense testing 33 program to evaluate accompZishment of continenta air de- fense wouZd ame Ziorate this condition. The establishment of a comprehensive priority system for weapon acquisition programs is an involved process. This is particularly true for weapon systems which fall out- side the category of "Strategic 1A" programs, (generally "super-systems" such as the ABM, POSEIDON, and MINUTEMAN for which high-level attention is readily available). Applica- tion of the ranking to other weapon acquisition programs is even less formal and specific. At present, there is a DOD-wide priority system which allocates certain scarce resources among the competing needs of the individual ongoing acquisition programs. This prior- itz4 ” system is deficient in two respects; it is not uniformZy appZied within each of the servi.ees (although it is reason- a5Z2 ~21’: ctpplied to conflicting needs between programs which are in different services) and it tends to deaZ onZy with certain Zimited categories of resources (such as mate- rials gbich are in short suppzy) and ignores the more crit- icaZ resoxrees such as overaZZ funding and personnez. Within the military departments, some sort of priority ranking system does exist; its value has not yet been proven. We ,h eZieve that the devezopment of a comprehensive DOD-wide priorit system is a first step toward alleviating an impor- tant part of the diffieuzty we found in DOD’s management procedures, Of course, an indisputable priority is estab- lished weapon system by weapon system through the annual bud- get review cycle. These budget-derived priorities, however, are not converted into a DOD-wide comprehensive priority rating which would also determine each program's relative priority for all critical resources. Also, insofar as we can see, theras no effective connection between these bud- get decisions and some longer range view which contrasts each potential acquisition against a master plan of overall mission requirements and available or developing capabili- ties of all the services. The Office of the Secretary of Defense has recently im- . plemented a new approach to analyzing the plans for a weapon in terms of the relevant military mission category such.as . "land warfare .'I This analysis, which includes identification 34 of major issues, is to provide the Secretary of Defense with a broad overview of each mission category. It also is in- tended to provide guidance for weapon acquisition to military departments and agencies which develop the programs for equipment to improve military effectiveness. Additional ob- jectives of the procedure are to eliminate competing systems, phase out obsolete equipment, identify deficiencies in ca- pabilities of the forces, establish performance character- istics needed, and set schedules for carrying out guidance. It is expected that the analytical procedure will raise and resolve major issues inherent in and between mission cate- gories. Although this procedure appears to satisfy many of the essentials of an overall priority system, it is still in its infancy. 35 Subsystem development phasing and interfacing The constituent subsystems of a weapon system must be available and compatible or system development will not be successful, When a weapon program includes development of a sub- system with high technical risk, the weapon program is sus- ceptible to slippage. When the subsystem development is out of phase with the development of the overall system, that system may be compromised in either schedule or per- formance$ or both, The mismatch of subsystems with the parent system ap- pears to occur most frequently when responsibility for de- velopment of parts of the system.is divided among two or more project managers. The difficulty is compounded when a subsystem is common to more than one weapon system yet sep- arately managed. Specific provision must be made to ensure that develop- ment and acquisition of the subsystem will coincide with technical requirements of each of the weapon systems for which it is to be used, The same considerations of phasing and interfacing are applicable to a weapon system such as SRAMwhich must work in conjunction with systems such as the B-52, B-l, and F'B-111 bomber aircraft. The increasing complexity of weapon systems has neces- sitated increasingly detailed, close control over design, development,. and production of the system by the program manager. He must give informed technical and administrative direction to ensure that proper provision is made for con- trol of development phasing and interfacing, He must re- quire performing organizations to (1) identify and document the functional and physical characteristics of the weapon system and its subsystems, (2) rigorously control changes to those characteristics, and (3) record and report all per- tinent aspects of the progress of system components and any changes to them, This quality of direction and control by the program manager is necessary to achieve integrity and continuity of design for technical performance, producibil- ity, operability, and supportability of the overall system, 36 To reiterate, developing subsystems must be kept in phase with one another to make sure they will work together and will be available when needed, or cost growth and schedule slippage will generally occur. Imbalance in de- velopment of subsystems can also cause shortfalls from performance objectives for the weapon system; that is, de- lay in the achievement of, or incompatibility among, con- stituent subsystems of the weapon, or related weapons, may impair performance of its mission. Following are some examples of instances where criteria were adequately applied and some where they were not ade- quately applied in subsystem development phasing and inter- facing. A, Adequate application of criteria F-15/B-l The F-15 and B-l programs incorporate manage- ment concepts intended to guard against or minimize the effects of pitfalls which have been encountered in other major acquisition programs, through use of total system responsibility and demonstration mile- stone provisions. A "total system performance re- sponsibility" clause has been incorporated in the F-15 contract, which makes the airframe contractor responsible for integration of the complete weapon system as well as for all actions necessary to en- sure that the total weapon system will meet perfor- mance requirements set forth in the system specifi- cation. In essence, the Government looks only to the airframe contractor for satisfactory perfor- mance of the aircraft and does not become involved in any problems concerning the engine or the sub- systems. Contractor-to-contractor relationships necessary to fulfill interface plan commitments are set forth in associate contractor agreements be- tween the prime contractor and his associate con- tractors. A similar approach has been incorporated into the B-l program wherein the airframe and en- gine contractors are working on an associate con- tractor basis, but the airframe contractor has total system integration responsibility. 37 Under the demonstration milestone provision, planned dates for accomplishment of specified technical milestones are established. The Air Force will determine whether the contractor has satisfactorily accomplished the milestones. The accomplishment of the milestone is contractually tied in with Government allocation of production funds. Failure to meet a milestone may result in a delay in the funding of a production increment, a delay in exercise of the option to which the dem- onstration milestone relates, or a partial allot- ment to sustain minimum production at the Gover- mentqs option. Any schedule adjustments due to de- lays will be made with no change in initial target cost or ceiling price. None of the milestones were scheduled to be accomplished at the time our review was completed. B. Inadequate application of criteria 1. CHAPARRAL/VULCAN The CHAPARRAL/VULCANair defense system was produced and deployed without the Forward Area Alerting Radar System (FMRS) which, coupled with other significant performance limitations, resulted in the system!s providing limited air defense capa- bility. When limited production of the CHAPARRALand VULCAN systems was approved in November 1965 and March 1966, respectively, the Army had not designed or developed the military characteristics for the system's radars even though it had determined in 1965 that existing systems could not be modified to fulfill the radar's mission. Production of the ra- dar was authorized in 1968, though earlier testing indicated that it did not meet performance repuire- ments. When technical difficulties arose, radar production was stopped in July 1969, This resulted in the deployment of the CHAPARRAL/VULCANsystem without FAARS. The present system requires the.op- eratorss visual detection and identification of en&my aircraft and his judgment that they are within ' range. 38 2. AN/BQQ-2 integrated sonar system The AN/BQQ-2 integrated sonar system is a com- plex system designed for installation and use aboard nuclear attack submarines. The accomplish- ment of the sonar and submarine projects is the responsibility of different project managers in the Naval Ships System Command, Successful accomplish- ment entails integrating the two systems at a pre- determined point in time. Performance and physical characteristics of the two systems had been identified; but develop- ment and production schedules for the two systems were out of balance. The sonar was acquired under an accelerated program to permit delivery at the predetermined time that a ship would be ready to accept it. This precluded an orderly design, de- velopment, and production of the sonar system and resulted in technical problems. Technical problems delayed delivery of the sonar system. The sonar delay resulted in a disruption of Navy shipbuilding schedules and in cost growth. The problem experienced with the sonar system development phasing and its ultimate interfacing with submarines was magnified because each weapon system had its own project management. 1 The Navy has now established a ship project directive system which provides ship acquisition project managers with procedures for directing man- agement actions of secondary managers to ensure proper integration of the total shipbuilding pro- gram. Thus, definitive tasking, scheduling, and funding for all support elements is effected. This policy should help significantly to prevent the situation described in the AN/BQQ-2 sonar example. 3. P-3C aircraft Development interface and subsystem phasing problems were encountered in the P-3C program 39 because the technical feasibility of certain sub- systems planned for the program had not been fully proven when the development program was approved. For instance, a succession of three different ver- sions of the acoustic signal processor has been attempted since program approval. The last of these, the one now included in the program, is known as DIFAR. The decision to incorporate the DIFAR processor gave additional capability to the P-3C, but it made the problem of interfacing and phasing of development,more difficult because the processor was still under development and was not available until about a year after P-3C deliveries began. The result has been a stretch-out in P-3C testing and the delivery to the fleet of aircraft short of desired equipment. This equipment had to be backfitted as processor production caught up with need. 40 Continuous trade-off between cost and performance As early as preparation of a design is completed for the weapon system, the program manager should initiate the iterative process of examining each proposed change in capa- bility for the weapon against its associated costs. His analysis should include estimates of technical feasibility of the design features of the proposed change, probable im- pact on the logistics and schedule, and cost of the capabil- ity in relation to military need. A continuous trade-off between performance and cost during the acquisition process will keep aZZ elements in balance. Flexibility during development is important. The Dep- uty Secretary of Defense has stated that The cost of developing and acquiring new weapon systems is more dependent upon making practical trade-offs between the stated operating require- ments and engineering design than upon any other factor." He has stated further that "trade-offs must be considered not only at the beginning of the program but continually through- out the development stage." Budget constraints have forced trade-offs of the nature de- scribed and have ensured continuing implementation of this philosophy. Following is an example of an instance where criteria for trade-offs between cost and performance were adequately applied. A. Adequate application of criteria F-15 aircraft The Deputy Secretary of Defense advised the Air Force in September 1969 that production funds for the F-15 program would be limited and that ac- ceptable performance cost trade-offs must be 41 determined so that F-15 costs would be within the approved program. The Air Force began a cost reduc- tion study based on assumptions that the F-15 engine and airframe would be unchanged and that the devel- opment phase would begin as originally scheduled. The study produced savings which brought the program within funding constraints while retaining an ac- ceptable operational and growth capability. The largest cost reductions were realized in avionics. Such changes as reducing range and ground map re- quirements for the radar, reducing redundancy in computation, reducing communications and navigation requirements and equipment, and a decrease in the amount of initial spares were effected. The esti- mated unit cost of production was reduced about $1.5 million per aircraft. Provisions were retained in the aircraft to permit the reinstatement of some hardware items at a later date, if feasible. Additional trade-off studies of the F-15 have been made to meet Air Force heeds more economically since the award of the development contract. The Air Force plans to continue its review of the F-15 program throughout the development phase for pos- sible reductions in cost, weight, and complexity. 42 TECHNICAL ASSESSMENT One of the results we observed of DOD's efforts to im- prove its weapon system acquisition process was the in- creased use of test resul,ts to anticipate specific techni- cal difficulties. The conduct of specified tests and use of their re- sults under current management concepts are incorporated into recent acquisitions programs, such as the F-15 and B-l aircraft programs. Clarification of assessment of techni- cal development is part of the implementation of total sys- tem responsibility; of milestone demonstration; and of thresholds for cost, schedule, and technical performance. Technical feasibility studies pinpoint technical high- risk areas. Special emphasis is now being given to mini- mizing these risks, and special testing is used to monitor planned progress. Tests are a valuable means of assessing subsystems and system design progress. Test results also provide a com- parison of actual progress with the planned progress. Test results provide management with information on which to base decisions such as to modify a design approach or to change*basic system development plans. The results of successful tests also can be used to curtail design ef- forts when sufficient confidence is gained to support a de- cision to proceed with production or to accept hardware for operational use. With inadequate data from test results, judgments of this kind become more subjective and suscep- tible to a greater degree of error. Omission of tests can result in the production of hardware that does not meet re- quirements. Following are some examples of instances where criteria for technical assessment were adequately applied and some where they were not. 43 A. Adequate application of criteria 1. DRAGONmissile The Army's technical development test plan for the DRAGONmissile has demonstrated that DRAGON's performance has not met established requirements for reliability and single-shot lcill probability. Additional technical problems were also revealed through testing. Test results that .the Army is using may influ- ence plans to let a limited production contract prior to completion of all service tests, 2. Improved HAWKmissile High-risk areas in technical objectives of this missile, for which extraordinary management ' . action was required, were identified in November 1968. At that time the Improved HAWKsystem en- tered an engineering test/service test program. The test program was scheduled to continue through . 1971, Flight tests were halted in December 1969 because a component failed to function properly, This component was modified and additional test ob- jectives were prescribed, An 18-month development program was instituted to develop another camp.onent as an alternative. In early 1970, flight testing was resumed but only limited success was obtained in meeting the objectives, Conclusive data have not been obtained on flights against low-altitude targets,maneuveringtargets, high-speed targets, long-range targets, and electronic countermeasure environments. To decide whether a production contract should be awarded for FY 1970 and FY 1971, the project manager had a risk analysis performed. Completed in April 1970, the analysis included an evaluation of technical, cost, and schedule data on the Im- proved HAWK system. A component was assessed as a high-risk item. To minimize this rislc, modifica- . tions to the component were proposed. After 44 evaluation of these modifications, other perfor- mance risks, and increased costs that would be in- curred by delaying procurement, the project manager recommended the immediate award of a production con- tract. 3. MAVERICK missile Production options included in the contract for MAVERICK development were to be exercised before scheduled completion of tests conducted by the con- tractor. Additional provisions afforded the Air Force opportunity to delay exercising production options for 420 days, upon payment of stipulated standby costs. This option period extends through the scheduled completion of contractor testing and almost to the midpoint of military service testing. The Air Force decision to use this option period, and thereby delay commencement of production until a substantial portion of Air Force-controlled dem- onstration test results are known, indicates that it is moving toward the DOD position of "fly before you buy" and is gathering more test data before committing a weapon system to production. B. Inadequate application of criteria . 1. SRAM Some degree of subjective evaluation must often be exercised in evaluating test results and that fact must be made clear to decisionmakers. This has not been the case in the SRAM program. Major milestone decisions which involve advanc- ing an acquisition program to its next phase must be based on broad information about actual accom- plishments as compared with planned accomplish- ments. Test programs are devised to provide that information. The SRAM flight test program has fallen somewhat below the ideal in that extrapola- tions of test results have been used. 45 Extrapolations of the SRAM flight test results include adjustments for conditions, such as the in- terim rocket motor, atmosphere, winds, launch alti- tude, launch speed, and missile weight. Adjusted test data based on technical extrapolations, engi- neering assumptions, and various other adjustments for simulation and probability analysis do not con- clusively demonstrate SRAM/carrier aircraft actual capabilities. Such test results demonstrate only calculated capabi Zities. Test data based on accom- plishment more closely resembling actual mission conditions would provide a reliable gauge for top management to judge the performance and progress of a testing program. 2. AN/BQQ-2 sonar system When planned test programs are abrogated, even for a good reason such as‘an overriding urgency to deploy, the effectiveness of the product is compro- mised. For example, the attack submarine program required that the AN/BQQ-2 sonar system components be installed at a specified time during a submarine shipbuilding schedule, The current version of the AN/BQQ-2 sonar sys- tem was designed to provide a given improvement in reliability and a larger improvement in maintain- ability over a prior version of the sonar system. Reliability and maintainability demonstration tests were not conducted on the first few production sys- tems. The Navy followed this course of action be- cause it felt the chances for success to be good since the current system is a follow-on to previ- ously designed and tested systems. The first pro- duction system was delivered before the production acceptance test was completed, The schedule demands of the shipbuilding pro- gram for the nuclear attack submarines required delivery of the sonar systems before complete test- ing to preclude delaying the ship construction pro- gram. However, this should not be a justification for skipping the required testing. 46 The system delivered required changes to meet requirements. It is easy to see that program de- cisions which ignore test results are apt to esca- late costs. 3. M60 tank The M6OAlE2 is a modified version of the M6OAl tank--currently the Army's standard battle tank. The E2 version was to have had a redesigned turret incorporating the SHILLELAGHweapon system. The SHILLELAGHwas already under development; and, in this case, the objective was to adapt it for use on the M6OAl tank and to provide, at an early date, a tank having a missile-firing capability. Devel- opment of the M60AlE2 began in 1964; although early testing of prototypes had disclosed major deficien- cies, the Army in 1966 authorized full-scale pro- duction of the tank before sufficient testing had been accomplished to validate the design, despite advice of qualified testing and user agencies. Technical difficulties occurred during produc- tion which should have been detected in the test- ing program. The technical problem which caused the greatest concern and prevented deployment of the tank was the inability to stabilize the turret. This was a basic design fault that caused the tank's gun to move erratically, making it extremely difficult to deliver effective firepower. Prema- ture decision to enter production brought delivery of 300 tanks and 243 turrets and components for which extensive modification is needed to satisfy the user's requirements. 47 ORGANIZATION FOR PROGRAMMANAGEMENT Managing the acquisition of complex weapon systems has evolved into one of the principal activities of the military services. It is quite different from other procurement and receives special attention in the military serviceso Weapon system management is the process of planning, organizing, coordinating, evaluating, controlling, and directing con- tractors and participating organizations to accomplish sys- tem program objectives. The program management approach to weapon acquisition is a distinct departure from the services' traditional method of establishing functionally oriented organizations to carry out well-defined, repetitive or continuous, long-term tasks. This approach requires the program manager to establish management arrangements among his organizations, other mili- tary organizations, and various contractors to efficiently coordinate their efforts to accomplish program objectives, A variety of program management organizations have been established. They range from a large, self-sufficient of- . fice to an austerely staffed focal point which operates on the matrix principle and which must draw all specialized support from the functional organization to which it is at- tached. These are illustrated in figure III. The self-sufficient program office is organized and structured to operate by itself without having to rely on functional organizations for technical and administrative support, Conversely, the program office operating on the matrix principle relies on functional organizations to per- form such tasks as research, development, logistics plan- ning, procurement, inspection, and supply and maintenance. There are advantages and disadvantages associated with both the self-sufficient program office organization and the functionally oriented (matrix) organization. The advantages of one organizational structure tend to be the disadvantage of the other and vice versa; e.geo a matrix organization fosters greater specialization with less technical duplica- tion but makes coordination and communication more difficult. A self-sufficient program structure fosters coordination and communication but makes specialization more difficult, and some technical duplication becomes inevitable,, 48 FUNCTlONAt MANAGEMENT ORGANIZATION NO PROGRAM OFFICE ESTABLISHED INCLUDES PERMANENT AND CONTINWNG SPECIALIST STAFFS PROGRAM A PROGRAM B PROGRAM MANAGEMENT ORGANIZATION PROGRAM OFFICE ESTABLISHED PROGRAM OFFICE STAFF ASSIGNED FOR THE DURATION OF THE ACQUISITION INCLUDES SPECIALIST STAFFS REQUIRED FOR PROGRAM ACCOMPLISHMENT MdTRlX ORGANIZATION INTERFACING OF PROGRAM AND FUNCTIONAL MANAGEMENT ORGANIZATION MANAGEMENT OF CORRELATED ACTIVITIES OF FUNCTIONS AND PROGRAMS FUNCTIONAL ORGANIZATIONS I II III ) PROGRAM A PROGRAM A PROGRAM A TASK TASK TASK PROGRAM B PROGRAM B PROGRAM B TASK TASK TASK FZCGUFSJ.I.1 . 49 In large part, the Air Force acquisition programs are in self-sufficient organizations, while the Navy projects are matrix oriented; and Army projects are organized some- where between the other two. Under its matrix concept, the Navy has provided only 14 people for its F-14 project manager's organization. Another 92 people are assigned to the functional organiza- tions within the Naval Air Systems Command. They are iden- tified with the F-14 program but they do not work directly for the project manager. Under this arrangement, there is need for considerable coordination between the organizations. The functional personnel associated with the F-14 program may or may not work exclusively on this program. Conflicts for their time must be negotiated on a case-by-case basis. We are informed that priority of work assignments on the F-14 project has not been a problem, but the potential for trouble obviously exists. In contrast, the Air Force F-15 program manager, with - . 243 staff members, essentially has a self-contained organi- zation, All the functions necessary to manage the develop- - ment program are manned by personnel directly responsible to the program manager, and work assignment priority can be handled by him. 50 Organizational "layering" One of the most troublesome features of the present program management structure is difficulty in obtaining de- cisions. It seems to us that the most likely cause of this problem is that decisionmaking layering is not commensurate with organizational layering. In general, the military ser- vices have not deemed it wise to place the project manager high in the organization because of some practical considera- tions, such as the large number of project managers and the need for them to work directly at lower levels of the orga- nizations. However, the effect has been to preserve levels of review authority which do not have clear roles in the process of formulating decisions. Most of the decisions that the project manager does not make himself are made at the highest levels of the service or by OSD. Between the project manager and top management are a large group of organizational units whose commanders attempt to keep themselves informed about a particular weapon system and study and deliberate on pending programs to recommend some course of action. As a rule, they have no direct approval powers. They can delay or stop a project but cannot make decisions to proceed, change direction, pro- vide money, or take other positive action. Military service organizations for weapon system acqui- sition are shown in the simplified charts on pages 53, 54 and 55. These charts do not show the many subdivisions that become involved or the special ad hoc panels and committees which inevitably arise in the weapon system acquisition pro- cess e All these organizational units, panels, and commit- tees impact heavily on the project manager. His program may be delayed or stopped while matters are being studied or while decisions are being made, or his program may proceed without timely decisions. In the Army, for instance, any significant decision that the project manager cannot make usually is made at the highest levels of the Department or in OSD. With respect to . these decisions, the primary role of the project manager is to make recommendations or to work with other groups that make recommendations. Recommendations go through the normal 51 chain of command; i.e., the Commanding General of the Com- modity Command, to the Commanding General of AMC, to the Army staff. To formulate recommendations though, it is necessary to coordinate a number of functional groups. These include functional groups within the project managers' organizations ( i.e., the Commodity Command) as well as organizations out- side the Command, such as Conarc and CDC. The essential task of these groups is to help formulate a recommendation, but their decisionmaking function is limited to agreeing or disagreeing with it. Once the recommendation is made, there are a number of functiona groups at the AMC and DA staff Zevels (about a dozen at DA staff alone) who can influence the decision. The contribution of all these groups iS much the same. They can either agree or disagree with the ree- ommendation made. The inevitable result of this process is the scheduling of repetitive meetings, briefings, and studies in an attempt to reach agreement on the recommendation to be made. sup- plying information to numerous groups can be almost a full- . time job for the project manager. During 1969, one project . manager spent about two thi-ds of his time conducting 166 briefings and from January to August 1970 participated in 62 *. additional briefings. From January 1969 through July 1970, another project manager participated in 124 briefings. Many of the briefings involved levels below the top head- quarters' staff, but the most important function of those participating was to recommend. In another instance of extensive layering, several re- views of a program were conducted between September 1969 and April 1970, including an in-depth review by several boards and committees at all levels. Of partieutar importance was the requirement that briefings for decisionmaking groups be previewed as many as 20 tG 30 times before presentation tG an action-taking body. The project manager spent a large part of his time participating in these reviews. 52 ARMY ORGANIZATION FOR ACQUlSIflON 1 i 3ARMY STAFF MY : :I .:I :I : i i i i i i I COMMAND I : : i II+ i : : I i : : : i : : i I I : : i : : i 5 : - Line of Authority i : : z wowed Staff Communication and Coordination Line i : : : I mII I I Expeditious Information and : Decision Line IIIIIlIIIIIlIIIBIIlIII~~IIIIIIsIII~ : FIGURE IV NAVY ORGANIZATION FOR ACQUISITION ~In1~11I1111a111111I~I~I SECRETARYOFTKENAVY - Line of Authority OFFICE OF THE CHIEF OF NAVAL OPERATIONS II~IIIIIIIW Staff Communication and Coordination Line n I I I II Expeditious Information and Decision Line 1111m1111111111~1111111 DEPUTY COMMANDER ASSISTANT COMMANDER FIGURE V AIR FORCE ORGANIZATION FOR ACQUISITION SECRETARY OF THE AIR FORCE CHIEF OF STAFF Y: : :: I E a I :: I III-1 I :: SYSTEMSCOMMAND i i L STAFF -4 i * . : : i : : : i I 1 : DIVISION COMMANDER I 1 i i i i ! ,z SYSTEM PROGRAM ‘- Line of Authority I 8JBJJJJlJJ Staff Communication and SYSTEM PROGRAM ‘Coordination Line OFFICE Expeditious Informal:ion and ‘p 1 z Decision Line : SCCRJ~JJ~JJJJJJJJUJJJJJJJJJJJJJJ JJJJJJJJIJJJJJJ~JU FIGUREVI DOD-proposed action on acquisition management problems The Deputy Secretary of Defense frequently has cited many of the problems in the organization and procedures for managing weapon system programs. He has stated that: 1. Program managers must be given authority to make decisions on major questions relating to the pro- gram, both in the conceptual phase and in the full- scale development phase. 2. Program managers must be given more recognition op- portunity for career advancement in all the services, and good managers must be rewarded just as good op- erational people are rewarded. 3. People in program management must be experts in that business and must be assigned to a given program long enough to become effective. 4. The overall structure of the program management function in all services needs to be appraised. Changes must be made to reduce the numerous layers of authority between the program manager and the service secretary. 56 CHAPTER3 SYSTEMCOSTEXPERIENCE Estimates of probable cost to develop major weapon sys- tems are required at various points in the development cycle. The initial estimate against which all program costs are originally considered is the "planning estimate." The planning estimate is a formal estimate of cost anticipated in acquiring a system in the quantities needed, It is prepared prior to the initiation of the formal acqui- sition cycle and usually serves as a basis for the first appropriation request. The planning estimate is prepared by a military department and is approved by the Secretary of Defense. The planning estimate is followed by an estimate of the cost to develop the system. The "development estimate" is a refinement of the planning estimate and is established during the period in which preliminary design and engineer- ing are verified or accomplished and contract and system management planning are performed. This period frequently extends over a period of one year. A third estimate, the "total cost estimate," is in- tended to be a current objective statement of the cost to be incurred in acquiring the total approved program. This estimate is adjusted for increases or decreases in quanti- ties, as well as for cost changes due to inflation, change in scope, capability in&ease, and program stretch-out. An estimate also is prepared to disclose costs which are related to the maintenance, operation, or improvement of a weapon system rather than its acquisition cost. Ex- amples are replenishment of spare parts, modifications, component improvement, and commonground equipment. Pro- jected operating costs are not included in this latter hind of estimate. Nine of the 70 systems we reviewed had just entered the development process, Their status precluded prepara- tion of precise estimates. A summary of program cost es- timates for the remaining 61 systems is shown in the table below. The estimated cost for these 61 systems increased some $33.4 billion from the cost anticipated by the planning estimate to the current estimate of cost through program completion. About one third of this increase, or $9.5 billion, represented the difference between the planning estimate and the development estimate. The remainder of the in- crease, $23.9 billi on, was due to changes in quantities to be acquired and to a combination of such things as engi- neering changes, revisions to correct estimates, and pro- visions for economic inflation, Cost Estimates as of June 30, 1970 Current estimate Number Cost changes through Total of Planning Development (note a) program cost fjystems estimate estimate Quantity Other completion (note b) ";,;$.; $ ;,;;;."Q $ ;z,;o':.; $ 17,197.6 Air Force (15) -4163214 81220/2 47:418:4 56,335.9 51.896.8 Total (611 $83,633.7 $93,135.6 $3,212.5 $20-697.2 $;17,045.3 $ %he cost changes shown represent the difference between the development estimates and the reported current estimate through program completion. b Includes additional procurement costs, PERCENTAGEOFGROWTH 59 The foregoing chart (figure VII> shows that current estimates through program completion have grown 40 percent in comparison with planning cost estimates for these pro- grams. Cost growth may result from such things as unantici- pated development difficulties, faulty planning, poor man- agement, bad estimating, or deliberate underestimating. However, it is important to recognize, in any analysis or discussion of cost growth, that not all cost growth can reasonably be prevented and that,some cost growth, even though preventable, may be desirable, For instance, un- usual periods of inflation may result in cost growth, Changes in technology may ma'ke it possible to incorporate modifications that result in an overall increase in the ef- fectiveness of the system, Such cpst growth cannot always be anticipated, particularly where a weapon system is in development and production over long periods of time. We stated in our February 6, 1970, report (B-163058) that data were unavailable from which'to make any specific identification of program cost estimate variances, We have suggested that DOD give increased attention to the problem of identifying: 1. Cost growth factors that are not entirely control- lable by DOD, such as inflation, or thos,e factors that may even be desirable and may be expected to continue, such as upgrading sys tern performance. 2. Items 'that are basic causes for cost growth and could be eliminated or reduced considerably by ap- propriate and effective DOD action. DOD has made a good start toward accomplishing the in- tent of our suggestion, Nine categories of cost variance have been established for use in the Selected Acquisition Reporting system (SAR), and program managers have attempted to quantify the impact of cost variances on their programs, Although the precision of these quantifications cannot be . completely verified, segregations being made can now be used to focus attention upon areas where improvements can be made, 60 ANALYSISOF GOSTCHANGES AS OF JUNE 30, 1970 The analysis of cost changes on the 52 weapon systems for which SAR data are available is shown in the table be- low. There has been a net increase in total cost of about $23,980 million. Quantity increases have amounted to about $12,600 million. Decreases in program quantities have amounted to about $10,216 million. Other changes such as engineering, schedule, and economic changes in the 52 weapon programs have amounted to about $21,597 million. Analysis of Cost Changes as of June 30. 1970 Type of cost change Army Navy Air Force Total (millions) Quantity change: Increase $1,371.1 $11,105.5 $ 122.3 $12,598.9 Decrease -3.098.8 -1.760.5 -5,357.l -10.216.4 Net -1.727.7 9,345.0 -5.234.8 2.382.5 Other changes: Engineering changes 489.3 463.8 3,119.4 4,072.5 support II 155.2 -57.7 1,268.5 1,366.0 Schedule II 462.1 1,308.7 844.7 2,615.5 Economic II 550.5 1,156.O 2,307.g 4,014.4 Estimating II 1,312.8 3,356.g 1,509.5 6,179.2 Sundry II -12.7 553.1 544.3 1,084.7 Unidentified II 2.264.9 - 2.264.9 Total 2.957.2 9.045.7 9,594.3 21.597.2 Total $1.229.5 -- $18,39= $4.359.5 $23,97g.7 Number of systems 12 29 11 52 61 QUANTITY CHANGES The approval of phase II of the SAFEGUARDsystem ac- counts for $1,365 million of the $1,371.1 million quantity increase reported by the Army. Three of the Army programs did not reflect any change in the number of units to be acquired. However, seven systems reflected decreases in program costs totaling more than $3 billion due to reduc- tions in the number of units to be acquired. The largest of these decreases involved the SAM-D ($1.8 billion) and the MBT-70 ($600 million). We were informed that many of these reductions were the result of a review by the Department of the Army of its priorities for weapon systems, which was made because of impending budget reductions, and the estab- lishment of the Army's eight highest priority systems. Analysis of the 29 Navy systems for which data were available shows that 10 systems reported no change in quan-. tities; nine systems reported increased costs totaling $11.1 billion (due to an increase in planned procurements), and 10 systems reported decreases totaling $1.8 billion. The largest part of the increase involves three ship pro- grams totaling more than $7 billion. Included in this amount is $1.6 billion for 20(l) additional DD-963's, rais- ing the total for this program from 30 ships to 50. Another large part of this increase comes from two aircraft programs totaling more than $3 billion. The Air Force reported only a relatively small increase in cost due to quantity, mostly related to the SRAM. Two systems, the F,15 and B-l, reported no change in quantity. Seven systems reported reduced costs due to quantity de- creases, totaling $5.4 billion. Of this amount, $4.4 bil- lion involved the F-111, the FB-111, and the C-5A and $600 million involved the AWACS. Instances of reductions in units acquired, in all ser- vices, were offset by increases in other costs for the weapon. Cost growth is obviously a significant reason for reducing the number of units to be acquired in all the services. 1 We were informed in August 1970 that these 20 ships were * not considered a firm program. 62 ENGINEERING CHANGES An alteration in the established physical or functional characteristics of a system is called an engineering change. Incomplete descriptions of initial performance specifica- tions and changes required to bring system performance up to expected standards have resulted in substantial need for engineering changes. Of the $4 billion dollars in engineer- ing changes reported by the three services, about $3,1 bil- lion was accounted for by the Air Force for the F-111, the C=5A, and the MINUTEMAN programs. Engineering changes to- taling $1.8 billion were required to bring the F-111 and C-5A to expected standards, and $730 million involved changes in the MINUTEMAN to upgrade the system to meet an increased threat. SUPPORT CHANGES . Support changes involve such items as spare parts, an- . cillary equipment, warranty provisions, and Government- furnished property/equipment. Relatively small amounts of _ * money were reported in this category for the Army and Navy systems. Support changes in the Air Force amounted to about $1.3 billion and represented an increase in initial spares for theCL5A($230 million) and the F-111 ($258 million). SCHEDULE CHANGES Schedule changes reflect adjustments in the delivery schedule, completion date, or some intermediate milestone of development or production. Cost increases of $2,615 mil- lion were reported as being due to schedule changes. Of this amount, $947 million involved three Navy aircraft pro- grams (EA-GB, P-3C and A-7E); $260 million involved the SPARROW missile; and $747 million involved the F-111. The largest portion of the increase ($460 million) in Army pro- grams is accounted for by the SAFEGUARD, SAM-D, MBT-70 and the LANCE. . For reporting purposes, identifying such schedule ad- 'justment is probably important. GAOfindings indicate that such adjustments are only indicative of other fundamental .problems. Schedule changes, as such, are not a primary cause of cost growth. 63 ECONOMIC CHANGES Economic changes reflect the influence of one or more factors in the econonrry. Included are specific contract changes deriving from economic escalation as well as changes in quantity --changing program estimates to reflect a re- vised economic forecast or changing actual contract quan- tities. We were informed by the Assistant Secretary of Defense (Comptroller) that the treatment of anticipated economic escalation in various reports was neither consistent nor uniform within or between services. To rectify these dis- crepancies, OSD stipulated on June 30, 1970, that the Sep- tember 30, 1970, SAR reports forecasting future price levels were to be based on a table of percentages. We have not evaluated this table, however, we believe that there are no reliable indexes on which to base esti- mates of inflation. ESTIMATE CHANGES Estimate changes in a program or project cost are due to corrections in the initial estimate. The principal estimate change reported on Army systems was $944 million for the SAM-D missile. The Army's justi- fication for this change in estimate was: "**The total estimate is based on analysis of our previous programs, deriving cost estimating relationships based on the actual growth experi- ence of cost estimates for earlier missile pro- grams, at comparable stages of development. Specifically, the estimating techniques anticipate unforeseen . changes in requirements, performance . characteristics, p rogram slippages, funding avail- ability, and quantities produced in specific years. The order of magnitude of those changes actually experienced on previous programs has been used to estimate the magnitude of these costs. While we have calculated the costs based on past 64 experience, we have also taken steps to seek to prevent the causes of cost growth from occurring on the SAM-D Program. As such, if our efforts are successful, the SAM-D will not require the total funds derived from extrapolating the actual experience of earlier programs. **I' Two programs in the Navy account for most of its re- ported changes. The Mark 48 torpedo cost estimate was in- creased $2,500 million to correct a series of underestimates which had been prepared from incomplete data. The new esti- mates projecting production costs were prepared by using the actual prototype costs incurred. The $300 million es- timating change on the Poseidon program corrected a series of overestimates and underestimates--an aggregate of smaller sums. Three programs account for most of the reported esti- . mating changes from the Air Force. The F-111 aircraft pro- gram reported price increases of about $670 million over earlier estimates on the contracts of numerous contractors involved in the program. The SRAM cost estimate was increased $398 million due to underestimation of the costs of development tasks in- volved, while the C-5A aircraft cost estimate was increased $301 million by the Air Force to rectify contractor under- estimates for producing this aircraft. UNIDENTIFIED CAUSES FOR COST CHANGE Summary data showing a cumulative variance analysis and the variance analysis changes since the last reporting pe- riod were either not provided or were incomplete for 15 Navy systems. For this reason, cost changes in Navy systems totaling $2,264.9 million could not be specifically allo- cated. We have been told by the Navy that cost changes will be allocated and shown in the December 31, 1970, SAR. 65 SAR SYSTEM As we reported to the Congress in February 1970, the SAR system represents a valuable management tool for mea- suring and monitoring the progress of major acquisitions, DOD has tried to improve the format, content,and data in the SAR. Although our review of the June 30, 1970, SAR con- firmed that improvements were made during the last year, some improvements still were needed. SAR does not (1) contain a summary statement regarding overall acceptability of the system for part or all of its mission, (2) show the status of major system components be- ing separately developed, nor (3) reflect the current status of program accomplishment. Separate development could re- sult in significant costs if the major system component en- countered development problems that adversely affected the entire weapon system's performance. Waivers of major milestone criteria, with an explana- . tion of the attendant risk therefrom, are not highlighted or ' discussed in the summary section of SAR. 66 CHAPTER 4 SYSTEMSCHEDULE EXPERIENCE 'Our review of the efforts of the military departments to correctly estimate initial delivery dates for about 50 weapon systems indicates that, on the average, the weapon systems experienced 33 percent schedule slippage. Average cost growth of these systems was approximately 30 percent. The following charts show the percentage of schedule slippage by commodity class of weapon systems (figure VIII> and the percentage of cost growth (figure IX>. The schedule percentages were determined by comparing the time originally estimated for reaching the initial op- erational capability date (initial delivery dates of the systems to the military departments) from the beginning of ' the acquisition cycle with the current estimate (as of June 30, 1970) of the same period. 67 SCHEDULE SUPPAGE FR, EtilOriginal estimate of time required to accomplish initial operational copability a Slippage in time required to accomplish initial operational capability 8 i--l I--- 29% 18% 6 ‘MO. of Systems (11) cm (8) (7) (6) in Sample (49) MIS51LES ELECTRONICS Al RCRAFT SHIPS VEHICLES 84 ORDNANCE FIGUREVIII 68 COST ‘GROV;TH f?J 0 n‘g’~na 1 es t-tmote of cost to complete weapon systems cl Current estimate of increase in cost to complete weapon systems 26% 41 33% 20 ................. .............. .............. .............. .............. I.. ........... .............. .............. .............. ............. .............. ............. .............. ............. .............. ............. .............. I.. ........... .............. 15 ............. .............. ............. .............. ............. .............. ............. ............. .............. 113% .............. ............. .............. ............. 10 ............... . . . ........... ! .............. .............. No. of Systems (11) (17) (8) (6) (6) in Somple (48) AiRCRAFT MIWL ES SHIPS ELECTRONKS VEHICLES & ORDNANCE FIGUREIX 69 CHAPTER 5 GENERALOBSERVATIONS, CONCLUSIONS AND RECOMMENDATIONS In the last several months, the Office of the Secretary of Defense and the military services have been engaged in a substantial effort to resolve problems identified as ad- versely affecting the acquisition of major weapon systems. These problems include compromised performance, delayed availability, and increased costs. Generally, the more re- cent weapon programs are characterized by a slower develop- ment pace and more conservative procurement practices than those of earlier periods. Because many of these programs are in early stages of acquisition, physical evidence of the success of changed concepts is not yet available for assess- ment; but the outlook is brighter. Troublesome problems re- main to be solved, particularly in selection of and assign- . ment of priorities to weapons for development and in orga- nizational matters. The statement of the Deputy Secretary of Defense in September 1970 before the Committee on Government Operations, House of Representatives, on organizational and other prob- lems related to new weapon systems development and acquisi- tion, leads us to conclude that he has accurately appraised the problems and the actions needed to resolve them. The ac- tions he proposes are basic, but their implementation will not be easy because they involve changes in traditional con- cepts and management practices that are firmly implanted in DOD. Programs are under way in the military departments to improve the acquisition process. For example, AMC started a comprehensive improvement program on October 1, 1969, called. PROMAP-70. Among this program's 52 objectives are imprqved definition of requirements, analysis of technical risk, up- graded selection criteria, and stabilized tours for officers assigned to project management, as well as improved coordina- tion and conduct of tests. The Army has informed us that results already obtained in this program have shown substan- tial progress in application of these improvements to cur- rent programs. An important consideration in our future 70 reviews will be an assessment of the success of these im- provement programs. General observations on the matters we have studied, conclusions we have drawn from that review, and our recom- mendations, follow. A. Identification of need for and relative priority of individual systems The clear identification of a new weapon's mission is probably the single, most fundamental task that must be com- 'pleted before the development process can begin. Our study of the history of a fairly large sample of weapon systems, however, leads us to conclude that the function of deciding which weapons will be developed is not yet being done with the degree of effectiveness that this important function warrants. Seemingly, the entire structure of the military service . and OSD are involved in this process, in one way or another, and the long and imprecise process of defining and justify- ing and of redefining and rejustifying a weapon system, through many layers of involvement, invariably has delayed decisions and has extended stated availability dates by years. The cumulative effect of the involvement of many dif- ferent organizational units in the decision to justify and then to proceed with development is the root cause of long delays in development decisions. Almost every weapon sys- tem we studied showed some substantial degree of uncertainty as to whether, when, or in what form the weapon should be developed. In addition to clarifying and improving the initial de- cision process (which is now going on in the DOD>, establish- ing a mechanism which defines the priority position of a weapon program in relation to its competitors is equally im- portant. We beZieve that the development of a comprehensive DOD-wide priority system is a first step toward aZZeviating a part of th e difficulty we observed in obtaining weapon systems development decisions and toward incorporating sta- * bility into programs. 71 Our study revealed an emerging effort, initiated within OSD during the summer of 1970, and termed "a new concept." It is intended to provide the Secretary of Defense with a broad overview of each mission category, including identifi- cation of major issues. Although this effort appears to embody many of the essentials of an overall priority system, it is still in its infancy. Recommendation--The Secretary of Defense should make every effort to develop and perfect the DOD-wide method-- now in its early stages of development--designed to be followed by all military services for determining two things: First, what weapon systems are needed in rela- tion to the DOD missions. Second, what the priority of each should be in relation to other systems and their missions. B. Definition of performance characteristics and assessment of technical risks In the last several months, persistent problems in de- fining performance characteristics of weapon systems and in - . determining technical feasibility for achievement have been receiving extensive attention at both OSD and the military service levels. On the basis of our study of recent weapon systems procurement, we see many encouraging signs that these problems are being abated. Extensive efforts are being applied, early in the pro- cess, to identifying high-risk design areas and to construct- ing and testing actual hardware to demonstrate feasibility of high-risk components before proceeding with further de- velopment. Similarly, current use of the demonstration milestone provisions in development contracts limits the Government's financial commitment pending a system's demon- strated performance. C. Standards for and consistent use of cost-effectiveness studies We saw wide variation in the quality of preparation and . follow-through given to cost-effectiveness determinations supporting weapon systems acquisition decisions. 72 The variations in quality may be due to evolving meth- odology for, and use of, cost-effectiveness studies. There is no evidence that DOD criteria for judging the adequacy of cost-effectiveness studies am being appZie&. We are convinced that the Zack of cZear guidelines for the preparation and appZieation of cost-effectiveness studies has resulted in misunderstanding of their purpose, has con- tributed signifieantzy to diversity in execution by the miZ- itary services; and has lessened the value of eost- effectiveness studies to the entire acquisition process. Recommendations--The Secretary of Defense should re- quire that (1) cost-effectiveness studies meet certain standards (including the identification of which weapon system and which considerations should be included in such studies) and (2) cost-effectiveness studies be up- dated at each point where a major program alternative is considered. With regard to the latter recommendation, we noted that instructions now require cost-effectiveness studies to be prepared at major decision points in the program. These de- cision points are validation, full-scale development, and production. D. Subsystem development phasing and interfacing A major problem recurring in the weapon systems acquisi- tion process is the compromise of system performance that occurs when a principal element of the system follows a de- velopment cycle not compatible with that of the primary sys- tem. This incompatibility occurred most frequently when the responsibility for the deveZopment of the parts of a system was divided among two or more project managers. The results were imbalance in time-phasing of subsystems in some weapon programs and incompatibiZity of technica interfaces in others. We believe that the program manager authority should cover all technical effort on all principal elements of the .weapon. Whenever a principal element is common to more than one weapon system, specific steps must be taken to ensure its .development and acquisition in order to meet the technical 73 specifications required by each of the major systems which will employ it. One way of handling that might be to give authority over the element to the manager of the more cru- cial major system. E. Assessment of technical performance In weapon programs we examined which were well along in the acquisition process, were finishing development, or were in production, we noted that assessment of progress against the development program was hampered by lack of early test results from technical high-risk areas. When techni- cal problems are revealed by testing, there have frequently been aspects that had not been formally identified as tech- nically risky early in the program and therefore had not been given the special attention needed during development. Some programs have encountered such serious technical prob- lems that degradation from required performance has been ac- cepted. More sharply defined technical risk analysis with spe- ciaZ emphasis applied to technica high-risk aspects of the -. new weapon system should give the military services a means of evaZuating deveZopment progress earZier, and more aeeur- ately, than is presently possible. In. recently initiated weapon programs, we found that special care is being taken to identify the high-risk eom- ponents and to fabricate them for testing in Zaboratory mod- els before proceeding with development of the compZete weapon system. We believe that this is a step in the right direction. F. Organization and procedures In our judgment, one of the most important unresolved problems in the management of major acquisitions is the prob- lem of organization. The problems arising from establish- . ment of need, for instance, are related to organizational deficiencies. The essence of the problem appears to come from at- tempts to combine the specialized roles of major weapon sys- , tems acquisition management into more or less historical 74 military commandstructure organizations. Because of this, there usually are a large number of organizational units not directly involved in the project which can only nega- tively influence it. In the Army and the Navy and to a lesser extent in the Air Force, project managers are part of organizations, whose basic missions are considerably broader than the managers* missions, with which organizations they must compete for resources. As a matter of fact, each military service alters tradi- tional organization patterns when faced with managing major programs. Although not recognized as a super program, in- herent organizational problems of the F-15 program were suc- cessfully overcome by the program's having been placed in the organization in such a way that the privileges of sub- stantial military rank could be exercised as a means of by- passing organizational layers. The value of this reorgani- zation is that the project manager has been given stature and authority so as to be unencumbered by normal frustra- tions produced by cooperation with the functional organiza- tions. Each of the services has begun to upgrade the rank of project managers. But military rank alone will not accom- plish what OSDand the military services are trying to do. In our opinion, lessons learned from organizational changes in structure for the super programs can aptly be ap- plied to the whole subject of weapon systems acquisition. It may be impractical to treat each of the large number of projects now under way in the military departments in a sim- ilar manner. But, it occurs to us that, ideally, there should be a direct relationship between the way weapon sys- tems requirements are categorized (strategic deterrent, land warfare, ocean control, etc.) and the organizational structure needed to acquire them. Such an arrangement would facilitate grouping related weapon systems in "packages" of commonmission and would permit putting together an acquisi- tion organization of appropriate size and stature to handle the expanded concept. We believe that eventually program management will evolve along mission lines. There are other alternatives, but whichever is chosen must clearly provide for someone to be in charge, to have 75 clear authority to make decisions, and to have full respon- sibility for the results. The Deputy Secretary of Defense recognizes that correction of this problem is fundamental to any real improvement and has stated that he plans to pursue it aggressively. Recommendations --The Secretary of Defense should place greater decisionmaking authority for each major acqui- sition in a single organization, within the service concerned, with more direct control over the operations of weapon system programs and with sufficient status to overcome organizational conflict between weapon system managers and the traditional functional organization. G. System cost experience Our analysis of the estimated costs to develop 61 major weapon systems which are prepared at various points in the development cycle shows that the current estimates through . _ program completion have grown 40 percent in comparison to the planning cost estimates for these programs. ." Cost growth may result from such things as unanticipated development difficulties, faulty planning, poor management, bad estimating, or deliberate underestimating. However, it should be realized that not all cost growth can be reasonably prevented, for instance, cost growth resulting from infla- tion. Further, some cost growth may even be desirable, for instance, incorporation of technological changes that im- prove the system effectiveness. Regarding our observations made last year, we found that DOD had made a good start toward developing data that specifically identifies the variances in program cost esti- mates for systems reported under the SAR system. We ob- served, however, that on 15 Navy systems the causes for cost change were either not provided or were incomplete. . DOD also has acted to improve the format, content, and data in the S.ARs. Our review confirmed those improvements . made during the last year. We found,however, that some im- . provements still are needed. 76 Recommendations--The Secretary of Defense should ensure that the SARs (1) contain a summary statement regarding the overall acceptability of the weapon for its mis- sion, (2) recognize the relationship of other weapon systems complementary to the subject system, and (3) re- flect the current status of program accomplishment. . . 77 APPENDIXES 79 APPENDIXI Page 1 SCHEDULE OF PROGRAM COST DATA AS OF JUNE 30, 1970, AND ARRANGED BY ACQUISITION PHASE AND MILITARY SERVICE Additional Planning Development Cost change Current procurement Total --estimate estimate . guantity Other estimate -costs __ costs . CohTcEPTUAL PHASE (9) (note d .VALIDATION/RATIFICATION (4): . ArOlJ.: None Imy : DLGN 38 769.2 769.2 3,210.8 1,510.3 5,490.3 5,490.3 SSN-688 1,650.0 1.658.0 2.376.0 245.7 4.279.7 4,279.7 Air Force: Ax 1,025.S ’ 1,025.S 1.025.5 1.025.5 OTH-B (note d) . 100.9 100.9 -3.9 104.8 104.8 ENGINEERING AND/OR OPERATIONAL SYSTEMS DEvEu)PMENI (57): Army : Cheyenne (note b) l 125.9 125.9 76.2 202.1 202.1 Shillelanh 357.4 357.4 -18.1 156.5 495.8 i5.8 521.6 SAFEGIJARli 4.185.0 *_ 1,365.0 389.0 5.939-o 5.939.0 DRAGON 382.2 404.2 -232.7 75.9 247.4 i7.4 284.8 SAM-D (note c) 4.916.8 3.989.0 -1,791.4 1.215.9 3,413.5 82.4 3.495.9 LANCE 586.7 652.9 108.2 761.1 90.4 851.5 TOM 410.4 727.3 -300.1 248.1 675.3 33.3 708.6 Improved HAWK 573.3 - 573.3 -79.8 210.8 704.3 107.2 811.5 M-60 AIE2 162.1 202.6 -15.8 172.5 359.3 16.5 375.8 MBT-70 2.126.5 2,091.4 -602.4 336.7 1,825.7 293.3 2,119.0 Sheridan Tank (note f) 422.5 375.6 -13.1 93.4 455.9 31.6 487.5 Sheridan Ammunition 370.1 370.1 -125.2 105.9 350.8 350.0 cAMM4 GOAT 69.1 163.9 16.9 186.9 il.7 198.6 CHAP/VULCAN 58.2 58.2 38;:; 78.4 523.8 138.8 662.6 TACFIRE 123.6 160.5 24.0 184.5 3.8 188.3 Navy : . s-3!% 1,763.8 2.891.1 42.7 2.933.8 20.6 2,954.4 F-14 6.166.0 6.166.0 2,0;6.1 77.0 8.279.1 294.4 8.573.5 EA-6B 689.7 817.7 -50.7 291.6 1,058.6 31.5 1.090.1 P-X 1.294.2 1.294.2 971.1 285.7 2,551-O 59.0 2.610.0 A-7E 1.465.6 1.465.6 -385.3 494.4 lJ74.7 91.7 1.666.4 AN/SQS-23 157.1 170.5 -82.7 144.5 232.3 50.8 283.1 AN/ SQS-26 95.7 88.8 30.8 119.6 119.6 AN/BQQ-2 126.9 179.0 86.2 265.2 i3.5 298.7 DIFAR 178.5 414.1 97.0 46.9 558.0 (e) 558.0 VAST ANfUSM-335 49.8 57.5 -26.6 22.5 53.4 54.8 ;ASVS~xiUSM-247 241.1 312.0 -182.2 282-6 412.4 73.: 489.7 370.8 677.4 642.3 181.2 1.500.9 11:o 1.511.9 CONDOR 356.3 441.0 -220.9 131.3 351.4 353.3 POSEIDON 4,568.7- -243.6 790.2 5,1x.3 1,74::(: 6.855.5 Standard A?% Go.3 241.6 -10.4 -20.2 211.0 17.2 228.2 Sparrow E 687.2 740.7 -459.8 11.7 292.6 32.5 325.1 Sparrow F 139.8 453.6 114.7 489.9 1,058.2 26.3 1.084.5 Standard 313.2 --7.0 34.0 340.2 468.1 808.3 Mark 48 Hod O&l 720.5 7i4.0 488.9 2.554.3 3,757.Z 28.2 3.785.4 LHA 1.380.3 1.380.3 47.5 1.427.8 8.2 1.436.0 CVAN-68 427.5 116.7 544.2 544.2 CYAN-69 519.0 - 519.0 519.0 DE-1052 1.285.1 1.2g9.7 167.9 1.427.6 (2 1.427.6 SSN-632 2,515.8 397.3 2,913-l 2,913.l SSN-ii85 l&.8 151.7 23.4 175.1 175.1 DIX modernization 698.8 698.8 153.4 852.2 852.2 324.4 328.5 -1G9.8 -12.6 186.1 196.1 DSRV loo.2 '143.7 -41.3 101.9 204.3 234.9 DD-963 1,704.4 2.581.2 1.595.4 4.176.6 AXIS (note 9) 388.0 427.6 13.0 440.6 81 APPENDIX I Page 2 Additional Planning Development Cost change Current procurement Total estimate -- ~estimate puJntity --Other estimate costs costs (millions) ENGINEERING AND/OR OPERATIONAL SYSTEMS DEVELOPMENT (57) (continued): Air Force: B-l 0.954.5 10,107.a 10.107.8 392.9 10,500.7 F-15 (note h) 6,039-l 7.355.2 7,356.4 763.7 8,120.l C-5A 3,423.0 3,413.2 -756.2 1.63::: 4,308.6 285.7 4,594.3 F-111 AICIDfEtF 4,686.6 5.505.0 -2,581.3 3,456.6 6.380.3 960.3 7,340.6 FB-111 1.781.5 1.781.5 -1.043.3 468.7 1.206.9 231.6 1,438.5 A-7D 1.379.1 -282.6 303.1 1.399.6 173.5 1,573.l AUACS 2.661.6 2.661.6 126.0 2Jg.f MAVERICK --257.9 383.4 -;3.6 33.8 343.6 8.0 TITAN III 932.2 814.1 373.8 1,187.9 (4 1.18719 SRAM 167.1 $36.6 118.3 735.8 1.090.7 590.9 1.681.6 Minuteman II 3,014.l 4.254.9 4.0 207.5 4.466.4 583.2 5.049.6 Minuteman III 2,695.5 4I673.0 -37.7 999;1 5;635.2 362.6 5.997.8 777 COMSAT 133.5 138.0 5.1 143.1 143.1 "Comparative cost data not available'for systems in this phase. bThe Cheyenne costs represented research and development costs only. The production contract Gas termi- nated on May 19, 1969. Due to pending litigations, the Army's liability was unknown. c Army officials advised us that, while the SAM-D had gone through contract definition, contract award had been limited to advance development. dCost data as of August.31. 1970. for the OTH-9. %ata were not available for inclusion. fThe DOD considered this as an annex to the Sheridan vehicle and not a weapon system itself. gResearch and development costs only. %I e original Development Concept Paper No. 19 dated Sept. 28, 1968. contained a preliminary planning estimate for lower quantity of F-15's as $5,137 million. 82 APPENDIXII Page 1 DIRECTOR OF DEFENSE RESEARCH AND ENGINEERING WASHINGTON, D C. 20301 22 JAN 1971 Mr. C. M. Bailey Director, Defense Division United States General Accounting Office Washington, D. C. 20548 Dear Mr. Bailey: This letter is in response to the request of Mr. Hassell Bell of your organization for informal comments on GAO draft report “The First Report on the Continuing Evaluation of the Acquisition of Major Weapons Systems” (OSD Case #3219). I know that you appreciate the extremely limited time the DOD had to review this report. However, in recognition of the equally limited time which Mr. Bell indicated the GAO has to meet its commitment for submission of the report to the Congress, we have done our best to prepare a general reaction to it. Because of the nature and importance of this subject, we will want to examine the final report further in a more thorough and logical fashion. It would be appreciated if your report to the Congress could indicate the fact that the DOD has not had sufficient time to make such a review. We have reviewed the draft report and believe that your recom- mendations address important aspects of the weapon system process. We agree in particular that we have not yet solved some of the organizational problems and we will see that your report is made available to the Services and OSD offices which are working on those problems, We do appreciate the recognition that you give to the DOD efforts to improve its management of weapon systems acquisition, and we know that you realize we are giving considerable time and attention to further improvements. 83 APPENDIXII Page 2 As you know, we are carrying on comprehensive evaluations of this management problem here in the Department. GAO reviews, such as this, will be of benefit to us, particularly by giving us an independent review and evaluation of our options. We are pleased to assist you by providing these informal comments on the draft report. We will forward more detailed comments after we have made a more thorough evaluation of the report, if you feel that would be helpful. Sincerely, U.S. GAO Wash.. C 84
Acquisition of Major Weapon Systems
Published by the Government Accountability Office on 1971-03-18.
Below is a raw (and likely hideous) rendition of the original report. (PDF)