oversight

Tactical Missiles: Issues Concerning the Navy's Requirements Determination Process

Published by the Government Accountability Office on 1990-09-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

   .                                                                        -.
                   United   States   General   Accounting   Office


GAO                Report to the Chairman, Subcommittee
                   on Defense, Committee on
                   Appropriations, U.S. Senate


September   1990
                   TACTICAL MISSILES
                   Issues Concerning the
                   Navy’s Requirements
                   Determination Process




                      EESTBICTED--       Not to be released outside   the
                      General Accounting   Of&e unless specifically
                      approved by the Office of Congressional
                      Relations.


GAO/NSIAD-90-233
GAO
             United States
             General Accounting  Office
             Washington, D.C. 20548

             National Security and
             International Affairs Division

             B-240050

             September 12,199O

             The Honorable Daniel K. Inouye
             Chairman, Subcommittee on Defense
             Committee on Appropriations
             United States Senate

             Dear Mr. Chairman:

             The costs associated with developing and procuring Navy tactical mis-
             sile systems exceed billions in current year dollars. In view of reduced
             defense spending, you expressed concern about the Navy’s plan to pro-
             cure large numbers of missile systems during the 1990s. In response to
             your request, we examined the Navy’s requirements determination pro-
             cess, including (1) how operational requirements were developed,
             defined, and stated and (2) how the total number of weapons (inventory
             objectives) that are needed to support the requirement was established.

             Initially, we examined the requirements documentation for 19 tactical
             missile systems. At the Chairman’s suggestion, however, we limited our
             review to three missile systems. We selected the Standoff Land Attack
             Missile (SLAM),the Advanced Interdiction Weapon System (AIWS), and
             the Standard Missile II Block IV System (AEGIS ER) because they illustrate
             the types of deficiencies we found at various stages of the requirements
             determination process.


             The requirements process consists of two elements: (1) the development
Background   of requirements documents, which state the acquisition strategy and can
             refer to generic capabilities or specific operational or performance capa-
             bilities and (2) the establishment of inventory objectives, a process of
             determining what to buy, when to buy it, and how many to buy. Integral
             to this process is the Non-Nuclear Ordnance Requirements (NNOR) pro-
             cess. The NNOR process establishes estimates of inventory objectives for
             conventional (non-nuclear) munitions. It relies on (1) military assess-
             ments and (2) computer simulations.

             A number of significant issues (e.g., requirements documentation, inven-
             tory objectives, affordability, and rationale) surround the Navy’s ability
             to acquire and support missile systems successfully.




             Page 1                                    GAO/NSIAMO-233   Navy Tactical   Missiles
                       E-240060




                       We found that adherence to the Office of the Secretary of Defense (OSD)
Results in Brief       and Navy guidance in determining requirements for the three systems
                       we reviewed was inconsistent. Key requirements documents were often
                       unapproved or missing. Furthermore, the process by which inventory
                       objectives were established was not always clear or well documented,
                       and it was not possible to quantitatively match inventory objectives of
                       Navy tactical missiles to military requirements.

                       The Navy has not consistently followed procedures for establishing
                       requirements or setting inventory objectives for the three missile sys-
                       tems we reviewed. We found that

                       the process for determining SLAMinventory objectives was unclear and
                       predicated largely on assumptions and military judgments;
                       the stated operational capabilities for the baseline AIWS may differ from
                       actual capabilities, thus rendering key requirements documentation mis-
                       leading; and
                       a major system in development, the AEGIS ER, may proceed to production
                       without an approved operational requirement for the system or other
                       required documentation and formal agency approval.


                       The Navy establishes estimates of inventory objectives for conventional
Concerns With the      munitions through the NNOR process. All requirements for naval muni-
Non-Nuclear Ordnance   tions are developed through models that rely heavily on intelligence
Requirements Process   estimates, naval force doctrine, judgment, subjective information, and
                       probability modeling. Concerns have been raised within OSD as to the
                       reasonableness and soundness of assumptions in determining the inven-
                       tory objectives for missile systems. While requirements planners and
                       reviewers have been generally satisfied with the results of the process,
                       the assumptions and projections entered in the NNOR model have some-
                       times led to inflated inventory objectives.


                       The SIAM, a derivative of the HARPOON antiship missile, is an advanced
Inventory Objectives   air-to-surface medium range missile to be used in a high-threat environ-
of SLAM Not            ment. Its flyaway unit cost has increased from $800,000 in 1989 to over
Supported              $900,000 in 1990. Originally intended as an interim system, the Navy
                       recently doubled the SLAMinventory objective. However, Xavy require-
                       ments planners could not provide justification for this action or show
                       support for the original inventory objective. In addition, some i%avy offi-
                       cials have reported plans to develop a follow-on to SLAM, which will be



                       Page 2                                    GAO/NSIADgO-233   Navy Tactical   Missiles
                       more expensive than the baseline SLAMand which will be designed to
                       meet military requirements not met in the baseline SLAM program.


                       The AIWS, an advanced standoff weapon to be used against low value
AIWS May Not Meet      fixed land targets, was designated to be a low cost replacement for the
Basic Requirement or   SLAMand a replacement for the Maverick, Walleye, Paveway, and
Improve Existing       Skipper missiles. However, the baseline AIWSmissile may not provide a
                       similar capability equal to the systems it is to replace. In addition, the
Capability             baseline missile, which has a $60,000 limit placed on its unit cost by the
                       Secretary of the Navy, may actually cost about $80,000. The AIWS is in
                       the demonstration/validation    phase of its acquisition program.


                       The AEGIS ER, a surface-to-air missile, is designed to provide fleet air
AEGIS ER Acquisition   defense for surface ships by countering high altitude missiles. The Navy
Proceeding Without     has not developed or approved an operational requirement for the
Formal Approval        system. While there were initial indications that the Navy might proceed
                       into production without operational testing, Navy officials said such
                       tests will be conducted. If not, it could lead to a commitment of signifi-
                       cant production funds before technical and operational problems are
                       identified and resolved.


                       The requirements determination process is described in greater detail in
                       appendix I, including a discussion of the three systems identified as
                       illustrative of deficiencies in the requirements process. Our objectives,
                       scope, and methodology are presented in appendix II.

                       As discussed with your staff, we did not obtain official agency com-
                       ments. However, we discussed our findings with responsible agency offi-
                       cials and included their comments where appropriate.

                       Unless you publicly announce its contents earlier, we plan no further
                       distribution of this report until 30 days from the date of this letter. At
                       that time, we will send copies of this report to the Chairmen, House
                       Committees on Appropriations and on Government Operations and
                       Senate Committee on Governmental Affairs, and the Secretaries of
                       Defense and the Navy, Copies will be made available to other interested
                       parties on request.




                       Page 3                                    GAO/‘NSL4&90-233   Navy Tactical   Misdlea
B-240060




Please contact me at (202) 2756504 if you or your staff have any ques-
tions concerning this report. Major contributors to this report are listed
in appendix III.

Sincerely yours,




Martin M Ferber
Director, Navy Issues




 Page 4                                   GAO/NSLADSO-233   Navy Tactical   Missiles
Page 5   GAO/NSIAD!W233   Navy Tactical   Missiles
                                                                      ~-
Coritents


                                                                       1

Appendix I                                                             8
Deficiencies in the     The Requirements Planning Process
                        The NNOR Process
                                                                       8
                                                                       9
Requirements            The Three Tactical Missile Systems Reviewed   10
Determination Process
Appendix II
Objectives, Scope,and
Methodology
Appendix III
Major Contributors to
This Report




                        Abbreviations

                        AEGISER AEGIS Extended Range Missile
                        AIWS    Advanced Interdiction Weapon System
                        GAO     General Accounting Office
                        NNOR    Non-Nuclear Ordnance Requirements
                        CBD     Office of the Secretary of Defense
                        SLAM    Standoff Land Attack Missile


                        Page 6
Page 7   GAO/NSIADBM33   Navy Tactical   Missiles
Appendix I

Deficienciesin the Requirements
Determination Process

                       The Navy’s requirements process provides the foundation and structure
                       for planning, developing, evaluating, and fielding weapon systems to
                       respond to the threat within reasonable time frames and at a reasonable
                       cost. The degree of formality, extent of documentation, and level of deci-
                       sion authority vary with the dollar value of a program.

                       The management of major weapon system programs is normally divided
                       into phases to provide effective oversight during development and pro-
                       curement.’ These phases include concept demonstration/validation,    full-
                       scale engineering development, limited production, and full-rate produc-
                       tion Department of Defense approval is normally required at key deci-
                       sion points, or milestones, before these programs can proceed from one
                       phase to the next.


                       The term “requirements” has several meanings. In the broadest sense, it
The Requirements       can be applied to generic capabilities or it can be applied to specific
Planning Process       operational or performance characteristics. It also applies to contract
                       specifications that dictate how a weapon system will be built, tested,
                       and inspected. For the purpose of this review, requirements refers to the
                       military capability.

                       When dealing with state-of-the-art technology, no process, however
                       sound and well staffed, can anticipate all future uncertainties in fielding
                       a weapon system that has yet to be developed, let alone put into active
                       service. Incorrect estimates and assumptions that occur during the
                       requirements planning process can become magnified later in the form
                       of program instability, production delays, or excess costs.

                       Department of Defense and Navy guidance provide the structure and
                       procedures for managing and funding weapon system programs. The fol-
                       lowing principal documents are required.

                   l Tentative Operational Requirements statement identifies deficiencies in
                     meeting the threat in the current mission and outlines, in general terms,
                     key capabilities desired.
                   l Development Options Paper outlines alternative means of solving defi-
                     ciencies in meeting the threat.
                   . Operational Requirement contains the statement of capabilities
                     required.

                       ‘F’~~~EuIwthat are expected to exceed $200 million in research, development, test, and evaluation or
                       $1 billion in procurement funds are classified as major weapon system programs.



                       Page 8                                                  GAO/NSIAD-90-232     Navy Tactical   Missiles
                       Appendix I
                       Lkflciencies  in the Requirements
                       Determination    Prow739




                   l   Decision Coordinating Paper identifies program options based on initial
                       studies of design concepts, alternative acquisition strategies, planned
                       operational capabilities, and cost estimates.
                   l   Test and Evaluation Master Plan contains performance parameters
                       defined in the Decision Coordinating Paper, outlines the plan for testing
                       and evaluating systems, and reflects changes and updates in capabilities
                       as systems proceed through development.

                       These documents provide background for milestone decisions associated
                       with research and development and production of weapon systems.
                       They outline the weapon system’s purpose; the new system’s uniqueness
                       as compared to existing systems; the evaluation of risks, funding, and
                       scheduling aspects; the acquisition strategy; and the test and evaluation
                       strategies.


                       Integral to the acquisition strategy is an assessment of how many
The NNOR Process       weapons will be bought. In the Navy, calculation of inventory objectives
                       for most types of tactical missiles is done through the NNOR model.

                       The NNOR model plays a key role in determining inventory requirements
                       for most tactical missiles in the Navy. The numbers generated by this
                       model, after a review by high ranking Navy officials, are used as a basis
                       for the Navy’s Program Objectives Memorandum submission to OSD.

                       Factors that affect inventory requirements and that are entered into the
                       NNOR model are revised estimates on the threat, platform availability,
                       adjustments in the probability of kill percentages, and revisions in false
                       target ratios. Different allocations of targets to a missile, trade-offs
                       between systems, needed upgrades in weapon systems, and program ter-
                       mination are also factors.

                       The Office of the Assistant Secretary of Defense for Production and
                       Logistics tasked the Rand Corporation to review the Navy’s procedures
                       for estimating conventional munitions. The study, published in April
                       1989, identified several deficiencies in the NNOR mode1.2For example, the
                       model assumes the complete independence of every U.S. ship and air-
                       plane, which is in contrast to evolving strategies and warfare tactics. In
                       addition, cost trade-off values among comparable munitions are not con-
                       sidered as the model assumes no budget constraints.


                       “A Rand Note: Conventional Munitions Requirements Estimation in the Navy.



                       Page 9                                               GAO/NSIAD-99-222       Navy Tactical   hiidles
                     Appendix I
                     Deficiencies  in the Requirements
                     Determination    Process




                     Another deficiency, according to Rand analysts, is that an analysis of
                     the validity and soundness of assumptions is not performed to deter-
                     mine which are the key variables that affect requirements. Several vari-
                     ables inflate inventory objectives, including the assumption of the need
                     to destroy 100 percent of the threat to win the war, excessive categori-
                     zation of targets (more than one type of weapon assigned to a target),
                     and false targets treated like real targets. The NNORmodel does not
                     account for maldistribution of weapons or, in some cases, loss of muni-
                     tions (i.e., such as when supply ships are sunk). OSDand Navy require-
                     ments planners generally concurred with the Rand findings.

                     Thus, it is difficult to measure the adequacy or soundness of inventory
                     objectives generated by the model. Navy planners and CXDprogram
                     evaluators familiar with requirements planning have reported that they
                     have occasionally rejected NNOR-generated inventory objectives for being
                     too inflated or not supportable. However, generally they have been sat-
                     isfied with the results of the process.


                     During our review of the three missile systems, we found that the Navy
The Three Tactical   did not always follow its own or OSDguidance and procedures for estab-
Missile Systems      lishing requirements or setting inventory objectives. For example,
Reviewed             (1) the Navy did not go through prescribed procedures for determining
                     inventory objectives for SLAM,(2) the baseline AIWSwill not provide an
                     enhanced capability over systems it is designed to replace, and (3) the
                     AEGISERmay go forward in development and/or production without
                     formal approval of required documentation.

                     The following systems illustrate deficiencies we found in various stages
                     of the requirements determination process.


Status of SLA.M Is   The SLAMis a land attack variation of the HARPOON missile. The total
                     unit cost of SLAMis about $1.121 million; the missile flyaway unit cost is
Uncertain            about $910,000 in current year dollars. With its high cost, the SLAMwas
                     originally considered an interim solution in meeting the low inventory of
                     air-launched modular standoff weapons available for a multiservice
                     role.

                      The acquisition strategy of SLAMhas not been well defined. Even though
                      it was originally scheduled for only limited production, Navy officials
                      said that the inventory objectives had recently doubled. Furthermore,



                      Page 10                                 GAO/NSIADMb233   Navy Tactical   .Misailes
                         Appendix I
                         Defldendes    in the Requirementa
                         Determination    Process




                         Navy officials disagree as to the future of SLAM.While one official indi-
                         cated the SLAMprogram will not be expanded, other officials indicated
                         that the requirement for SLAMmay be even greater than currently stated
                         and that a follow-on and costly enhancements are being considered.

                         Navy requirements planners who execute the NNOR model reported that
                         they could not support or justify how the Navy established the initial
                         inventory objective, nor were they provided a rationale for doubling the
                         inventory objective that was outside of the routine NNOR modeling
                         process.


Proposed AIWS May Not    The AIWSis an advanced air-to-surface, standoff missile designed to pro-
                         vide close air support, battlefield interdiction, and destruction of low
Be an Improvement        value ground targets in a high-threat environment. It was initially
                         viewed as a low cost alternative to SLAM, which costs $910,000 per unit,
                         and a replacement for the Walleye, Skipper II, Laser Maverick, and
                         Paveway Guided Bomb. However, its estimated unit cost has already
                         exceeded the $60,000 limit the Secretary of the Navy placed upon it. In
                         fact, the currently projected baseline ADJSunit cost may rise to over
                         $80,000. In addition, a proposed planned product improvement may cost
                         over $170,000 per unit.

                         A rationale for developing a new weapon system, among other justifica-
                         tions, is to provide a similar capability at a lower cost over existing
                         weapon systems. We found, and Navy officials acknowledge, that the
                         baseline AIWSmissile, currently in demonstration/validation,      may fail to
                         provide a capability equal to systems it is to replace. 0s~ documents
                         show a pre-planned product improvement of this missile, proposed for
                         future development upon completion of the baseline production pro-
                         gram, may actually serve as the replacement missile. The AIWSproposed
                         pre-planned product improvement would provide enhanced range and
                         targeting capability. The requirements documentation did not clearly
                         distinguish the capabilities of the baseline and the improved AIWS. Navy
                         officials stated that it has been their intent to follow a two-pronged
                         approach to AMOSdevelopment, and they acknowledged that the require-
                         ments documentation can be misinterpreted.


AEGIS ER Is Lacking      The AEGISER,also known as the Standard Missile II Block IV, is a
                         surface-to-air missile that will replace the Standard Missile II Block II on
Critical Documentation   AEGIS cruisers and destroyers equipped with the MK-41 vertical launch



                         Page11                                     GAO/NSIAD9O-233Navy   TacticalMissiles
Appendix I
Deficiencies  in the Requirements
Determination    Process




system. The missile, which is in development, is being designed to pro-
vide fleet air defense for surface ships by countering high altitude
missiles.

The Navy has awarded contracts worth over $231 million to Raytheon
and General Dynamics, codevelopers of the AEGIS ER missile, but it has
not determined the total procurement costs for the AEGIS ER. Develop-
ment costs exceed $360 million in current year dollars.

Navy program sponsors report that the AEGIS ER is an upgrade, not a new
development effort. General Dynamics disagrees, however, indicating
that the missile is over 60 percent new and, in many ways, is a major
development effort. According to Navy guidance, an operational
requirement must be developed and approved to support new research
and development programs. This is particularly true for such major
acquisition programs as AEGISER. However, the Navy has not developed
or approved an operational requirement for the system.

The Navy has not approved the Decision Coordinating Paper or the Test
and Evaluation Master Plan, yet the Navy has decided to purchase 300
AEGISER missiles in the early 1990s through a pilot production program.
The Decision Coordinating Paper would provide a description of the
weapon system’s concept and an assessment of technological risks.
Master plans are required for all major defense acquisition programs.
Furthermore, test and evaluation of system capabilities cannot begin
until the master plan is approved.

We were initially told that the Navy might forego a series of operational
tests normally conducted in a realistic operational environment that
would demonstrate the combat effectiveness and suitability of the AEGIS
ER and resolve technical uncertainties and problems at the conclusion of
the pilot production phase before entering full-rate production. How-
ever, Navy officials recently indicated that they plan to conduct opera-
tional tests and evaluation before entering into the full-rate production
phase.




 Page 12                                 GAO/NSiAIMO-233Navy Tactical.Missiles
Appendix II

Objectives,Scope,and Methodology


              The Chairman, Subcommittee on Defense, Senate Committee on Appro-
              priations, asked us to examine the procedures the Navy followed in
              matching inventory objectives to military requirements for tactical mis-
              siles. Our objectives were to review how operational requirements were
              developed, defined, and stated and how the total number of weapons
              needed to support the requirement was established.

              Initially, we examined the requirements documentation for 19 tactical
              missile systems. At the Chairman’s suggestion that we narrow our
              review, we selected three systems-the SLAM,the AIKS, and the AEGISEX.

              To examine the procedures used in the requirements determination pro-
              cess, we met with officials at the Offices of Research and Engineering;
              Weapons Systems Assessment; Program Analysis and Evaluation, Pro-
              duction and Logistics; Tactical Warfare Programs; and Joint Technical
              Coordinating Group/Munitions Effectiveness in the Office of the Secre-
              tary of Defense. We also met with Navy officials from Ordnance Mate-
              rial Management, Aviation Plans and Requirements, Weapons Analysis
              Branch (Tactical Air Surface and Electronic Warfare Development
              Branch), Commander in Chief, Headquarters Atlantic Fleet Ordnance;
              program managers from Naval Air Systems Command and Naval Sea
              Systems Command; and the former Commander in Chief, Headquarters
              Atlantic Fleet, and Sixth Fleet commander. We also held discussions
              with issue area experts from the Rand Corporation, Brookings Institute,
              Center for Strategic and International Studies, and the Analytical Sci-
              ences Corporation.

              We conducted our review from May 1989 to May 1990 in accordance
              with generally accepted government auditing standards.




              Page 13                                  GAO/NSLADSO-333   Navy Tactical   Missiles
Appendix III

Major Contributors to This Report


National Security and   John J. D’Esopo, Assistant Director
International Affairs   Lenora Fuller, Evaluator-in-Charge
Division,               Jean Fox, Evaluator
                        Daniel Hoagland, Evaluator
Washington, DC.         Laura Jackson, Evaluator
                        Daniel Tikvart, Evaluator




                         Page 14                              GAO/‘NSIAD-90-233   Navy Tactical   Misdes
 (394312)
    ‘k

.




         Ordering   Information

         The first five copies of each GAO report are free. Additional copies
         are $2 each. Orders should be sent to the following address, accom-
         panied by a check or money order made out to the Superintendent
         of Documents, when necessary. Orders for 100 or more copies to be
         mailed to a single address are discounted 25 percent.

         U.S. General Accounting Offrce
         P.O. Box 6015
         Gaithersburg, MD 20877

         Orders may also be placed by calling   (202) 2756241.