oversight

Acquisition of Major Weapon Systems

Published by the Government Accountability Office on 1971-03-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

Acquisition Of
Major Weapon Systems     E-763058




Department of Defense




BY THE COMPTROLLER    GENERAL,      I
OF THE UNITED  STATES        --
                      COMPTROLLER      GENERAL       OF    THE      UNITED   STATES
                                     WASHINGTON.      D.C.       2O!X8




    Bm i63058




    To the      President         of the Senate      and the
    Speaker       of the       House    of Representatives

               This   is our report    on the acquisition       of major  weapon
    systems       by the Department        of Defense.       Our review   was made
    pursuant       to the Budget    and Accounting        Act,   1921 (31 U.S.C.  53),
    and the      Accounting    and Auditing     Act of 1950 (31 U.S.C.       67).

             In addition,   you will receive       a classified     supplement          con-
    taining    summaries       of our evaluations       of the individual        weapon
     systems      covered  by our study.        More    detailed   classified        studies
    have been prepared          on each weapon       system.     Copies       of  these
    studies     will be provided     on request.

           Copies    of this          report        are being    sent to the Director,
    Office  of Management                and       Budget;   the Secretary   of Defense;
    and the Secretaries               of the       Army,   Navy,    and Air Force.




                                                             Comptroller              General
                                                             of the United            States




.




                50 TH ANNIVERSARY                  1921-     1971
                              Contents
                                                                        Page

    DIGEST                                                                1

    CHAPTER

       1      INTRODUCTION                                               4
                  The Development Process        for   a Major
                    Weapon System                                         5
                  Concepts of This Study                                 11

      2       ASSESSMENTOF CRITICAL MANAGEMENTACTIONS                    13
                  Requirements                                           13
                       Identification       of need for a system         13
                      Definition        of performance     character-
                          istics                                         19
                      Obtaining assurance of feasibility
                          of performance requirements                    24
                      Cost-effectiveness          determinations         28
                       Stability      of the program and its re-
                          lationship      to other programs              33
                       Subsystem development phasing and
                          interfacing                                    36
                      Continuous trade-off          between cost
                          and performance                                41
                  Technical Assessment                                   43
                  Organization       for Program Management              48
                       Organizational       "layering"                   51
                       DOD-proposed action on acquisition
                          management problems                            56

       3      SYSTEMCOST EXPERIENCE                                      57
                  Analysis of Cost Changes as of June 30,
                    1970                                                 61
                  Quantity     Changes                                   62
                  Engineering     Changes                                63
                  Support Changes                                        63
                  Schedule Changes                                       63
.                 Economic Changes                                       64
                  Estimate Changes                                       64
                  Unidentified     Causes for Cost Change                65
                  SAR System                                             66
                                                                            .


CHAPTER                                                          Page

          4   SYSTEMSCHEDULEEXPERIENCE                            67
          5   GENERAL OBSERVATIONS, CONCLUSIONS, AND REC-
                OMMENDATIONS                                     70

APPENDIX

          I   Schedule of program cost data as of June 30,
                1970 and arranged by acquisition phase and
                military service                                 81

     II       Letter   from Director  of Defense Research and
                 Engineering   of January 22, 1971, commenting
                 on this report                                  83

                                 ABBREVIATIONS

              Army Materiel       Command

CDC           Combat Development          Command

CONARC        Continental       Army Command

DA            Department      of the Army

DOD           Department      of Defense

GAO           General     Accounting      Office

OSD           Office     of the Secretary          of Defense

SAR           Selected      Acquisition     Reporting

SOR           Specific      Operational     Requirement


                                                                        .   ”
,

COMPTROLLERGENERAL'S                             ACQUISITIONOF MAJORWEAPON
                                                                         SYSTEMS
REPORTTO THECONGRESS                             Department of Defense B-163058


DIGEST
------

    WHYTHEREVIEWWASMADE
              The large investment required in recent years for acquisition   of major
              weapons has impacted heavily on the resources available   for other na-
              tional goals and priorities.

              Acquiring these major weapons involves substantial     long-range commitment
              of future expenditures.     Because of deep concern in the Congress on these
              matters and because of evidence that the weapon systems acquisition     pro-
              cess has serious weaknesses, the General Accounting Office (GAO) has un-
              dertaken to provide the Congress and the Department of Defense (DOD)
              with a continuing series of appraisals    of those factors most closely re-
              lated to effective   performance in procuring major weapons. This report
              represents GAO's first    such appraisal.


FINDINGSAND CONCLUSIONS
    .         1
              I .    Concurrent with GAO's studies,    over the last several months the Of-
                     fice of the Secretary of Defense (OSD) and the military       services
                     have been engaged in a substantial    effort  to identify   and solve prob-
                     lems that have adversely affected the acquisition       of major weapon
                     systems in terms of compromised performance, delayed availability,
                     and increased costs.    GAO has found that generally the newer weapon
                     procurements are following   a slower development pace and procurement
                     practices are more conservative    than those of earlier    periods.   Be-
                     cause many of the current programs are in early states of acquisition,
                     evidence of the results of the changed concepts is not yet available
                     to adequately assess them, but the outlook is brighter.

              2. The identification     of need for a weapon system and the relative
                 priority    to be assigned its development is a fundamental problem in
                 acquisition     of weapon systems.

                     Initial  decisions as to which weapon system will be developed
                     and the priority    of its development is made by any one of the mili-
                     tary services,    but DOD has no organized method by which such pro-
                     posals can be measured against its total needs. Such a method is          _
                     now under development but it is in its infancy.                 s-r
         .
              3. In recent months, the Office of the Secretary of Defense and the mil-
                 itary services have paid extensive attention  to the persistent prob-
                 lems of defining performance characteristics  of weapon systems and

        Tear Sheet
                                                                                                                      .
     of determining           the technical   feasibility               of achieving         that  perfor-
     mance.     There       are many encouraging          signs         that  these       problems    are
     being   abated.

     Extensive    efforts        are being applied--early              in the weapon development
     process --to     identifying        areas with high design             risks   and to con-
     structing    and testing         the hardware      itself       to demonstrate       the feasi-
     bility    of high-risk         components   before        proceeding      with  further    devel-
     opment.

4.   In the preparation       of and attention  given   to cost-effectiveness
     determinations,     there   was a wide range of quality.          This variation
     has lessened     the value of these studies      to the entire       acquisition
     process.

5.   One of the most important              unresolved       problems    in the management          of
     major acquisitions         is the problem         of organization.          The essence        of
     the problem       appears    to be attempts         to combine      the specialized        roles
     of major weapon systems             acquisition       management      into more or less
     traditional       military     cotnnand structures.            Because of this,       there
     usually       are a large    number of organizations             not directly      involved
     which      can only negatively         influence      the project.

     It occurs to GAO that ideally                      there      should      be a direct       relationship           * ;
     between       the missions         for which weapon systems                  requirements          are deter-        I
     mined;      e.g.,      strategic      deterrent,           land warfare,        ocean control,           etc.,   * , ;
     and the organizational                structure          needed to acquire            them.      Such an             I
     arrangement          would facilitate            grouping        related     weapon systems           in pack-       I
     ages of common mission                and would permit              putting     together        an acquisi-          I
     tion     organization           of appropriate           size and stature           to handle       these
     matters.          Eventually,        GAO believes,            program      management       and organiza-
     tion will         evolve      along mission          lines.

     There       are other       alternatives       involved,       but whichever     is chosen must
     clearly        provide      for someone to be in charge,                to have authority   to                         I
                                                                                                                            I
     make      decisions        and to have full         responsibility        for the results.     The                     I
     Deputy       Secretary        of Defense     has recognized          that   the correction  of                         I
     this      problem      is fundamental        to any real         improvement    and has stated
                                                                                                                            i
     that      he plans       to pursue       it aggressively.                                                              I
                                                                                                                            I
                                                                                                                            I
6.   GAO found that,        on 61 weapon systems           where complete        cost data were
     available,     estimates       to develop       and produce     the weapon system had                                  i
                                                                                                                            I
     increased     some $33.4 billion.            About one third        of this     increase,        or                    I
     $9.5 billion,       represented      the difference        between     the estimate        pre-                        I
     pared when the system was first                approved    for development          (the plan-                       . 1
     ning estimate)        and an updated        estimate    prepared     when the system was                               I
     about to be placed under a development                  contract.        The remaining
                                                                                                                          * I
     $23.9 billion       increase     was due to changes          in quantities        to be acquired.                      ,
     and to a combination           of such things        as engineering        changes,     revisions                      I
                                                                                                                            I
     to estimates,       and provisions        for increased        cost due to economic            in-                     I
     flation.      (See p. 58.)                                                                                             I
                                                                                                                            I
                                                                                                                            I
                                                                                                                            I
                                                                                                                            I
                                                                                                                            I
                                                  2
                                                                                                                            ;
                                                                                                                            I
                                                                                                                            I
,
I
I
                    RECOI'@dENLlATIONS
                                  ORSUGGESTIONS
                            The Secretary   of Defense should:

I
I                           1. Make every effort to develop and perfect a       Department-wide method--
                               now in its early stages of development--to       be followed by all mili-
                               tary services for determining two things:        first,   what weapon systems
                               are needed in relation  to the Department's      missions; second, what
                               the priority  of each should be in relation      to other systems and their
                               missions.

                            2. Establish  guidelines    and standards for the preparation  and utiliza-
                               tion of cost-effectiveness      studies.  These guidelines should require
                               that studies be updated and reviewed as part of the decision process
                               when major changes in cost and/or performance require revised sched-
                               ules for funding commitments.

                            3. Place greater decisionmaking authority     for each major acquisition
                               in a single organization   within the service concerned, with more
                               direct control over the operations of weapon systems programs and
                               with sufficient status to overcome organizational       conflict    between
                               weapon system managers and the traditional     functional     organization.
    I
    I
                ’           4. Ensure that each selected acquisition    report (a) contain a summary
                               statement regarding the overall acceptability     of the weapon for its
                               mission, (b) recognize the relationships    of other weapon systems
                               complementary to the subject systems, and (c) reflect     the current ,
                               status of program accomplishment.

    I

    I
    I               AGENCY
                         ACTIONSANDUNRESOLlrED
                                            ISSUES
    L

                            DOD has been actively     pursuing a program to improve the management of
                            the acquisition    of major weapons. The Deputy Secretary of Defense has
                            assumed a significant     role in this improvement program.     It is too
                            early to say how effective       many of these actions will be; but, if ef-
                            fectively   pursued, they should result in better management. As GAO has
                            noted previously,     beneficial   results of some of these actions have be-
                            come apparent.
        I
        I                   The comments by DOD on this report express only a general reaction due
         II                 to the limited amount of time GAOwas able to allow for DOD review.
        1I
                            Because of the nature and importance of this subject, DODwants to ex-
          I                 amine the final report further.
          I ,

        I
        I

        1.          iWi"TE&I.S
                            FORCONSIDERATION
                                          BY THE CONGRESS
                            This report provides the Congress with an independent appraisal of the
                            complex problems associated with weapon systems development and procure-
                            ment by DOD--a matter of serious concern in the Congress.,
         I
        I           Tear Sheet
            I
                                                                 3
 COMPTROLLER
           GENERAL'S                    ACQUISITION OF MAJORWEAPONSYSTEMS
 REPORT
      TO THECONGRESS                    Department of Defense B-163058


 DIGEST
 ------

     WHYTHEREVIEWWASMADE
        The large investment required in recent years for acquisition of major
        weapons has impacted heavily on the resources available for other na-
        tional goals and priorities.

        Acquiring these major weapons involves substantial     long-range commitment
        of future expenditures.     Because of deep concern in the Congress on these
        matters and because of evidence that the weapon systems acquisition     pro-
        cess has serious weaknesses, the General Accounting Office (GAO) has un-
        dertaken to provide the Congress and the Department of Defense (DOD)
        with a continuing series of appraisals    of those factors most closely re-
        lated to effective   performance in procuring major weapons. This report
        represents GAO's first    such appraisal.

 r


s FINDINGSANDCONCLUSIONS
         1. Concurrent with GAO's studies,    over the last several months the Of-
            fice of the Secretary of Defense (OSD) and the military       services
            have been engaged in a substantial    effort  to identify   and solve prob-
            lems that have adversely affected the acquisition       of major weapon
            systems in terms of compromised performance, delayed availability,
            and increased costs.    GAO has found that generally the newer weapon
            procurements are following   a slower development pace and procurement
            practices are more conservative    than those of earlier    periods.   Be-
            cause many of the current programs are in early states of acquisition,
            evidence of the results of the changed concepts is not yet available
            to adequately assess them, but the outlook is brighter.

         2. The identification     of need for a weapon system and the relative
            priority    to be assigned its development is a fundamental problem in
            acquisition     of weapon systems.

            Initial  decisions as to which weapon system will be developed
            and the priority    of its development is made by any one of the mili-
            tary services,    but DOD has no organized method by which such pro-
            posals can be measured against its total needs. Such a method is
            now under development but it is in its infancy.

         3. In recent months, the Office of the Secretary of Defense and the mil-
            itary services have paid extensive attention  to the persistent prob-
            lems of defining performance characteristics  of weapon systems and
   of determining the technical feasibility of achieving that perfor-
   mance. There are many encouraging signs that these problems are
   being abated.

   Extensive efforts        are being applied--early    in the weapon development
   process--to     identifying     areas with high design risks and to con-
   structing    and testing the hardware itself       to demonstrate the feasi-
   bility    of high-risk      components before proceeding with further devel-
   opment.

4. In the preparation of and attention   given to cost-effectiveness
   determinations,   there was a wide range of quality.    This variation
   has lessened the value of these studies to the entire acquisition
   process.

5. One of the most important unresolved problems in the management of
   major acquisitions     is the problem of organization.     The essence of
   the problem appears to be attempts to combine the specialized         roles
   of major weapon systems acquisition       management into more or less
   traditional   military   command structures.    Because of this, there
   usually are a large number of organizations       not directly  involved
   which can only negatively     influence the project.

   It occurs to GAO that ideally       there should be a direct relationship
   between the missions for which weapon systems requirements are deter-
   mined; e.g., strategic    deterrent,     land warfare, ocean control,  etc.,
   and the organizational    structure     needed to acquire them. Such an
   arrangement would facilitate      grouping related weapon systems in pack-
   ages of common mission and would permit putting together an acquisi-
   tion organization    of appropriate size and stature to handle these
   matters.   Eventually,   GAO believes,     program management and organiza-
   tion will evolve along mission lines.

   There are other alternatives  involved, but whichever is chosen must
   clearly provide for someone to be in charge, to have authority    to
   make decisions and to have full responsibility   for the results.    The
   Deputy Secretary of Defense has recognized that the correction    of
   this problem is fundamental to any real improvement and has stated
   that he plans to pursue it aggressively.

6. GAO found that, on 61 weapon systems where complete cost data were
   available,     estimates to develop and produce the weapon system had
   increased some $33.4 billion.        About one third of this increase, or
   $9.5 billion,      represented the difference  between the estimate pre-
   pared when the system was first       approved for development (the plan-
   ning estimate) and an updated estimate prepared when the system was
   about to be placed under a development contract.         The remaining
   $23.9 billion      increase was due to changes in quantities   to be acquired
   and to a combination of such things as engineering changes, revisions
   to estimates,      and provisions for increased cost due to economic in-
   flation.      (See p- 58.)


                                  2
' RECO~NDATIONSORSUGGESTIONS
            The Secretary   of Defense should:

            1. Make every effort   to develop and perfect a     Department-wide method--
               now in its early stages of development--to       be followed by all mili-
               tary services for determining two things:        first,  what weapon systems
               are needed in relation   to the Department's     missions; second, what
               the priority  of each should be in relation      to other systems and their
               missions.

            2. Establish guidelines     and standards for the preparation  and utiliza-
               tion of cost-effectiveness      studies.  These guidelines should require
               that studies be updated and reviewed as part of the decision process
               when major changes in cost and/or performance require revised sched-
               ules for funding commitments.

            3. Place greater decisionmaking     authority for each major acquisition
               in a single organization   within the service concerned, with more
               direct control over the operations of weapon systems programs and
               with'sufficient status to overcome organizational       conflict    between
               weapon system managers and the traditional     functional     organization.

            4. Ensure that each selected acquisition    report (a) contain a summary
               statement regarding the overall acceptability    of the weapon for its
               mission, (b) recognize the relationships    of other weapon systems
    .          complementary to the subject systems, and (c) reflect     the current
               status of program accomplishment.


        AGENCY
             ACTIOK?ANDUNRli'SOLVED
                                 ISSUES
            DOD has been actively     pursuing a program to improve the management of
            the acquisition    of major weapons. The Deputy Secretary of Defense has
            assumed a significant     role in this improvement program.     It is too
            early to say how effective       many of these actions will be; but, if ef-
            fectively   pursued, they should result in better management. As GAO has
            noted previously,     beneficial   results of some of these actions have be-
            come apparent.

            The comments by DOD on this report express only a general reaction due
            to the limited  amount of time GAOwas able to allow for DOD review.
            Because of the nature and importance of this subject,  DOD wants to ex-
            amine the final report further.


        MATTERS
              FORCONSIDERATION
                            BY THE CONGRESS
.
            This report provides the Congress with an independent appraisal of the
            complex problems associated with weapon systems development and procure-
            ment by DOD--a matter of serious concern in the Congress.


                                                 3
                              CHAPTER1

                            INTRODUCTION

        The investment      to acquire major Department of Defense
(DOD) weapons impacts heavily          on allocation    of the Nation's
resources.        Acquiring   these major weapons involves     substan-
tial    long-range     commitment of futureexpenditures.       Because
of this and because evidence exists that the weapon systems
acquisition       process has not been conducted in an efficient
manner, there has been considerable           congressional   and public
attention      focused upon improving the process for acquiring
major weapon systems.

        In the past year, several studies of the acquisition
process for major weapon systems have been completed.               These
 include studies by the Department of Defense Blue Ribbon
Panel, the National     Security  Industrial      Association,    and the
Defense Science Board Task Force on Research and Development
Management.     All these studies were critical         of the systems
acquisition    process to some degree.         More recently   the Gov-
ernment Operations     Committee, House of Representatives,          held
hearings on policy changes in weapon systems acquisition.
The Committee report on this subject,           dated December 10,
1970, contained recommendations for several improvements and
the Commission on Government Procurement is including              major
acquisitions    as one of the subjects       in its study.

       Recently,   the Congress has called upon GAO to report
periodically     on the progress of various acquisition       programs
and to provide other forms of assistance       that would make
available    to its committees and members more reliable         infor-
mation on which to base judgments concerning        issues that in-
volve its oversight,     as well as its legislative     function.

        In order to effectively     respond to the interest  and
needs of the Congress to obtain more timely and comprehen-
sive data on which to base an evaluation         of the management
of ongoing procurements,        the General Accounting Office has
initiated    a long-term    program which will help provide data
for continuing     appraisal.
           This report presents the basic format which GAO intends
    to use in its long-term       evaluation.   The GAO program is an
    effort   to establish     an approach conducive to nurturing
    greater    agreement among the Congress, GAO, and DOD which
    will clarify     facts and issues and result     in improved man-
    agement of the acquistion       process.   Our intent   is to de-
    velop an orderly      process which will lead to a constantly
    improving body of basic data to assist all participants           in
    $he making of critical       weapon systems decisions.

           Another objective       of this GAO program is to provide a
    recurring    series of evaluations        of the weapon systems ac-
    quistion    process.     In these reiterations,       GAO will  (1) re-
    examine overall      acquisition     process efficiency     and (2) make
    detailed    and comprehensive examinations          of the process fol-
    lowed in most, if not all,          of the individual     major acquisi-
    tion programs.       The consistency      of format and the recurring
    nature of the evaluation         program should aid in the annual
    review of these acquisitions          by the Congress, as well as
    provide DOD with an independent assessment of the weapon
.   acquisition     process.

          Finally,  the GAO program is structured   for recognition
    and appraisal    of any improvement programs that DOD initiates
    for its acquisition    process.

          It is not the intention   of GAO to judge the propriety
    of technical    decisions made by DOD but rather to evaluate
    the efficiency    of the management and decisionmaking  pro-
    cesses applied.

    THE DEVELOPMENTPROCESSFOR
    A MAJOR WEAPONSYSTEM

          Developing major weapon systems is a primary function
    of DOD. The development process is highly     structured   and
    complex.    The combined process involves close interactions
    between needs of the user and the ability    of the developer
    to fulfill   them.

            A substantial   portion of personnel of OSD and the mili-
    tary services      are involved in the acquisition     process.
    Costs of weapon development consume a large portion            of the
    military    budget each year.    Large segments of industry       are
    engaged in producing the needed weapons.          More  than

                                        5
$150 billion   is estimated     to be necessary to acquire the
weapon systems currently      under development.         Some $95 bil-
lion of that amount is yet to be appropriated             by the Con-
gress.    An oversimplified     representation     of the manner in
which weapon systems evolve from an idea to production             is
shown in the following      chart.     (See figure   I.)

       Conceptual     phase--This     is the initial    phase in weapon
acquisition.        In this phase, need for new military          capabil-
ities     is established,    concepts are developed for a weapon
system which will provide those capabilities,,              and technical
feasibility      is explored and determined.          The objective    of
this phase is to provide the technical,              economic, and mili-
tary bases for initiating         full-scale     development of the
weapon system,        Advancement to the next phase, validation,
is dependent upon satisfying            criteria  designed to measure
achievement of the conceptual            phase's objective.

       There are six objectives              which should be accomplished
in the conceptual        phase.       First,     mission and performance
envelopes should be defined.                 Second, a thorough trade-off
analysis     must be made among the elements of cost, schedule,
and performance to ensure that the most effective                      product
is obtained when it is needed and at the most reasonable
cost.     Third,    a military       service must ensure that the best
technical      approaches have been selected for the new weapon
system.      Fourth, the service must provide assurance that
engineering      rather than experimental             effort   remains upper-
most in the program and that the needed technology                       is avail-
able.     Fifth,    that the cost effectiveness              of the proposed
weapon must have been determined to be favorable                      in relation
to the cost effectiveness             of competing systems on a DOD-wide
basis.     Sixth and last,         the service must ensure that,               inso-
far as it can, the cost and schedule estimates                     are both
credible     and acceptable.          When these prerequisite           criteria
have been fulfilled,           the weapon program is ready to go into
the validation        phase.      Secretary      of Defense approval is re-
quired to authorize          the program to move into the validation
phase.

       Validation    phase--In      this phase, the preliminary     de-
signs and engineering        for    the weapon system are verified      or
accomplished;      management      plans are made; proposals    for en-
gineering     development are       solicited  and evaluated;   and the
                                         6
H
development contractor           selected.    The objective    of this
phase is to verify         that the technical      and economic bases
for initiating       full-scale      development of the weapon system
are valid.      Advancement to the next phase, full-scale              de-
velopment,     depends upon establishment          of achievable    perfor-
mance specifications          for the weapon system that are sup-
ported by an acceptable           proposal from the development con-
tractor    selected.       Secretary     of Defense approval is re-
quired for the program to move into the development phase.

      Pull-scale     development--In     this phase, the design and
engineering     of the weapon system is accomplished.             The de-
velopment contract      is negotiated      and awarded; the prototype
of the weapon system is developed, produced, and tested;
and the detailed      specifications     for manufacturing      the
weapon system are prepared.           The objective   of this phase is
to develop a weapon system acceptable            for production,      Ad-
vancement to the production          phase must be authorized       by the
Secretary    of Defense.

      The development phase overlaps the production  phase
since development is not considered complete until   adequacy                    ..
of the production   model of the weapon system has been vali-
dated by a series of production   acceptance tests.

       Production--    In this phase, the weapon system is pro-
 duced in quantity       for deployment.        It begins when the pro-
duction contract       is negotiated     and awarded.        Production    ac-
ceptance tests are conducted to validate                the adequacy of
the production      model of the weapon system.             Quantity pro-
duction is initiated         and the first      operational    unit is
equipped with the weapon system and trained                 in its use.
Advancement to the operational           phase occurs when the first
operational     unit equipped with the weapon system is de-
ployed.     Production      continues,   however, until       all required
quantities     of the weapon system are produced.              The produc-
tion phase includes production           tests,    service acceptance
tests,    and user acceptance tests.

      Many potential     weapon systems never progress beyond the
early stages of consideration,       e.g., conceptual      phase.                . -
There are many reasons for this:          unavailability     of neces-
sary technology,     realization  that a potential       system may


                                      8
become too costly        for its intended purpose, anticipated
obsolescence      in terms of threat that the system is intended
to counter,     or another system concept subsequently          may com-
pete more effectively.           As a system passes through valida-
tion,    however, the Government's         commitment to it becomes
firmer.      By the time the system reaches full-scale           develop-
ment,    the  Government's      commitment   has become   so great   and
the structure       of the program so definite       that major adjust-
ments to the program are difficult            because they almost al-
ways delay critical         delivery   dates and are costly.       Few re-
ally acceptable       options are available      to the Government once
the design is approved and a decision             is made to begin pro-
duction.

      The pattern    of deeper involvement   and decreasing op-
tions is shown in the following       chart (figure   II).  The
greatest  opportunity    for broad decisions    occurs during the
early stages of acquisition.




                                    9
           H       .
       P
‘O-6       H




               ,       -
CONCEPTSOF      THIS    STUDY

       It was clear to GAO, when this study began, that the
underlying   management difficulties,      as well as the problems
of actually   executing  sound day-to-day     actions at all levels,
were probably deep seated and could best be evaluated by a
systematic   review of the entire     process by using specific
systems and phases as a basis for case studies.

         At the outset,   critical  major weapon acquisition      man-
agement actions and decisions,        which would occur in every
acquisition,      were outlined.    In determining   these critical
actions,      DOD's own criteria   and objectives  were used,       The
critical     management activities    examined pertain    to

       --requirements     for systems,
       --assessment     of technical   progress,        and
       --organization     and procedures.

       We selected specific         weapon systems now being acquired
on which to conduct reviews on the basis of the criteria
which had been developed.             Several factors       influenced       our
selection      of specific      weapon systems,      First,      we selected
some of the systems where the Congress or DOD would have
future options regarding           a further    course of action.            Sec-
ond, we selected a number of weapon systems which recently
proceeded into the early phase of the acquisition                     process.
This factor       is most important,       because problems occurring
in the earlier        phases may plague the system for years and
adversely      affect   the cost, schedule, and performance                of the
system at a point when adjustments              are difficult       to make.
As was noted earlier,           it is also the point in time when the
greatest      number of options are available           to both DOD and
the Congress,         Relatively     small sums of money are committed
at this stage, and therefore             it is easier to change the
direction      of a program.       As the program progresses,            how-
ever, choices will          decrease and the responsible           officials
will tend to become committed to a particular                    course of ac-
tion,    until    no options are left,         Although there is little
to be gained by dwelling           on problems which have occurred in
weapon systems where options were low, we have included a
few such systems in our study since they provide the best
means of assessing the full            import of sound as well as un-
sound past actions.

                                       11
      To fulfill    our task, 45 systems (14 Air Force, 14 Navy,
17 Army) were reviewed.       In addition, we reviewed cost and
schedule data from a number of other systems.        Still   other
systems have been reviewed at the request of congressional
committees.      In all,  the data in this report are distilled
from studies of some aspect of 70 weapon systems.

      In chapter 2, several of the management actions   criti-
cal to weapon systems acquisition  are described   in some de-
tail and are followed  by examples of good and poor perfor-
mance.

        In chapters 3 and 4, information collected in this
study on the costs and schedules of programs studied is
presented in summary form to provide a useful basis for
further    analysis.

     Chapter 5 contains       our observations,      conclusions,     and
recommendations.

,Scope

       In order to review current policies      and practices,   we
examined weapon systems which were in various phases of
acquisition    --conceptual, validation, full-scale     development,
or production.

        Information     on these programs was obtained by review-
ing plans, reports,          correspcxldence,    and other records and
by interviewing       officials     at the system program office,
intermediate       and higher commands throughout          the military
departments,       and the Office of the Secretary           of Defense,
We evaluated management policies              and the procedures and
controls     related    to the decisionmaking        process,   but we did
not make detailed         analyses or audits of the basic data sup-
porting     program documents.         We made no attempts to (1) as-
sess the military         threat   or the technology,      (2) develop
technological       approaches,     or (3) involve ourselves        in de-
cisions while they were being made.




                                    12
                              CHAPTER 2

           ASSESSMENTOF CRITICAL MANAGEMENTACTIONS

         In this chapter,   several of the management actions     crit-
ical to weapon systems acquisition        are described and are fol-
lowed by examples of adequate and inadequate application           of
criteria.       Although each example is based on an evaluation
of the management of the particular        weapon system procure-
ment which is cited,      these examples are mainly illustrative.
It is not the purpose of this report to focus on any partic-
ular acquisition.

REQUIREMENTS

        Establishing      requirements    for weapon systems is an in-
volved process.         It is the basis for getting         the system off
on the right       track and for controlling         the development pro-
cess.     The process begins with identification              of need for a
specific     capability      and proceeds through such steps as de-
fining    performance characteristics,           assessing the feasibil-
ity of achieving        them, establishing       some relative    priority
of need, and selecting           the system that promises to be most
cost effective.         Once requirements       have been firmly     estab-
lished,     a basis for important        actions during the development
process exists.         Such actions are controlling          changes, con-
tinually     making trade-offs       between performance and cost, and
controlling       system phasing and interfaces.          Requirements
provide a yardstick          against which action can be measured.

      The following   section outlines     our understanding of gen-
eral criteria    to be followed   in principal   acquisition process
steps and some illustrative      examples where the performance
in meeting the criteria      has been both good and bad.

Identification    of need for    a system

       The first   step in weapon system acquisition        includes
(1) evaluating     the products of documented military         department
threat   studies,    (2) projected    enemy force structures,     and
(3) operating     command statements     of requirements    and trans-
lating   them into specific      mission requirements    and technology
assessments.

                                    13
        The justification       for selecting      a particular     major
weapon system to fulfill           the need includes analysis          of con-
cepts of existing         and alternate      capabilities,      as well as the
establishment       of a relative     priority     of need.      The clear
identification        of new mission requirements          establishes     a
firm basis for initial         and subsequent weapon systems and
production.

      Key considerations       for   establishing    needs are:

      Threat studies--Prepare     future military   risk         positions
      and provide a justification       of future needs.

      Mission requirements--Define          system capabilities     in
      terms of specific    objectives       and tasks required    of the
      potential  system or systems,         including operational      and
      logistics  concepts.

      Current capabilities--Review       abilities      of existing   sys-
      tems or modifications      to them in relation       to defined
      mission requirements,      and identify      areas of required
      technological    advances.

      Technological    advances required--Analyze          alternate     tech-
      nical approaches and generally       identify       technical    risk
      areas in relation    to mission objectives.

      Tactical    concept of employment--Construct a detailed
      plan for the use of the weapon which sets the opera-
      tional   limits,

        We have not attempted to pass judgment on military
threat assessment; but we have examined methods that mili-
tary services have followed      to estimate current    capability,
to assess the potential      for technological   advancement,and
to apply priority    to a program relative     to other weapon ac-
quisitions.

       Following are some examples of instances where criteria
were adequately applied and some where they were not ade-
quately applied in determining    the need for the system.

     The success which some of the weapon programs achieve
in meeting their objectives for performance,  schedule, and

                                     14
cost confirms the usefulness of DOD's own criteria      for se-
lecting the specific weapon system to be acquired.
     A. Adequate application     of criteria
        S-3A
            The S-3A need was identified   and the decision
        to develop this weapon was made by a comprehensive
        analysis of future military requirements for carrier-
        based airborne antisubmarine warfare capability.        In
        the analysis,   the  Navy and  DODconsidered  mission   re-
        quirements, technological advancements required to
        develop the S-3A, and possible alternative.ways      of
        satisfying   the need. For example, the system that
        the S-3A is to replace was examined to determine
        whether it could meet the military    requirement and
        whether it could possibly be modified to meet the re-
        quirement. Also considered was the feasibility       of
        achieving technological advances needed to meet the
        performance planned for the S-3A program. (In addi-
        tion to this early attention to ensure technical
        success, actual commencementof the development cycle
        of the S-3A aircraft    was slowed considerably because
        of continuing reassessment of program priority      with
        the-land-based antisubmarine weapon system.)
     B. Inadequate application        of criteria
        1. LAMPS
             As early as 1957, the Navy stated a need to extend
        the weapon delivery range of destroyers to take ad-
        vantage of improved submarine detection capability.
        The Drone Anti-Submarine Helicopter program was first
        developed to fill   this need. (This program was can-
        celed later due to limited capability   and unreliabil-
        ity.)
            The Navy then considered filling   the need for an
        antisubmarine warfare (ASW) capability    on destroyers
        with a manned helicopter,   the Light Airborne ASWVe-
        hicle (LAAV). Shortly after LAAV entered the con-
        ceptual phase, however, it was canceled, and effort

                                 15
was directed    to development of the Light          Airborne
Multi-Purpose     System (LAMPS).

     This was done because it was felt within     the Navy
that OSD support for a strictly      ASW system would be
withheld.    In order to %ellt'   the system, the mission
profile   was expanded to include an Anti-Ship      Missile
Defense (ASMD) capability.      At this point ASMD was
added to the LAMPS capabilities      and was given prior-
ity over the ASW mission,

    Thus, although need for an ASW helicopter          had
been clearly     demonstrated  for a number of years, the
Navy decided to develop a multipurpose         helicopter.
This decision     led to 2-l/2 years of debate on how
these mission requirements-were      to be met within       the
weight restrictions      that had been imposed on the he-
licopter  because of anticipated     interface     problems
with the ship.

2. Mechanized    Infantry        Combat Vehicle   (MICV)

    The MICV project    began because of forecasts     of
threat to U.S. Forces and, in 1964, a change in Army
mechanized infantry     doctrine.     From 1964 to 1966, the
Army began a program to acquire an MICV for the
1960's--on   an urgent basis.       This effort  was dis-
continued because the vehicles       were too heavy, were
not mobile enough, and were not cost effective.           In
the meantime, the Army embarked upon a second pro-
gram, to acquire an MICV for the 1970's.          This pro-
gram had high priority,       since the vehicle was sched-
uled for early deployment.

      The schedule has not been met,     The vehicle   is
still    in the conceptual   phase, and deployment is ex-
pected to be 5 years later than originally        scheduled.
The program has been drawn out for various reasons,
including     the priority for available   fundsd

     The first   major delay came in defining    the vehi-
cle's mission and characteristics.       Approval of the
definition     was scheduled for March 1967 but was not
made until     October 1968.   The delay occurred,   in

                            16
part, because certain  studies were considered     inade-
quate and additional  work was required.      An impor-
tant factor was the complexity    and resultant   slow-
ness of the Army's decisionmaking    process.

     The October 1968 vehicle definition             assigned the
MICV "Priority       I" and called for development on an
"'urgent basis."       In mid-1969, the project          manager
and higher commands sought, without              success, Depart-
ment of the Army (DA) approval to move the MICV out
of the concept formulation         phase.       DA reviewed the
program in the light        of several factors        including
anticipated     budget cuts, an increase in the MUX's
estimated     cost, and possible use of alternative               ve-
hicles which did not exist at the time of earlier
studies.      It considered     several program alternatives,
including     (1) deferring     the program a year,(2)           de-
ferring    it a year and testing        additional      alterna-        '
tive vehicles,and        (3) terminating      the program and
developing one of the alternative             vehicles.       DA de-
cided to authorize        a review of all feasible           compet-
ing vehicle      systems before deciding         to move the
MICV out of concept formulation.

    During the new review, completed in April 1970,
the project   manager established     the concept of an
"austere"   MICV, which would have a lower cost be-
cause of deletion   of features    that the Army earlier
had termed "essential."      In July 1970, he and higher
commands recommended its adoption.        At the time of
our review, the DA decision was still       pending.

3. SAM-D

    Development of the SAM-D system began although
there was uncertainty        over the utility   of the system,
the character    of the threat which was to be coun-
tered, and the capabilities         of companion weapons
with which the system would operate.           Because of
these uncertainties,       in May 1967 the Secretary    of
Defense delayed the system's entry into full-scale
development.      Instead,    the system was placed in an
advanced development program to be conducted over a
3-year period.       After 2-years in the advanced

                           17
development phases the system was studied in March
1969 to determine whether it should enter full-
scale development.       The Deputy Secretary of Defense
directed    that the system be continued     in the ad-
vanced development phase through fiscal          year 1970
and that the decision      to place the system into full-
scale development be deferred until       fiscal       year 1971.
His position     was that the system would not be needed
until    sometime later,   the number'of batteries        needed
and how the system would be deployed in the field
was unknown, and the system was neither          fully    de-
fined nor justified,

    In March 1970, the Army subjected the system to
review by the Air Defense Evaluation            Board.    The
Board was directed        to again analyze the threat that
the system had to meet,to identify           the air defense
capabilities      required     to defend against this threat,
and to identify       existing    air defense capabilities
and deficiencies        to meet the threat,      The Board's
report was approved by the Chief of Staff on Novem-
ber 19, 1970, and, in essences confirmed the Army's
position     on the need for the SAM-D, As of December
1970, no action had been taken by OSD.




                        18
Definition      of performance characteristics
        Determination of weapon system operational requirements
and performance characteristics        (speed, range, accuracy,
etc.) depends on well-defined mission statements.           Perfor-
mance characteristics       are used to determine parameters of
trade-off studies, performance feasibility         studies, and
phased system acquisition projection.         Performance specifica-
tions prepared from these characteristics         are the basis for
initial    design feasibility    studies and validation    efforts.
       System design studies and development test programs
derive from performance specifications.      Absence of well-
defined system specifications     can cause underdesign or over-
design. Completion of the entire development process, with-
out actual satisfaction     of system mission requirements can
result from this absence. Conversely, it is more likely
that an explicit   definition   of systm performance character-
istics will result in an improved product.      This is not to
say, however, that system performance characteristics,      once
defined, must never be changed. This is an iterative       pro-
cess which becomes more firm as one approaches final design
for production.    Program management, to be effective,    should
allow for trade-offs as system development progresses and
for unanticipated technical unknowns which are surfaced by
detailed engineering design.
      Following aresomeexamples of instances where criteria
for defining performance characteristics  were adequately
applied and some where they were not.
      A. Adequate application       of criteria
             1. A-X
                   The definition of performance characteristics
             for the A-X weapon system flows from a clear, pre-
             cise statement of the mission this weapon system is
             intended to perform in support of the Army mission.
                   The A-X mission is defined as close supporting
             fire for ground forces, armed escort, and armed re-
             connaissance in battle areas. It was determined, by
             contractor studies, that an aircraft   with twin

                                    19
engines, capable of takeoff         and landing at forw&d
operating      bases, surviving   hits by light     antiair-
craft   artillery    projectiles,    having a rapid-fire      gun
and carrying      bombs or rockets;     having high subsonic
speeds and a range sufficient         for effective     close
air support,      is required.

       A minimum of avionics    for visual control     is to
 be included initially    with the weapon delivery.
 The aircraft,  however, is to be designed with extra
 space and power so that more sophisticated        avionics
 could bring its capability     up to all-weather    use,
 which the Army considers     essential.

         The Air Force awarded competitive          prototype
 development contracts          for the A-X close air support
 aircraft    on December 30, 1970.          This will give the
 Air Force actual hardware upon which to base a deci-
 sion for further       full-scale     development and should        .
 provide a sound basis uponwhich             to establish     firm       1
 performance     specifications.

 2. Heavy Lift    Helicopter     (HLH)

       The basic military    mission for HLJJ was articu-
lated by representatives      of the operating    command.
Some objectives     of the mission were revised,      how-
ever, during early attempts to gain approval.           One
revision    changed the mission emphasis from tactical
to logistical.      Another revision   resulted   from an
Army/Navy compromise initiated       by congressional
interest    in developing   an HLH that would satisfy
both the Army and Navy.

        Basic mission requirements     established by the
representatives     of operational    commands were in
clear and concise terms.        The change in mission        '
emphasis caused appropriate        changes in the mission
statements and subsequently        in performance character-
istics.

      Performance characteristics,   as well as all
changes for the first   two9 were developed by study
groups from various Army organizations.      These study

                          20
groups included representatives           from the field,       the
project    office,      and engineers with various       func-
tional    capabilities.       The characteristics      finally
selected were considered          the most desirable       to
achieve the established         mission.     Our review was
completed before the Army/Navy compromise was ap-
proved, but preliminary          studies were conducted in-
dividually      by Army and Navy engineers who examined
the compromise position.            Their results   indicated
that a compromise on performance characteristics
would limit      some of the mission requirements            of both
the Army and Navy.

      In September 1970, the Secretary  of Defense ap-
proved a program to develop high-risk   critical  com-
ponents for the HLH before full-scale   development
is approved, on the basis of performance require-
ments agreed to by the ArmyandNavy.

         This approach assumes that advanced technolog-
 ical development of the critical              components is nec-
essary to (1) determine whether technology                   is avail-
able to build such an aircraft              system and to iden-
tify     the best technical        approach offered       by the he-
licopter       industry     and (2) establish     realistic     cost
estimates.         Related studies concerning          further   re-
finement of the mission,            technology,     and economy of
the HLH will be made but will be subject to even
 further     refinement      on the basis of results         of the
 component development program.              If the critical
 component development is a success, this should per-
mit a decision          to be made whether or not to proceed
with full-scale           development.

3. HARPOON

        The HARPOONmissile       is a good example of the
Navy's thoroughness         in defining  the performance
characteristics       required   of a system.    The potential
enemy threat and the mission profile          of a new missile
to meet thisthreatwere          defined by the Chief of
Naval Operations        in June 1969. He specified      certain
restricting      design characteristics.      For example, the
missile     range required     and the range desired were

                            21
   specified.      The maximum weight of the missile        to
   be launched from a ship and from an aircraft           was
   given.      The requirement also specified       that the mis-
   sile have an all-weather     capability,     i.e.,   be able
   to hit a target under specified        adverse weather
   conditions.

          For a year, the Navy conducted numerous studies
   to determine how best to meet the requirement.              The
   aerodynamic qualities     of various,.missiPe      designs
   combined with different     kinds of propulsion        systems
   were studied to ascertain      whether the desired range
   could be obtained.     Additionally,      the reasonableness
   of the weight limitations      was verified      and studies
   were conducted of subsystems to find out if an effec-
   tive missile   could be made within       the limitation.
   The adequacy of the size of the warhead, which is
   one of the factors    having a direct       bearing on the
   weight, was tested by blowing up a number of obso-
   lete ships.

         Problems in selection   of a seeker       with all-
   weather capability   were anticipated.         Different
   kinds of seekers were tested in flight          before the
   kind of seeker desired was identified.            These studies
   and tests provide reasonable assurance          that the re-
   quirements   can be met before proposals        are solicited
   from contractors.

B. Inadequate   ap$ication        of criteria

   LAMPS

         Although performance characteristics     of the
   LAMPS have been under study since early in its de-
   velopment,   the Navy has had difficulty   in agreeing
   on the gross takeoff   weight of the helicopter,

            The significant    factor contributing     to this
   difficulty      is the fact that the program has been
   managed by various committees within            the Office of
   the Chief of Naval Operations         and had no consistent
   sponsor to guide and control        it.    Naval committee
   representatives        have varying vested interests      in

                             22
the program and, as a result, agreement has been
delayed on major questions such as the gross take-
off weight of the LAMPShelicopter.
      Committee representatives  from one organization
within the Navy were pressing for a heavier helicop-
ter.   In their opinion, a light aircraft    would not
have the desired mission capability.      Representatives
from another organization wanted a light helicopter
because it would fit on the DE-1052 class ship and
would be available to the fleet sooner. Controversy
centered around the question of whether the deck of
the DE-1052 was strong enough td support the helicop-
ter.
       Although the Deputy Chief of Naval Operations
(Air) requested data on the maximumdeck strength
of the DE-1052 early in the LAMPSprogram, testing
of the deck for maximumallowable landing weight
was not'accomplished until 2% years after the LAMPS
program was started.     The tests occurred in Novem-
ber 1970, shortly after the Deputy Chief of Naval
Operations (Fleet Operations and Readiness) succeeded
the Deputy Chief of Naval Operations (Air) as the
official    primarily responsible for the LAMPS.
     The Navy's long delay in specifying the weight
of the helicopter will result in a significant de-
lay in delivery of the UMPS to the fleet.




                       23
Obtaining   assurance of feasibility
of performance requirements

        The probability        of a technically        successful      develop-
ment depends upon an assessment of the availability                          of
proven technical         knowledge required         to build the item(s),
by identifying        design risk areas and assessing the likeli-
hood of resolving         them early in the development process.
Feasibility      of performance requirements              is usually assessed
as a part of conceptual            studies and confirmed during the
validation      phase,      Entering     into full-scale        development
without     establishing      design feasibility          can result       in at-
tempts to achieve unrealistic               technical     progress within          a
specific     test and schedule plan.             Positive     identification
of these design risk areas will permit the program manager
to facilitate       the system development process by bringing
his resources of men and money to bear upon critical                          ele-
ments and streamline          the development schedule.

      Following  are some examples of instances where criteria                          . ,
for obtaining   assurance of feasibility  of performance re-
quirements were adequately   applied and some where they were                           .
not.

       A. Adequate      application      of criteria

           1. AEGIS missile         system
                  In the cases of the AEGIS missile        system, a
          group of highly qualified        people from the Navy and
           industry    performed a risk analysis      as part of a
          comprehensive missile       system study.      In evaluating
          results,     the Navy directed    a laboratory    model to be
          constructed      and tested to demonstrate the feasibil-
          ity of high-risk       components before proceeding fur-
          ther with development.        Added assurance that the
          system was technically       feasible   was obtained
          through an independent evaluation.

                 The successful  demonstration    of the highest
          technical   risk component has been established      as                      . .
          the first   critical  milestone   in the current   engi-
          neering development contract.


                                        24
2. Heavy Lift      Helicopter   (HLH)
      The HLH is planned to lift heavy loads over
short distances in support of combat missions and
peacetime operations.     The HLH is planned as an im-
provement in lift capability    over present transport
and flying crane aircraft,    and has no counterpart
in the current Department of Defense inventroy.
      The development approach for the HLH differs
from many major system acquisitions  in that the
early phase of the acquisition process includes de-
velopment of critical  hardware in contrast to paper
studies.
     The Army has identified  high-risk components
for immediate development effort.     If the critical
component development is a success, full-scale      de-
velopment can proceed.
3. F-14 Aircraft
      The F-14 aircraft is composed of three basic
subsystems; namely, avionics, propulsion, and air-
frame. The potential risks in developing each sub-
system were studied and analyzed by the Navy before
proposals were received from interested bidders.
The Navy analysis indicated that the risks associ-
ated with the avionics and propulsion subsystems
were low because these subsystems had been devel-
oped for use on another aircraft.    For example, the
engine to be used on the F-14 was available and was
tested on the ground in a simulated F-14.
       The airframe was considered a normal develop-
ment risk although various potential problems were
identified.      Plans were developed to resolve poten-
tial problems including identification     of possible
backup items which could be used to provide an in-
terim capability,     if required.
      The Navy also used risk analysis in consider-
ing the reasonableness of the contractors'  propos-
als. Therefore, when the Navy entered the

                           25
   development and production contract, risk had been
   minimized.   Identification  of program risk also en-
   abled the project manager to more adequately moni-
   tor development of the airframe.
   4, Airborne Warning and Control System (AWACS)
          In the AWACS,the high-risk area was identified
   as the overland radar, the extent of which will be
   determined by building and testing actual hardware.
   Actual demonstration that the AWACS,including the
   overland radar subsystem, will work as intended is
   stipulated as a condition of continuing develop-
   ment.
         Two competing overland radar systems will be
   developed, and a fly-off   <ompetition with AWACS
   configured aircraft  held, to demonstrate their re-
   spective merits and detect shortcomings.    If a suc-
   cessful system is demonstrated, the AWACSprogram         .
   will be allowed to begin the remainder of the full-
   scale development program.
B. Inadequate application        of criteria
   1. DRAGON
         The DRAGON weapon system was approved for full-
  scale development before essential technology was
  available to correct major system limitations.     At
  that-time,   the development of a required night
  sight was not believed to be within the state of
  the art.    Other technical risks (i,e.,  friendly
  electronic interference and enemy countermeasures>
  had not been assessed as thoroughly as was pos-
  sible.
        Several DODreview groups, while acknowledging           _
  major system difficulties  and performance limita-
  tions, recommended an accelerated development
  schedule for limited production of the system.           . -
  Operational need was stated as a basis for these
  recommendations.


                            26
      Demonstrated    DRAGON performance      has not   met
requirements     contained in the Army’s Qualitative
Materiel     Requirement.       This problem comes from
failure    to properly      assess high technical    risks,
and from granting        approval   to proceed prior     to the
resolution     of risks.      This is contrary    to DOD rules.
2. Short Range Attack Missile        (SRAM)
      The feasibility of developing the motor re-
quired for SlUM was not adequately considered be-
fore commencementof full-scale    development. The
missile motor represented a high-risk area that
neither the Air Force nor its contractors ad,e-
quately evaluated during the contract definition
phase.
      After award of the full-scale development con-
tract, the contractors concluded that the rocket
motor required to meet design and performance con-
tract specifications  was beyond the state of the
art.   The Air Force now estimates that the rocket
motor planned for production will have a total im-
pulse less than expected at the time of the devel-
opment contract award. Performance was thus com-
promised. Development of the rocket motor delayed
completion of system development several years and
raised costs as well.
3. C-5A aircraft
       Similar to the SRAMprogram experience, the
C-5A program encountered technical difficulties
which were appraised but which may not have been
adequately recognized at higher levels during the
validation    process. These technical problems
proved difficult    and costly to resolve and caused
cost growth and schedule slippages.
     In the case of the C-5A, the development
schedule also was unrealistic.   At the outset it
was overly optimistic with no allowance for set-
backs in the development program.


                         27
Cost-effectiveness      determinations

       Cost-effectiveness     studies are one accepted means of
selecting     a system.    They are particularly   useful during
concept formulation       of a weapon program.    Systems selected
for consideration       should include equipment already in inven-
tory and should specify        the degree to which such systems
provide needed mission capability.

        A cost-effectiveness       study considers      the need that a
system is supposed to fill,           the alternative      technical    solu-
tions that are available         to meet that need, technical           per-
formance characteristics         of each alternative,         cost associ-
ated with each possible         solution,    and criteria      for choosing
among alternatives.          The overall   study should emphasize sig-
nificant     issues to clarify      merits of alternative         systems.
Also, the analysis        should be updated when changes in basic
assumptions occur.         Updating ensures continuing          cost effec-
tiveness    of the system selected by allowing for changes in
threat,    technological      advancement, or desired level of de-
fense.

       GAO's examination      was limited     to the questions whether
(1) the military     service had fulfilled         a requirement  that
cost-effectiveness       studies be performed,(2)        studies had
been made of competing equipment systems, (3) each study was
evaluated,     used, and became part of the competing weapon
program records,     and (4) realistic        equipment operating    envi-
ronments and personnel training           levels were included as
conditions     for performance of the equipment end-item.

        Cost-effectiveness       studies provide a measure for eval-
uating changes as the program proceeds and for making con-
tinuing     trade-offs     between cost and performance.         With such
studies,      we have a technique by which balance can be main-
tained between cost and performance.                Without such studies,  ,
ill-advised      program decisions      affecting     performance and
schedules can seriously         jeopardize      program cost estimates.

      Following are some examples of instances where criteria
for performing  cost-effectiveness studies were adequately
applied and some where they were not.



                                    28
A. Adequate application   of criteria
   1. DD-963
         Ihe program for development and production of
   new destroyers (DD-963) to replace World War II
   ships was initiated    in August 1966. In September
   1967, the Office of the Chief of Naval Operations
   completed a study comparing the cost and antisub-
   marine warfare effectiveness of the DD-963 class
   destroyer with alternatives.      Formal approval to
   enter contract definition     for the DD-963 was granted
   in February 1968.
         The scope of the DD-963 cost-effectiveness
   study included a comparison of existing, modernized,
   and new design destroyers.      Results of the DD-963
   study showed that the DD-963 could provide antisub-
   marine warfare effectiveness with substantially.
   fewer ships at a lower life-cycle       cost, and at ap-
   proximately the same total investment cost as any
   alternative   ship.
         The DD-963 study relied heavily upon previous
   studies for such things as postulation of threat
   and estimate of differences in effectiveness among
   the various antisubmarine warfare components used
   on the competing ships. Since our review did not
   encompass earlier studies, we did not determine rea-
   sonableness of the assumptions used in the DD-963
   study regarding these or other significant        aspects
   of the cost-effectiveness     question.     Nevertheless,
   our limited review has shown that the Navy (1) pre-
   pared the DD-963, cost-effectiveness      study early in
   the acquisition process,(Z) considered a number of
   alternative   systems in DOD's inventory,and (3) ap-
   parently selected the most cost-effective        alter-
   native.
   2. AEGIS
        A cost-effectiveness     study of the AEGIS Ad-
   vanced Surface Missile System was made in early 1965.
   The principal characteristics      of the missile system
   reconrmendedfor development at that time were es-

                          29
sentially   the same as those currently           approved for
development of what is now called the AEGIS system.
In that study9 comparisons were made of the perfor-
mance of various     individual     systems in a wide vari-
ety of tactical    situations     and of alternative       com-
binations   of systems providing        for the defense of
specific  naval forces.       Alternative      systems eval-
uated included existing       Navy missile       systems, air-
craft equipped with air-to-air          missiles,     and several
versions  of the AEGIS system,,

      Costs of alternatives      considered    included devel-
opment, investment,     and annual operating       costs,     The
cost and effectiveness       of alternatives    were compared,
The conclusion    showed the AEGIS system to be supe-
rior in cost effectiveness       to the alternatives.
Although a formal updated cost-effectiveness            study
was not prepared,    the Development Concept Paper sub-
mitted to the Defense Systems Acquisition           Review
Council following    receipt    and evaluation     of contrac-
tors? proposals,    compared the cost and effectiveness
of alternate   systems against various threats.

       Although we did not question validity       of basic
assumptions,    we believe methods used in this study
conform to acceptable     cost-effectiveness-study     prac-
tices.

3. Armored Reconnaissance         Scout Vehicle      (SCaVT>

        A cost-effectiveness        determination      was made by
comparing threat,        mission,     and effectiveness      analyses
with schedule, cost, and feasibility               studies.    By
providing     seven different       firms with such data as
scope of work, description            of the system, vehicle
design, etc. 9 various concept designs were sub2
mitted.      These designs were consolidated            with in-
house effort,       and the results       were furnished     to a
research firm.         In addition,     Army research organi-
zations supplied auxiliary            data on threat analysis.
In the assessment of design, cost, and combat effec-
tiveness     (parametric     design/cost-effectiveness
study),    the research firm analyzed (1) effectiveness
evaluations      of nine concept vehicles          and eight

                            30
   reference     vehicles     in computer simulations       of repre-
   sentative     missions and (2) life-cycle          costs of each
   of the 17 candidate          systems.     The comparisons    of
   effectiveness       required     evaluation   of the candidate
   vehicles     in different       threat and geographical      envi-
   ronments as well as in the performance of two dif-
   ferent    types of mission--security          and reconnais-
   sance.

B. Inadequate    application        of criteria-

   1. A-X aircraft

          The Air Force cost-effectiveness        studies for
   the A-X aircraft      considered    only the A-W, A-7D,
   A-37B, F-4C/D, and the improved OV-10. These are
   all Air Force fixed-wing        type aircraft.      Such pos-
   sible candidates      as the Army's AH-56 Cheyenne he-
   licopter,    the Marine's    AH-LJ Cobra helicopter,       and
   the Marine's    Harrier--   a vertical/short     takeoff   and
   landing aircraft--     -were not covered in the studies.
   Also excluded is a more expensive version of the
   A-X which incorporates       an all-weather    capability.

   2. A-7D aircraft

          The Air Force recommended, on the basis of
   cost-effectiveness     studies of existing   DOD-wide com-
   peting systems, the procurement of a slightly       mod-
   ified version of the Navy's A-7 in-production        air-
   craft    to fulfill its need for close ground support
   missions and interdiction      in future years.

          Subsequent to these studies and DOD's approval
   of the procurement,     major configuration        changes in
   avionics    invalidated  initial     plans and contributed
   to the Air Force procurement         of a more sophisticated
   and expensive aircraft.          The average unit weapon
   system cost increased about 110 percent between
   DOD's approval in November 1965 and June 30, 1970.
   These changes also contributed          to delay in estab-
   lishing   firm detail specificationsand         attaining
   delivery    schedules.   We were informed by Air Force
   personnel     that the cost effectiveness       of the A-7D

                               31
 was not revalidated      to determine if the aircraft  was
 still  cost effective      after these changes were made
 when compared with      competing existing  systems.

         The principal    management weakness in adminis-
  tering   this program was failure        to give formal rec-
I ognition    in the management process to the effect         of
  these changes.       Such recognition      would have subjected
  the revised A-7D plan to the same basic decision-
  making process as the initial         plan, including   a val-
  uable cost-effectiveness       comparison of the changed
  A-7D configuration      with other DOD systems.

 3. A-7E aircraft

          Similarly,     the Navy A-7 aircraft     program began
  with the Navy A-7A version,          which was developed to
I fulfill    the requirement      for light   attack aircraft
  with increased range and load carrying            capability
  to replace the all-weather          A-4E.   This A-7A was sub-
  jected'to      the full-scale     development cycle.       The
  next Navy version,         the A-7B, was basically     the same
  asthe     A-7A except for a different        engine which pro-
.vided increased acceleration            and decreased takeoff
y digtance.

         In developing     the "E" version,    the Navy started
 with its existing       "B" version and developed a sig-
 nificantly     improved light attack aircraft.           The
 Specific    Operational      Requirement &OR) for this air-
 craft    has not been changed,        However, the improved
 avionics    system and engine of the A-7E represent
 significant     advances in the military       capabilities
 over the A-?B, one being increased bombing accuracy.
 Like the A-7D above, the cost effectiveness              of the
 ultimate    A-7E configuration       was not validated.




                          32
Stability    of the program and its
relationship    to other programs

       Effective    pursuit     of program   objectives     requires      stabit-
ity   of priorit      2nd of aZZo,cation     of all. critical       resources
in combination      with    eZarity   and consistency      of program      diree-
tion.

       The discipline      imposed by OSD and the service secre-
taries   upon the military       services'    weapon acquisition      organi-
zations has helped to bring about a more orderly                 management
process.     The rather long period of time required              for acqui-
sition   has been broken down into logical           stages.      Comprehen-
sive criteria     have been established         for an acceptable     pro-
posal for a program to advance from one stage such as the
conceptual    phase to the next.         Detailed   OSD direction     has
been given to the military         services on all aspects of pro-
curement, such as 'bake-or-buy,"           national   priorities     and de-
fense materials       systems, and small business set-aside.

         The rigorous    structuring    and close management control
mentioned above do not address the question of need for, and
priority     of, a specific       weapon acquisition    program relative
to others.       That process of determination        and execution     of
the relative       need/priority     is accomplished principally
through the formulation           of budget and is reconsidered       annu-
ally for each weapon program, in each appropriation               in-
volved, with consequent instability            permeating    all program
direction.

      The impact of instability        is illustrated       by the reduc-
tion in capability     experienced     by the Defense Satellite             Com-
munications  System and its associated            earth terminals.          In
another USAF mission area, the bomber air defense system,
in which provision     was made to systematically           develop, pro-
cure, and deploy the system components in a preplanned,                     well-
organized manner, the principal          components are the OTH-B
radar, AWACS, F-15 air superiority           fighter,    or an as yet un-
determined interceptor      fighter.      For two system components
 (OTH-B & AWACS), the initial        operational      capability      (IOC)
dates do not coincide and the full           operational       capability
dates bear no relation      to one another.         An overa       aequisi-
tion    management     plan,  with provisions       for     integration    and
coordination       of mutuaZZy    interdependent          weapons required      for
mission     performance,     and an interrelated           air defense   testing
                                       33
 program         to      evaluate      accompZishment                 of    continenta      air   de-
 fense        wouZd        ame Ziorate      this   condition.


      The establishment        of a comprehensive priority         system
for weapon acquisition         programs is an involved process.
This is particularly        true for weapon systems which fall out-
side the category of "Strategic           1A" programs,      (generally
"super-systems"      such as the ABM, POSEIDON, and MINUTEMAN for
which high-level      attention     is readily   available).       Applica-
tion of the ranking to other weapon acquisition                programs is
even less formal and specific.

         At present,      there is a DOD-wide priority        system which
 allocates     certain      scarce resources among the competing needs
 of the individual          ongoing acquisition     programs.     This     prior-
itz4 ” system   is deficient       in two respects;     it is not    uniformZy
appZied       within        each   of the servi.ees           (although        it     is reason-
a5Z2    ~21’:     ctpplied       to conflicting         needs       between       programs
which     are     in different           services)      and it        tends    to deaZ       onZy
with    certain          Zimited     categories       of resources           (such       as mate-
rials     gbich       are     in short      suppzy)     and ignores         the      more   crit-
icaZ    resoxrees           such   as overaZZ       funding         and personnez.


      Within the military  departments,                                     some sort of priority
ranking system does exist;   its value                                     has not yet been proven.
We ,h eZieve            that       the      devezopment          of a comprehensive           DOD-wide
priorit           system           is a first           step     toward    alleviating        an impor-
tant       part    of          the   diffieuzty           we   found    in DOD’s       management
procedures,    Of course, an indisputable        priority      is estab-
lished weapon system by weapon system through the annual bud-
get review cycle.      These budget-derived        priorities,       however,
are not converted     into a DOD-wide comprehensive priority
rating   which would also determine each program's               relative
priority   for all critical     resources.      Also, insofar        as we
can see, theras      no effective      connection     between these bud-
get decisions    and some longer range view which contrasts
each potential    acquisition    against a master plan of overall
mission requirements       and available   or developing         capabili-
ties of all the services.

      The Office of the Secretary   of Defense has recently        im-                                    .
plemented a new approach to analyzing     the plans for a weapon
in terms of the relevant  military   mission category such.as                                             .
"land warfare .'I This analysis,   which includes   identification
                                                          34
of major issues, is to provide the Secretary of Defense with
a broad overview of each mission category.       It also is in-
tended to provide guidance for weapon acquisition       to military
departments and agencies which develop the programs for
equipment to improve military    effectiveness.    Additional ob-
jectives of the procedure are to eliminate competing systems,
phase out obsolete equipment, identify deficiencies       in ca-
pabilities    of the forces, establish performance character-
istics needed, and set schedules for carrying out guidance.
It is expected that the analytical      procedure will raise and
resolve major issues inherent in and between mission cate-
gories.    Although this procedure appears to satisfy many of
the essentials of an overall priority      system, it is still    in
its infancy.




                                35
 Subsystem development        phasing      and interfacing

      The constituent    subsystems of a weapon system must be
available   and compatible    or system development will not be
successful,

       When a weapon program includes   development of a sub-
system with high technical    risk,  the weapon program is sus-
ceptible   to slippage.   When the subsystem development is
out of phase with the development of the overall       system,
that system may be compromised in either      schedule or per-
formance$ or both,

        The mismatch of subsystems with the parent system ap-
pears to occur most frequently      when responsibility   for de-
velopment of parts of the system.is       divided among two or
more project    managers.  The difficulty     is compounded when a
subsystem is common to more than one weapon system yet sep-
arately    managed.

       Specific   provision    must be made to ensure that develop-
ment and acquisition        of the subsystem will coincide with
technical    requirements     of each of the weapon systems for
which it is to be used,          The same considerations  of phasing
and interfacing      are applicable    to a weapon system such as
SRAMwhich must work in conjunction          with systems such as
the B-52, B-l, and F'B-111 bomber aircraft.

         The increasing     complexity     of weapon systems has neces-
 sitated    increasingly     detailed,     close control      over design,
development,. and production           of the system by the program
manager.       He must give informed technical            and administrative
direction      to ensure that proper provision            is made for con-
trol of development phasing and interfacing,                   He must re-
quire performing        organizations      to (1) identify       and document
the functional       and physical      characteristics      of the weapon
system and its subsystems,            (2) rigorously      control    changes
to those characteristics,           and (3) record and report all per-
tinent     aspects of the progress of system components and any
changes to them,         This quality      of direction     and control       by
the program manager is necessary to achieve integrity                     and
continuity       of design for technical        performance,      producibil-
ity,    operability,     and supportability         of the overall     system,


                                      36
       To reiterate,   developing   subsystems must be kept in
phase with one another to make sure they will work together
and will be available     when needed, or cost growth and
schedule slippage will generally       occur.  Imbalance in de-
velopment of subsystems can also cause shortfalls        from
performance objectives      for the weapon system; that is, de-
lay in the achievement of, or incompatibility        among, con-
stituent    subsystems of the weapon, or related     weapons, may
impair performance of its mission.

        Following   are some examples of instances   where criteria
were adequately      applied and some where they were not ade-
quately     applied  in  subsystem development phasing and inter-
facing.

     A, Adequate      application      of criteria

         F-15/B-l

                 The F-15 and B-l programs incorporate                  manage-
         ment concepts intended to guard against or minimize
         the effects       of pitfalls       which have been encountered
         in other major acquisition              programs, through use of
         total    system responsibility            and demonstration          mile-
         stone provisions.           A "total      system performance re-
         sponsibility"        clause has been incorporated              in the
         F-15 contract,        which makes the airframe             contractor
         responsible       for integration         of the complete weapon
         system as well as for all actions necessary to en-
         sure that the total           weapon system will meet perfor-
         mance requirements           set forth in the system specifi-
         cation.       In essence, the Government looks only to
         the airframe        contractor      for satisfactory         perfor-
         mance of the aircraft             and does not become involved
         in any problems concerning the engine or the sub-
         systems.        Contractor-to-contractor            relationships
         necessary to fulfill            interface     plan commitments are
         set forth       in associate       contractor     agreements be-
         tween the prime contractor               and his associate         con-
         tractors.        A similar      approach has been incorporated
         into the B-l program wherein the airframe                     and en-
         gine contractors         are working on an associate               con-
         tractor     basis, but the airframe             contractor     has total
         system integration           responsibility.

                                       37
           Under the demonstration          milestone    provision,
   planned dates for accomplishment               of specified
   technical       milestones      are established.      The Air
   Force will determine whether the contractor                  has
    satisfactorily        accomplished the milestones.            The
   accomplishment         of the milestone      is contractually
   tied in with Government allocation               of production
   funds.       Failure to meet a milestone          may result       in
   a delay in the funding of a production                increment,
   a delay in exercise            of the option to which the dem-
   onstration       milestone      relates,   or a partial     allot-
   ment to sustain minimum production               at the Gover-
   mentqs option.          Any schedule adjustments         due to de-
   lays will be made with no change in initial                  target
   cost or ceiling         price.      None of the milestones         were
   scheduled to be accomplished at the time our review
   was completed.

B. Inadequate     application        of criteria

   1. CHAPARRAL/VULCAN

           The CHAPARRAL/VULCANair defense system was
   produced and deployed without        the Forward Area
   Alerting    Radar System (FMRS) which, coupled with
   other significant    performance limitations,      resulted
   in the system!s providing      limited   air defense capa-
   bility.

           When limited    production        of the CHAPARRALand
   VULCAN systems was approved in November 1965 and
   March 1966, respectively,            the Army had not designed
   or developed the military            characteristics       for the
   system's radars even though it had determined in
   1965 that existing        systems could not be modified to
   fulfill    the radar's     mission.        Production     of the ra-
   dar was authorized        in 1968, though earlier           testing
   indicated     that it did not meet performance repuire-
   ments.     When technical       difficulties       arose, radar
   production      was stopped in July 1969,             This resulted
   in the deployment of the CHAPARRAL/VULCANsystem
   without    FAARS. The present system requires                 the.op-
   eratorss    visual    detection      and identification         of
   en&my aircraft       and his judgment that they are within                '
   range.
                                38
2. AN/BQQ-2 integrated         sonar   system
      The AN/BQQ-2 integrated      sonar system is a com-
plex system designed for installation        and use
aboard nuclear attack submarines.         The accomplish-
ment of the sonar and submarine projects        is the
responsibility    of different    project  managers in the
Naval Ships System Command, Successful         accomplish-
ment entails   integrating     the two systems at a pre-
determined point in time.

       Performance and physical        characteristics      of
the two systems had been identified;            but develop-
ment and production       schedules for the two systems
were out of balance.         The sonar was acquired under
an accelerated      program to permit delivery         at the
predetermined     time that a ship would be ready to
accept it.     This precluded      an orderly     design, de-
velopment,    and production      of the sonar system and
resulted    in technical     problems.    Technical     problems
delayed delivery       of the sonar system.         The sonar
delay resulted      in a disruption     of Navy shipbuilding
schedules and in cost growth.

      The problem experienced with the sonar system
development phasing and its ultimate     interfacing
with submarines was magnified   because each weapon
system had its own project   management.                           1
       The Navy has now established        a ship project
directive    system which provides      ship acquisition
project    managers with procedures      for directing     man-
agement actions      of secondary managers to ensure
proper integration      of the total    shipbuilding    pro-
gram. Thus, definitive         tasking,  scheduling,     and
funding for all support elements is effected.              This
policy    should help significantly      to prevent the
situation    described    in the AN/BQQ-2 sonar example.

3. P-3C aircraft

      Development interface         and subsystem phasing
problems were encountered         in the P-3C program


                          39
because the technical       feasibility         of certain      sub-
systems planned for the program had not been fully
proven when the development program was approved.
For instance,      a succession of three different               ver-
sions of the acoustic       signal processor has been
attempted since program approval.                The last of
these, the one now included in the program, is
known as DIFAR. The decision to incorporate                    the
DIFAR processor      gave additional         capability      to the
P-3C, but it made the problem of interfacing                   and
phasing of development,more           difficult      because the
processor was still       under development and was not
available    until   about a year after P-3C deliveries
began.     The result    has been a stretch-out            in P-3C
testing    and the delivery     to the fleet          of aircraft
short of desired equipment.            This equipment had to
be backfitted      as processor production           caught up
with need.




                           40
Continuous trade-off         between
cost and performance

        As early as preparation      of a design is completed for
the weapon system, the program manager should initiate                the
iterative     process of examining each proposed change in capa-
bility    for the weapon against its associated         costs.     His
analysis     should include estimates      of technical   feasibility
of the design features        of the proposed change, probable im-
pact on the logistics        and schedule,   and cost of the capabil-
ity in relation      to military    need.

      A continuous   trade-off    between performance  and cost
during the acquisition      process will  keep aZZ elements in
balance.
     Flexibility      during development        is important.     The Dep-
uty Secretary      of Defense has stated        that

      The cost      of developing   and acquiring new weapon
      systems is     more dependent upon making practical
      trade-offs     between the stated operating   require-
      ments and     engineering   design than upon any other
      factor."
He has stated      further   that
      "trade-offs  must be considered not only             at the
      beginning of the program but continually              through-
      out the development stage."
Budget constraints have forced trade-offs    of the nature de-
scribed and have ensured continuing   implementation  of this
philosophy.

       Following    is an example of an instance          where criteria
for trade-offs      between cost and performance          were adequately
applied.
      A. Adequate      application     of criteria
          F-15 aircraft

                The Deputy Secretary of Defense advised the
          Air Force in September 1969 that production   funds
          for the F-15 program would be limited  and that ac-
          ceptable performance cost trade-offs  must be

                                       41
determined     so that F-15 costs would be within         the
approved program.        The Air Force began a cost reduc-
tion study based on assumptions that the F-15 engine
and airframe would be unchanged and that the devel-
opment phase would begin as originally            scheduled.
The study produced savings which brought the program
within   funding constraints      while retaining     an ac-
ceptable operational       and growth capability.        The
largest   cost reductions      were realized    in avionics.
Such changes as reducing range and ground map re-
quirements    for the radar, reducing redundancy in
computation,     reducing communications      and navigation
requirements     and equipment,    and a decrease in the
amount of initial      spares were effected.        The esti-
mated unit cost of production         was reduced about
$1.5 million     per aircraft.     Provisions    were retained
in the aircraft     to permit the reinstatement        of some
hardware items at a later date, if feasible.

      Additional   trade-off  studies of the F-15 have
been made to meet Air Force heeds more economically
since the award of the development contract.         The
Air Force plans to continue its review of the F-15
program throughout     the development phase for pos-
sible reductions     in cost, weight,  and complexity.




                        42
TECHNICAL ASSESSMENT

      One of the results  we observed of DOD's efforts    to im-
prove its weapon system acquisition    process was the in-
creased use of test resul,ts to anticipate    specific techni-
cal difficulties.

         The conduct of specified       tests and use of their re-
sults under current management concepts are incorporated
into recent acquisitions          programs, such as the F-15 and B-l
aircraft     programs.    Clarification      of assessment of techni-
cal development is part of the implementation               of total    sys-
tem responsibility;       of milestone      demonstration;     and of
thresholds      for cost, schedule, and technical          performance.

      Technical feasibility   studies pinpoint   technical high-
risk areas.    Special emphasis is now being given to mini-
mizing these risks,    and special testing   is used to monitor
planned progress.

      Tests are a valuable means of assessing subsystems and
system design progress.    Test results also provide a com-
parison of actual progress with the planned progress.

       Test results    provide management with information        on
which to base decisions       such as to modify a design approach
or to change*basic      system development plans.       The results
of successful     tests also can be used to curtail       design ef-
forts when sufficient       confidence   is gained to support a de-
cision to proceed with production          or to accept hardware for
operational     use.   With inadequate data from test results,
judgments of this kind become more subjective           and suscep-
tible   to a greater degree of error.         Omission of tests can
result    in the production     of hardware that does not meet re-
quirements.

     Following  are some examples of instances where criteria
for technical  assessment were adequately  applied and some
where they were not.




                                     43
A. Adequate   application         of criteria

   1. DRAGONmissile

          The Army's technical        development test plan for
   the DRAGONmissile         has demonstrated      that DRAGON's
   performance has not met established             requirements
   for reliability        and single-shot    lcill probability.
   Additional      technical   problems were also revealed
   through testing.

         Test results   that .the Army is using may influ-
   ence plans to let a limited     production    contract
   prior  to completion    of all service tests,

   2. Improved    HAWKmissile

         High-risk      areas in technical        objectives      of
   this missile,      for which extraordinary          management '           .
   action was required,        were identified        in November
   1968. At that time the Improved HAWKsystem en-
   tered an engineering        test/service       test program.
   The test program was scheduled to continue through                             .
   1971, Flight      tests were halted in December 1969
  because a component failed            to function      properly,
  This component was modified             and additional       test ob-
  jectives    were prescribed,         An 18-month development
  program was instituted          to develop another camp.onent
  as an alternative.         In early 1970, flight           testing
  was resumed but only limited             success was obtained
  in meeting the objectives,             Conclusive      data have
  not been obtained on flights             against low-altitude
  targets,maneuveringtargets,              high-speed targets,
  long-range     targets,    and electronic        countermeasure
  environments.

        To decide whether a production         contract  should
  be awarded for FY 1970 and FY 1971, the project
  manager had a risk analysis       performed.      Completed
  in April    1970, the analysis     included an evaluation
  of technical,     cost, and schedule data on the Im-
  proved HAWK system.       A component was assessed as
  a high-risk    item.    To minimize this rislc, modifica-               .
  tions to the component were proposed.           After

                             44
   evaluation   of these modifications,    other perfor-
   mance risks,   and increased costs that would be in-
   curred by delaying procurement,      the project  manager
   recommended the immediate award of a production       con-
   tract.

   3. MAVERICK missile

           Production    options included in the contract
   for MAVERICK development were to be exercised                    before
   scheduled completion         of tests conducted by the con-
   tractor.      Additional     provisions     afforded     the Air
   Force opportunity         to delay exercising        production
   options for 420 days,         upon   payment     of  stipulated
   standby costs.        This option period extends through
   the scheduled completion           of contractor      testing     and
   almost to the midpoint          of military      service testing.
   The Air Force decision          to use this option period,
   and thereby delay commencement of production                   until
   a substantial       portion   of Air Force-controlled            dem-
   onstration      test results      are known, indicates         that
   it is moving toward the DOD position                of "fly before
   you buy" and is gathering           more test data before
   committing      a weapon system to production.

B. Inadequate     application        of criteria
         .
   1. SRAM

        Some degree of subjective    evaluation   must often
   be exercised  in evaluating  test results    and that
   fact must be made clear to decisionmakers.       This
   has not been the case in the SRAM program.

          Major milestone    decisions   which involve advanc-
   ing an acquisition     program to its next phase must
   be based on broad information       about actual accom-
   plishments    as compared with planned accomplish-
   ments.     Test programs are devised to provide that
   information.     The SRAM flight    test program has
   fallen    somewhat below the ideal in that extrapola-
   tions of test results      have been used.



                                45
        Extrapolations     of the SRAM flight         test results
include adjustments        for conditions,        such as the in-
terim rocket motor, atmosphere, winds, launch alti-
tude, launch speed, and missile weight.                  Adjusted
test data based on technical            extrapolations,       engi-
neering assumptions,         and various other adjustments
for simulation       and probability      analysis      do not con-
clusively     demonstrate     SRAM/carrier      aircraft     actual
capabilities.        Such test   results     demonstrate      only
calculated     capabi Zities.      Test data based on accom-
plishment     more closely resembling         actual mission
conditions     would provide a reliable          gauge for top
management to judge the performance and progress of
a testing     program.

2. AN/BQQ-2 sonar       system

       When planned test programs are abrogated,          even
for a good reason such as‘an overriding           urgency to
deploy,    the   effectiveness   of   the product   is compro-
mised.     For example, the attack submarine program
required     that the AN/BQQ-2 sonar system components
be installed      at a specified    time during a submarine
shipbuilding      schedule,

        The current version       of the AN/BQQ-2 sonar sys-
tem was designed to provide a given improvement in
reliability      and a larger     improvement in maintain-
ability      over a prior version of the sonar system.
Reliability      and maintainability       demonstration   tests
were not conducted on the first            few production     sys-
tems.       The Navy followed    this course of action be-
cause it felt      the chances for success to be good
since the current        system is a follow-on      to previ-
ously designed and tested systems.             The first   pro-
duction system was delivered           before the production
acceptance test was completed,

       The schedule demands of the shipbuilding    pro-
gram for the nuclear attack submarines required
delivery   of the sonar systems before complete test-
ing to preclude delaying     the ship construction  pro-
gram. However, this should not be a justification
for skipping    the required   testing.

                           46
      The system delivered required changes to meet
requirements.   It is easy to see that program de-
cisions which ignore test results are apt to esca-
late costs.
3. M60 tank
      The M6OAlE2 is a modified version of the M6OAl
tank--currently   the Army's standard battle tank.
The E2 version was to have had a redesigned turret
incorporating the SHILLELAGHweapon system. The
SHILLELAGHwas already under development; and, in
this case, the objective was to adapt it for use
on the M6OAl tank and to provide, at an early date,
a tank having a missile-firing   capability.     Devel-
opment of the M60AlE2 began in 1964; although early
testing of prototypes had disclosed major deficien-
cies, the Army in 1966 authorized full-scale      pro-
duction of the tank before sufficient     testing had
been accomplished to validate the design, despite
advice of qualified testing and user agencies.
       Technical difficulties   occurred during produc-
tion which should have been detected in the test-
ing program. The technical problem which caused
the greatest concern and prevented deployment of
the tank was the inability     to stabilize  the turret.
This was a basic design fault that caused the
tank's gun to move erratically,     making it extremely
difficult   to deliver effective firepower.     Prema-
ture decision to enter production brought delivery
of 300 tanks and 243 turrets and components for
which extensive modification     is needed to satisfy
the user's requirements.




                      47
ORGANIZATION FOR PROGRAMMANAGEMENT

       Managing the acquisition      of complex weapon systems has
evolved into one of the principal           activities     of the military
services.     It is quite different       from other procurement and
receives    special attention    in the military         serviceso     Weapon
system management is the process of planning,                organizing,
coordinating,     evaluating,   controlling,        and directing     con-
tractors    and participating    organizations         to accomplish sys-
tem program objectives.

      The program management approach to weapon acquisition
is a distinct      departure      from the services'     traditional       method
of establishing      functionally       oriented   organizations      to carry
out well-defined,       repetitive      or continuous,     long-term      tasks.
This approach requires          the program manager to establish
management arrangements           among his organizations,        other mili-
tary organizations,        and various contractors         to efficiently
coordinate    their    efforts      to accomplish program objectives,

       A variety   of program management organizations           have been
established.      They range from a large,       self-sufficient     of-               .
fice to an austerely       staffed  focal point which operates on
the matrix principle       and which must draw all specialized
support from the functional        organization    to which it is at-
tached.      These are illustrated     in figure   III.

       The self-sufficient         program office     is organized and
structured    to operate by itself          without having to rely on
functional   organizations         for technical     and administrative
support,    Conversely,        the program office       operating  on the
matrix principle       relies     on functional    organizations     to per-
form such tasks as research,            development,     logistics   plan-
ning, procurement,         inspection,     and supply and maintenance.

       There are advantages and disadvantages                 associated     with
both the self-sufficient            program office       organization     and the
functionally       oriented     (matrix)     organization.       The advantages
of one organizational           structure     tend to be the disadvantage
of the other and vice versa; e.geo a matrix organization
fosters     greater    specialization        with less technical        duplica-
tion but makes coordination              and communication more difficult.
A self-sufficient         program structure        fosters    coordination       and
communication but makes specialization                  more difficult,      and
some technical        duplication      becomes inevitable,,
                                          48
                    FUNCTlONAt MANAGEMENT ORGANIZATION

                               NO PROGRAM    OFFICE   ESTABLISHED
               INCLUDES      PERMANENT    AND CONTINWNG        SPECIALIST                   STAFFS




PROGRAM A
PROGRAM B




                          PROGRAM MANAGEMENT ORGANIZATION
                                     PROGRAM      OFFICE     ESTABLISHED
   PROGRAM        OFFICE     STAFF      ASSIGNED       FOR   THE DURATION          OF     THE ACQUISITION
    INCLUDES       SPECIALIST         STAFFS     REQUIRED       FOR PROGRAM             ACCOMPLISHMENT




                                       MdTRlX ORGANIZATION
    INTERFACING           OF PROGRAM     AND       FUNCTIONAL         MANAGEMENT             ORGANIZATION
    MANAGEMENT             OF CORRELATED           ACTIVITIES       OF FUNCTIONS            AND    PROGRAMS


                                                                     FUNCTIONAL              ORGANIZATIONS
                                                                          I                      II              III
                                                                )   PROGRAM A PROGRAM A                       PROGRAM A
                                                                      TASK      TASK                            TASK

                                                                    PROGRAM B PROGRAM B                       PROGRAM B
                                                                      TASK      TASK                            TASK


                                              FZCGUFSJ.I.1
                                                   .   49
      In large part, the Air Force acquisition      programs are
in self-sufficient   organizations,   while the Navy projects
are matrix oriented;    and Army projects   are organized some-
where between the other two.

        Under its matrix concept, the Navy has provided only
14 people for its F-14 project             manager's organization.
Another 92 people are assigned to the functional                organiza-
tions within      the Naval Air Systems Command. They are iden-
tified    with the F-14 program but they do not work directly
for the project       manager.      Under this arrangement,      there is
need for considerable          coordination    between the organizations.
The functional       personnel associated        with the F-14 program
may or may not work exclusively             on this program.     Conflicts
for their     time must be negotiated         on a case-by-case     basis.
We are informed that priority             of work assignments on the
F-14 project      has not been a problem, but the potential             for
trouble    obviously     exists.

        In contrast,  the Air Force F-15 program manager, with                - .
243 staff members, essentially     has a self-contained     organi-
zation,     All the functions  necessary to manage the develop-               -
ment program are manned by personnel directly        responsible
to the program manager, and work assignment priority          can be
handled by him.




                                    50
          Organizational     "layering"

           One of the most troublesome         features   of the present
    program management structure         is difficulty      in obtaining    de-
    cisions.      It seems to us that the most likely          cause of this
    problem is that decisionmaking          layering    is not commensurate
    with organizational      layering.      In general,    the military     ser-
    vices have not deemed it wise to place the project                manager
    high in the organization        because of some practical         considera-
    tions,     such as the large number of project         managers and the
    need for them to work directly          at lower levels of the orga-
    nizations.      However, the effect      has been to preserve levels
    of review authority      which do not have clear roles in the
    process of formulating       decisions.

           Most of the decisions    that the project   manager does not
    make himself    are made at the highest   levels of the service
    or by OSD. Between the project       manager and top management
    are a large group of organizational      units whose commanders
    attempt to keep themselves informed about a particular
    weapon system and study and deliberate        on pending programs
    to recommend some course of action.       As a rule,    they have no
    direct   approval powers.    They can delay or stop a project
    but cannot make decisions     to proceed, change direction,      pro-
    vide money, or take other positive      action.

           Military    service organizations        for weapon system acqui-
    sition    are shown in the simplified        charts on pages 53, 54
    and 55. These charts do not show the many subdivisions                that
    become involved or the special ad hoc panels and committees
    which inevitably       arise in the weapon system acquisition         pro-
    cess e All these organizational          units,     panels, and commit-
    tees impact heavily on the project           manager.      His program may
    be delayed or stopped while matters are being studied or
    while decisions      are being made, or his program may proceed
    without     timely decisions.

           In the Army, for instance,  any significant    decision
    that the project   manager cannot make usually     is made at the
    highest levels of the Department or in OSD. With respect to
.   these decisions,   the primary role of the project     manager is
    to make recommendations    or to work with other groups that
    make recommendations.     Recommendations go through the normal


                                          51
chain of command; i.e.,        the Commanding General of the Com-
modity Command,     to  the  Commanding     General of AMC, to the Army
staff.     To formulate    recommendations     though, it is necessary
to coordinate     a number of functional       groups.    These include
functional    groups within      the project   managers' organizations
( i.e.,   the Commodity    Command)   as  well   as organizations   out-
side the Command, such as Conarc and CDC. The essential
task of these groups          is to help formulate         a recommendation,
but their      decisionmaking        function   is limited     to agreeing    or
disagreeing       with   it.    Once the recommendation          is made, there
are a number of functiona               groups at the AMC and DA staff
Zevels     (about    a dozen at DA staff        alone)    who can influence
the decision.          The contribution       of all   these groups iS much
the same.       They can either         agree or disagree      with  the ree-
ommendation       made.
        The inevitable       result     of this process is the scheduling
of repetitive        meetings,     briefings,      and studies in an attempt
to reach agreement on the recommendation to be made. sup-
plying information         to numerous groups can be almost a full-                .
time job for the project            manager.       During 1969, one project            .
manager spent about two thi-ds                of his time conducting     166
briefings     and from January to August 1970 participated                in 62    *.
additional      briefings.       From January 1969 through July 1970,
another project        manager participated          in 124 briefings.
Many of the briefings           involved levels below the top head-
quarters'     staff,    but the most important           function  of those
participating        was to recommend.

      In another instance of extensive        layering,     several re-
views of a program were conducted between September 1969 and
April 1970, including    an in-depth   review by several boards
and committees at all levels.       Of partieutar       importance  was
the requirement     that briefings      for decisionmaking        groups be
previewed     as many as 20 tG 30 times before         presentation      tG
an action-taking     body.     The project    manager spent a large
part   of his time participating        in these reviews.




                                       52
                                      ARMY ORGANIZATION FOR ACQUlSIflON




                                           1
                                           i

                                                                                                   3ARMY
                                                                                                   STAFF
                                                                                                           MY
                                                                                                             :
                                                                                                             :I
                                                                                                            .:I
                                                                                                            :I
                                                                                                             :




                                           i
                                           i
                                           i
                                           i
                                           i
                                           i   I   COMMAND
                                                                       I
                                                                                                                 :
                                                                                                                 :
                                           i                                                               II+


                                           i                                                                     :

                                                                                                                  :
                                                                                                                  I

                                           i                                                                     :

                                                                                                                 :
                                                                                                                  :


                                           i                                                                      :
                                                                                                                  :

                                           i                                                                     I

                                                                                                                 I
                                                                                                                  :

                                                                                                                  :

                                           i                                                                     :
                                                                                                                 :

                                           i                                                                      5
                                                                                                                  :
-         Line of Authority
                                           i                                                                     :
                                                                                                                  :
                                                                                                                  z

wowed     Staff Communication and
          Coordination Line                i                                                                     :
                                                                                                                 :
                                                                                                                 :
                                                                                                                 I
mII     I I Expeditious Information and                                                                          :
            Decision Line                                    IIIIIlIIIIIlIIIBIIlIII~~IIIIIIsIII~
                                                                                                                 :



                                          FIGURE IV
                           NAVY ORGANIZATION FOR ACQUISITION



~In1~11I1111a111111I~I~I          SECRETARYOFTKENAVY




                                                               -                          Line of Authority
                                     OFFICE OF THE CHIEF
                                     OF NAVAL OPERATIONS       II~IIIIIIIW               Staff Communication and
                                                                                         Coordination Line
                                                               n             I   I   I II Expeditious Information and
                                                                                          Decision Line

 1111m1111111111~1111111




                                      DEPUTY COMMANDER                                            ASSISTANT COMMANDER




                                           FIGURE V
                                                        AIR FORCE ORGANIZATION FOR ACQUISITION



                                                                   SECRETARY OF THE AIR FORCE

                                                                                  CHIEF OF STAFF




                                                                                                                                   Y:
                                                                                                                                    :
                                                                                                                                    ::
                                                 I




                                                                                                                                    E
                                             a
                                             I
                                                                                                                                     ::
                                             I
                                                     III-1             I                                                            ::
                                                                                 SYSTEMSCOMMAND
                                             i
                                             i                                                    L                   STAFF -4
                                             i                                                                      *       . :
                                                                                                                               :
                                             i                                                                                 :
                                                                                                                               :
                                                                                                                               :
                                             i                             I                              1                    :
                                                                                       DIVISION
                                                                                      COMMANDER           I
                                                 1
                                                 i
                                                 i
                                                 i
                                                 i
                                                 ! ,z
                                                             SYSTEM PROGRAM



‘-                Line of Authority
                                                                   I
     8JBJJJJlJJ    Staff Communication and                   SYSTEM PROGRAM
                  ‘Coordination Line                             OFFICE
                  Expeditious Informal:ion and ‘p                                                     1                              z
                  Decision Line                                                                                                      :
                                                                   SCCRJ~JJ~JJJJJJJJUJJJJJJJJJJJJJJ           JJJJJJJJIJJJJJJ~JU


                                                               FIGUREVI
      DOD-proposed action on
      acquisition management problems
     The Deputy Secretary  of Defense frequently   has cited
many of the problems in the organization   and procedures    for
managing weapon system programs.    He has stated that:

     1. Program managers must be given authority    to make
        decisions  on major questions relating   to the pro-
        gram, both in the conceptual  phase and in the full-
        scale development phase.

     2. Program managers must be given more recognition  op-
        portunity for career advancement in all the services,
        and good managers must be rewarded just as good op-
        erational people are rewarded.

     3. People in program management must be experts in that
        business and must be assigned to a given program
        long enough to become effective.

     4. The overall    structure   of     the program management
        function    in all services      needs to be appraised.
        Changes must be made to         reduce the numerous layers
        of authority    between the      program manager and the
        service secretary.




                                 56
                          CHAPTER3

                  SYSTEMCOSTEXPERIENCE
       Estimates of probable cost to develop major weapon sys-
tems are required at various points in the development
cycle.
     The initial   estimate against which all program costs
are originally   considered is the "planning estimate."
       The planning estimate is a formal estimate of cost
anticipated in acquiring a system in the quantities needed,
It is prepared prior to the initiation    of the formal acqui-
sition cycle and usually serves as a basis for the first
appropriation request.     The planning estimate is prepared
by a military department and is approved by the Secretary
of Defense.
       The planning estimate is followed by an estimate of the
cost to develop the system. The "development estimate" is
a refinement of the planning estimate and is established
during the period in which preliminary design and engineer-
ing are verified or accomplished and contract and system
management planning are performed. This period frequently
extends over a period of one year.
      A third estimate, the "total cost estimate," is in-
tended to be a current objective statement of the cost to
be incurred in acquiring the total approved program. This
estimate is adjusted for increases or decreases in quanti-
ties, as well as for cost changes due to inflation,    change
in scope, capability   in&ease, and program stretch-out.
      An estimate also is prepared to disclose costs which
are related to the maintenance, operation, or improvement
of a weapon system rather than its acquisition  cost. Ex-
amples are replenishment of spare parts, modifications,
component improvement, and commonground equipment. Pro-
jected operating costs are not included in this latter hind
of estimate.
           Nine of the                 70 systems we reviewed had just                                 entered
    the development                  process,    Their status precluded                                prepara-
    tion of precise                  estimates.    A summary of program                                cost es-
    timates for the                  remaining 61 systems is shown in                                  the table
    below.

           The estimated   cost for these 61 systems increased some
    $33.4 billion     from the cost anticipated   by the planning
    estimate    to the current estimate    of cost through program
    completion.

        About one third of this increase,      or $9.5 billion,
 represented   the difference    between the planning estimate
 and the development estimate.        The remainder of the in-
 crease, $23.9 billi on, was due to changes in quantities          to
 be acquired and to a combination       of such things as engi-
 neering changes, revisions      to correct  estimates,   and pro-
 visions   for economic inflation,

                                  Cost Estimates            as of June 30, 1970

                                                                                           Current
                                                                                          estimate
      Number                                                    Cost     changes           through           Total
         of               Planning     Development               (note a)                  program            cost
      fjystems            estimate      estimate             Quantity   Other            completion         (note b)



                                                            ";,;$.;         $ ;,;;;."Q   $ ;z,;o':.;    $ 17,197.6
Air    Force     (15)                                         -4163214        81220/2      47:418:4       56,335.9
                                                                                                          51.896.8
       Total     (611 $83,633.7         $93,135.6            $3,212.5       $20-697.2    $;17,045.3     $
%he cost changes shown represent   the difference   between the development                                 estimates
 and the reported current estimate   through program completion.
b
    Includes     additional    procurement         costs,
PERCENTAGEOFGROWTH




        59
      The foregoing  chart (figure   VII> shows that current
estimates   through program completion have grown 40 percent
in comparison with planning cost estimates     for these pro-
grams.

       Cost growth may result         from such things as unantici-
pated development difficulties,            faulty  planning,     poor man-
agement, bad estimating,          or deliberate    underestimating.
However, it is important          to recognize,     in any analysis      or
discussion    of cost growth, that not all cost growth can
reasonably    be prevented and that,some cost growth, even
though preventable,        may be desirable,       For instance,      un-
usual periods of inflation           may result   in cost growth,
Changes in technology may ma'ke it possible               to incorporate
modifications     that result       in an overall     increase in the ef-
fectiveness     of the system,        Such cpst growth cannot always
be anticipated,      particularly       where a weapon system is in
development and production           over long periods of time.

        We stated in our February 6, 1970, report    (B-163058)
that data were unavailable     from which'to make any specific
identification    of program cost estimate variances,

     We have suggested that           DOD give    increased   attention   to
the problem of identifying:

      1. Cost growth factors    that are not entirely         control-
         lable by DOD, such as inflation,        or thos,e factors
         that may even be desirable    and may be expected to
         continue,  such as  upgrading  sys tern performance.

      2. Items 'that are basic causes for cost growth and
         could be eliminated  or reduced considerably by ap-
         propriate    and effective        DOD action.

      DOD has made a good start toward accomplishing          the in-
tent of our suggestion,      Nine categories     of cost variance
have been established    for use in the Selected Acquisition
Reporting   system (SAR), and program managers have attempted
to quantify   the impact of cost variances       on their programs,
Although the precision     of these quantifications       cannot be            .
completely   verified,  segregations   being    made  can  now be
used to focus attention     upon areas where improvements can
be made,

                                      60
ANALYSISOF GOSTCHANGES
                     AS OF JUNE 30, 1970
      The analysis of cost changes on the 52 weapon systems
for which SAR data are available is shown in the table be-
low. There has been a net increase in total cost of about
$23,980 million.   Quantity increases have amounted to about
$12,600 million.   Decreases in program quantities have
amounted to about $10,216 million.    Other changes such as
engineering, schedule, and economic changes in the 52 weapon
programs have amounted to about $21,597 million.

            Analysis   of Cost Changes as of June 30. 1970

         Type of
       cost change              Army              Navy      Air    Force     Total
                                                    (millions)
Quantity change:
    Increase                  $1,371.1         $11,105.5     $ 122.3        $12,598.9
    Decrease                 -3.098.8           -1.760.5    -5,357.l       -10.216.4
         Net                 -1.727.7            9,345.0    -5.234.8         2.382.5
Other changes:
    Engineering    changes       489.3             463.8     3,119.4         4,072.5
    support           II          155.2            -57.7     1,268.5         1,366.0
    Schedule          II         462.1           1,308.7       844.7         2,615.5
    Economic          II         550.5           1,156.O     2,307.g         4,014.4
    Estimating        II       1,312.8           3,356.g     1,509.5         6,179.2
    Sundry            II         -12.7             553.1       544.3         1,084.7
    Unidentified      II                         2.264.9         -           2.264.9

         Total                 2.957.2           9.045.7     9,594.3        21.597.2

         Total                $1.229.5
                               --              $18,39=      $4.359.5       $23,97g.7
Number of systems                12                29             11            52




                                          61
QUANTITY CHANGES

       The approval of phase II of the SAFEGUARDsystem ac-
counts for $1,365 million      of the $1,371.1 million    quantity
increase reported    by the Army. Three of the Army programs
did not reflect    any change in the number of units to be
acquired.    However, seven systems reflected     decreases in
program costs totaling     more than $3 billion   due to reduc-
tions in the number of units to be acquired.         The largest
of these decreases involved the SAM-D ($1.8 billion)          and the
MBT-70 ($600 million).       We were informed that many of these
reductions   were the result    of a review by the Department of
the Army of its priorities      for weapon systems, which was
made because of impending budget reductions,        and the estab-
lishment of the Army's eight highest priority        systems.

        Analysis of the 29 Navy systems for which data were
available     shows that 10 systems reported no change in quan-.
tities;    nine systems reported      increased costs totaling
$11.1 billion      (due to an increase in planned procurements),
and 10 systems reported        decreases totaling     $1.8 billion.
The largest     part of the increase involves three ship pro-
grams totaling      more than $7 billion.      Included in this
amount is $1.6 billion       for 20(l) additional      DD-963's, rais-
ing the total     for this program from 30 ships to 50. Another
large part of this increase comes from two aircraft              programs
totaling    more than $3 billion.

      The Air Force reported           only a relatively    small increase
in cost due to quantity,          mostly related      to the SRAM. Two
systems, the F,15 and B-l, reported no change in quantity.
Seven systems reported reduced costs due to quantity                de-
creases,   totaling      $5.4 billion.      Of this amount, $4.4 bil-
lion involved the F-111, the FB-111, and the C-5A and
$600 million      involved the AWACS.

      Instances of reductions    in units acquired,     in all ser-
vices, were offset  by increases      in other costs for the weapon.
Cost growth is obviously    a significant     reason for reducing
the number of units to be acquired in all the services.


1
 We were informed in August 1970 that          these   20 ships were         *
  not considered a firm program.

                                    62
        ENGINEERING
                  CHANGES
              An alteration    in the established physical or functional
        characteristics     of a system is called an engineering change.
        Incomplete descriptions of initial       performance specifica-
        tions and changes required to bring system performance up
        to expected standards have resulted in substantial need for
        engineering changes. Of the $4 billion         dollars in engineer-
        ing changes reported by the three services, about $3,1 bil-
        lion was accounted for by the Air Force for the F-111, the
        C=5A, and the MINUTEMAN     programs. Engineering changes to-
        taling $1.8 billion were required to bring the F-111 and
        C-5A to expected standards, and $730 million involved changes
        in the MINUTEMAN     to upgrade the system to meet an increased
        threat.
        SUPPORT
              CHANGES
    .      Support changes involve such items as spare parts, an-
. cillary     equipment, warranty provisions, and Government-
    furnished property/equipment.     Relatively small amounts of
_ * money were reported in this category for the Army and Navy
    systems. Support changes in the Air Force amounted to about
    $1.3 billion and represented an increase in initial      spares
    for theCL5A($230 million)     and the F-111 ($258 million).
        SCHEDULE
               CHANGES
              Schedule changes reflect adjustments in the delivery
        schedule, completion date, or some intermediate milestone
        of development or production.     Cost increases of $2,615 mil-
        lion were reported as being due to schedule changes. Of
        this amount, $947 million involved three Navy aircraft     pro-
        grams (EA-GB, P-3C and A-7E); $260 million involved the
        SPARROW  missile; and $747 million involved the F-111. The
        largest portion of the increase ($460 million)    in Army pro-
        grams is accounted for by the SAFEGUARD,     SAM-D, MBT-70 and
        the LANCE.

.              For reporting purposes, identifying    such   schedule ad-
        'justment is probably important.     GAOfindings     indicate that
          such adjustments are only indicative of other      fundamental
        .problems.    Schedule changes, as such, are not     a primary
          cause of cost growth.

                                        63
ECONOMIC CHANGES

        Economic changes reflect     the influence      of one or more
factors    in the econonrry.   Included  are   specific    contract
changes deriving    from economic escalation        as well as changes
in quantity --changing     program estimates     to reflect     a re-
vised economic forecast       or changing actual contract        quan-
tities.

       We were informed by the Assistant         Secretary     of Defense
 (Comptroller)    that the treatment     of anticipated      economic
escalation     in various reports    was neither     consistent    nor
uniform within      or between services.      To rectify     these dis-
crepancies,     OSD stipulated   on June 30, 1970, that the Sep-
tember 30, 1970, SAR reports       forecasting     future price
levels were to be based on a table of percentages.

     We have not evaluated        this table, however, we believe
that there are no reliable        indexes on which to base esti-
mates of inflation.

ESTIMATE CHANGES

     Estimate    changes in a program or project           cost   are due
to corrections    in the initial  estimate.

        The principal    estimate change reported on Army systems
was $944 million      for the SAM-D missile.     The Army's justi-
fication    for this change in estimate was:

     "**The     total estimate is based on analysis      of
     our previous programs, deriving       cost estimating
     relationships    based on the actual growth experi-
     ence of cost estimates    for earlier     missile  pro-
     grams, at comparable stages of development.

     Specifically,       the estimating      techniques     anticipate
     unforeseen . changes       in  requirements,     performance
                      .
     characteristics,       p rogram slippages,       funding avail-
     ability,     and quantities       produced in specific
     years.     The order of magnitude of those changes
     actually     experienced      on previous programs has been
     used to estimate       the magnitude of these costs.
     While we have calculated           the costs based on past

                                   64
          experience,    we have also taken steps to seek to
          prevent the causes of cost growth from occurring
          on the SAM-D Program.      As such, if our efforts
          are successful,    the SAM-D will not require    the
          total   funds derived from extrapolating    the actual
          experience    of earlier  programs. **I'

           Two programs in the Navy account for most of its re-
    ported changes.    The Mark 48 torpedo cost estimate was in-
    creased $2,500 million      to correct   a series of underestimates
    which had been prepared from incomplete         data.   The new esti-
    mates projecting    production    costs were prepared by using
    the actual prototype     costs incurred.     The $300 million    es-
    timating   change on the Poseidon program corrected        a series
    of overestimates   and underestimates--an       aggregate of smaller
    sums.

            Three programs account for most of the reported        esti-
.   mating changes from the Air Force.         The F-111 aircraft     pro-
    gram reported       price increases of about $670 million     over
    earlier     estimates    on the contracts of numerous contractors
    involved in the program.

           The SRAM cost estimate was increased $398 million      due
    to underestimation     of the costs of development tasks in-
    volved, while the C-5A aircraft      cost estimate was increased
    $301 million    by the Air Force to rectify    contractor under-
    estimates    for producing this aircraft.

    UNIDENTIFIED CAUSES FOR COST CHANGE

            Summary data showing a cumulative    variance analysis      and
    the variance analysis      changes since the last reporting       pe-
    riod were either     not provided or were incomplete      for 15
    Navy systems.      For this reason, cost changes in Navy systems
    totaling    $2,264.9 million    could not be specifically     allo-
    cated.     We have been told by the Navy that cost changes will
    be allocated     and shown in the December 31, 1970, SAR.




                                      65
SAR SYSTEM

      As we reported  to the Congress in February 1970, the
SAR system represents   a valuable management tool for mea-
suring and monitoring   the progress of major acquisitions,
DOD has tried to improve the format,    content,and data in
the SAR.

      Although our review of the June 30, 1970, SAR con-
firmed that improvements were made during the last year,
some improvements still   were needed.

        SAR does not (1) contain a summary statement regarding
overall    acceptability   of the system for part or all of its
mission,    (2) show the status of major system components be-
ing separately     developed,   nor (3) reflect  the current  status
of program accomplishment.        Separate development could re-
sult in significant      costs if the major system component en-
countered development problems that adversely        affected  the
entire weapon system's performance.

      Waivers of major milestone   criteria,     with an explana-      .
tion of the attendant  risk therefrom,       are not highlighted  or       '
discussed in the summary section of SAR.




                                66
                                 CHAPTER 4

                      SYSTEMSCHEDULE EXPERIENCE

       'Our review of the efforts       of the military  departments
  to correctly   estimate   initial    delivery dates for about 50
  weapon systems indicates       that,  on the average, the weapon
  systems experienced     33 percent schedule slippage.       Average
  cost growth of these systems was approximately         30 percent.

        The following charts show the percentage    of schedule
  slippage by commodity class of weapon systems (figure     VIII>
  and the percentage  of cost growth (figure   IX>.

         The schedule percentages         were determined by comparing
  the time originally        estimated    for reaching the initial     op-
  erational    capability     date (initial    delivery    dates of the
  systems to the military         departments)    from the beginning    of
' the acquisition       cycle with the current       estimate   (as of
  June 30, 1970) of the same period.




                                       67
                                                             SCHEDULE SUPPAGE
            FR,
                    EtilOriginal    estimate       of time   required    to accomplish        initial    operational      copability


                    a    Slippage   in time      required    to accomplish       initial   operational       capability




                8




                                               i--l I---
                                                    29%
                                                                                                            18%




            6




‘MO. of Systems          (11)                     cm                          (8)                         (7)                          (6)
in Sample    (49)                              MIS51LES                                          ELECTRONICS
                      Al RCRAFT                                              SHIPS                                               VEHICLES
                                                                                                                                      84
                                                                                                                                 ORDNANCE



                                                            FIGUREVIII
                                                                        68
                                                                                       COST ‘GROV;TH

                         f?J 0 n‘g’~na 1 es t-tmote                 of cost        to complete         weapon    systems


                         cl     Current   estimate               of increase           in cost     to complete       weapon         systems




                               26%
            41




                                                                        33%



         20                                           .................
                                                      ..............
                                                      ..............
                                                      ..............
                                                      ..............
                                                      I..      ...........
                                                      ..............
                                                      ..............
                                                      ..............
                                                        .............
                                                      ..............
                                                        .............
                                                      ..............
                                                        .............
                                                      ..............
                                                        .............
                                                      ..............
                                                      I..      ...........
                                                      ..............
          15                                           .............
                                                      ..............
                                                       .............
                                                      ..............
                                                        .............
                                                      ..............
                                                        .............
                                                       .............
                                                      ..............                             113%
                                                      ..............
                                                       .............
                                                      ..............
                                                       .............


          10



                                                     ...............
                                                      . . . ...........

              !




                                                      ..............
                                                     ..............



No.   of Systems              (11)                                (17)                           (8)                          (6)                  (6)
in Somple         (48)
                         AiRCRAFT                    MIWL                     ES                 SHIPS                ELECTRONKS              VEHICLES   &
                                                                                                                                              ORDNANCE


                                                                                   FIGUREIX
                                                                                           69
                                 CHAPTER 5

                 GENERALOBSERVATIONS, CONCLUSIONS

                          AND RECOMMENDATIONS

       In the last several months, the Office of the Secretary
of Defense and the military              services have been engaged in
a substantial        effort     to resolve problems identified          as ad-
versely     affecting       the acquisition      of major weapon systems.
These problems include compromised performance,                    delayed
availability,        and increased costs.          Generally,    the more re-
cent weapon programs are characterized                 by a slower develop-
ment pace and more conservative               procurement practices        than
those of earlier         periods.      Because many of these programs
are in early stages of acquisition,                physical   evidence of the
success of changed concepts is not yet available                    for assess-
ment; but the outlook is brighter.                 Troublesome problems re-
main to be solved, particularly               in selection    of and assign-      .
ment of priorities           to weapons for development and in orga-
nizational      matters.

      The statement of the Deputy Secretary      of Defense in
September 1970 before the Committee on Government Operations,
House of Representatives,   on organizational     and other prob-
lems related   to new weapon systems development and acquisi-
tion,  leads us to conclude that he has accurately        appraised
the problems and the actions needed to resolve them. The ac-
tions he proposes are basic, but their      implementation    will
not be easy because they involve changes in traditional          con-
cepts and management practices    that are firmly    implanted in
DOD.

        Programs are under way in the military              departments to
improve the acquisition          process.     For example, AMC started a
comprehensive improvement program on October 1, 1969, called.
PROMAP-70. Among this program's 52 objectives                  are imprqved
definition      of requirements,      analysis    of technical     risk, up-
graded selection       criteria,     and stabilized     tours for officers
assigned to project management, as well as improved coordina-
tion and conduct of tests.            The Army has informed us that
results     already obtained in this program have shown substan-
tial    progress in application        of these improvements to cur-
rent programs.       An important      consideration      in our future

                                     70
  reviews will be an assessment           of the success    of these   im-
  provement programs.

        General observations on the matters we have studied,
  conclusions  we have drawn from that review, and our recom-
  mendations,   follow.

  A. Identification      of need for
     and relative     priority  of individual        systems

         The clear identification       of a new weapon's mission is
  probably the single,      most fundamental    task that must be com-
 'pleted    before the development process can begin.        Our study
   of the history    of a fairly   large sample of weapon systems,
  however, leads us to conclude that the function         of deciding
  which weapons will be developed is not yet being done with
  the degree of effectiveness        that this important  function
  warrants.

         Seemingly, the entire   structure    of the military    service
. and OSD are involved in this process,        in one way or another,
  and the long and imprecise process of defining          and justify-
  ing and of redefining    and rejustifying      a weapon system,
  through many layers of involvement,       invariably    has delayed
  decisions   and has extended stated availability        dates by years.

        The cumulative  effect   of the involvement    of many dif-
  ferent organizational    units in the decision     to justify  and
  then to proceed with development is the root cause of long
  delays in development decisions.      Almost every weapon sys-
  tem we studied showed some substantial       degree of uncertainty
  as to whether, when, or in what form the weapon should be
  developed.

         In addition  to clarifying         and improving   the initial   de-
  cision process (which is now           going on in the DOD>, establish-
  ing a mechanism which defines           the priority    position   of a
  weapon program in relation        to    its competitors     is equally im-
  portant.    We beZieve that the   development of a comprehensive
  DOD-wide priority     system is a first  step toward aZZeviating
  a part of th e difficulty    we observed in obtaining    weapon
  systems development decisions     and toward incorporating    sta-
* bility into programs.


                                         71
      Our study     revealed an emerging effort,         initiated     within
OSD during the      summer of 1970, and termed "a new concept."
It is intended      to provide the Secretary      of Defense with a
broad overview      of each mission category,        including     identifi-
cation of major       issues.    Although this effort        appears to
embody many of      the essentials     of an overall     priority     system,
it is still   in    its infancy.

      Recommendation--The      Secretary    of Defense should make
      every effort     to develop and perfect      the DOD-wide method--
      now in its early stages of development--designed            to be
      followed  by all military      services for determining       two
      things:   First,    what weapon systems are needed in rela-
      tion to the DOD missions.         Second, what the priority       of
      each should be in relation        to other systems and their
      missions.

B. Definition of performance characteristics
   and assessment of technical risks

       In the last several months,            persistent  problems in de-
fining    performance characteristics            of weapon systems and in       - .
determining     technical    feasibility       for achievement have been
receiving     extensive   attention      at   both OSD and the military
service levels.        On the basis of        our study of recent weapon
systems procurement,       we see many        encouraging  signs that
these problems are being abated.

       Extensive efforts       are being applied,   early in the pro-
cess, to identifying        high-risk   design areas and to construct-
ing and testing       actual hardware to demonstrate       feasibility
of high-risk      components before proceeding with further            de-
velopment.       Similarly,    current use of the demonstration
milestone    provisions     in development contracts     limits     the
Government's      financial    commitment pending a system's demon-
strated    performance.

C. Standards for and consistent   use
   of cost-effectiveness  studies

       We saw wide variation    in the quality      of preparation        and     .
follow-through     given to cost-effectiveness        determinations
supporting     weapon systems acquisition      decisions.


                                      72
       The variations in quality  may be due to evolving                 meth-
 odology for, and use of, cost-effectiveness   studies.            There
 is no evidence       that    DOD criteria  for   judging the adequacy of
 cost-effectiveness          studies   am being   appZie&.

         We are convinced that the Zack of cZear guidelines                  for
 the preparation         and appZieation     of cost-effectiveness        studies
 has resulted       in misunderstanding       of their    purpose,   has con-
 tributed     signifieantzy      to diversity    in execution      by the miZ-
 itary    services;      and has lessened the value of eost-
 effectiveness        studies  to the entire     acquisition     process.

       Recommendations--The     Secretary     of Defense should re-
       quire that (1) cost-effectiveness          studies meet certain
       standards   (including   the identification       of which weapon
       system and which considerations         should be included in
       such studies)     and (2) cost-effectiveness       studies be up-
       dated at each point where a major program alternative
       is considered.

        With regard     to the latter       recommendation,     we noted that
 instructions    now    require    cost-effectiveness       studies to be
 prepared at major        decision points in the program.           These de-
 cision points are       validation,      full-scale   development,     and
 production.

 D. Subsystem development          phasing   and interfacing

       A major problem recurring      in the weapon systems acquisi-
 tion process is the compromise of system performance that
 occurs when a principal     element of the system follows    a de-
 velopment cycle not compatible      with that of the primary sys-
 tem.   This incompatibility     occurred most frequently   when the
 responsibility  for the deveZopment of the parts of a                  system
 was divided among two or more project     managers.     The            results
 were imbalance in time-phasing   of subsystems in some                 weapon
 programs and incompatibiZity   of technica    interfaces               in
 others.
        We believe   that the program manager authority   should
 cover all technical     effort on all principal elements of the
.weapon. Whenever a principal      element is common to more than
 one weapon system, specific         steps must be taken to ensure           its
.development and acquisition         in order to meet the technical


                                       73
specifications     required  by each of the major systems which
will   employ it.     One way of handling that might be to give
authority     over the element to the manager of the more cru-
cial major system.

E. Assessment     of technical   performance

        In weapon programs we examined which were well along in
 the acquisition      process, were finishing       development,     or
were in production,        we noted that assessment of progress
against the development program was hampered by lack of early
test results      from technical    high-risk   areas.       When techni-
cal problems are revealed by testing,           there have frequently
been aspects that had not been formally             identified    as tech-
nically     risky    early in the program and therefore         had not
been given the special attention           needed during development.
Some programs have encountered           such serious technical       prob-
lems that degradation         from required   performance has been ac-
cepted.

       More sharply defined technical  risk analysis     with spe-
ciaZ emphasis applied to technica     high-risk   aspects of the                  -.
new weapon system should give the military      services    a means
of evaZuating deveZopment progress earZier,      and more aeeur-
ately,   than is presently  possible.

        In. recently  initiated      weapon programs, we found that
special     care is being taken to identify        the high-risk    eom-
ponents and to fabricate          them for testing    in Zaboratory   mod-
els before proceeding         with development of the compZete
weapon system.       We believe that this is a step in the right
direction.

F. Organization     and procedures

       In our judgment, one of the most important     unresolved
problems in the management of major acquisitions       is the prob-
lem of organization.    The problems arising    from establish-                        .
ment of need, for instance,    are related   to organizational
deficiencies.

      The essence of the problem appears to come from at-
tempts to combine the specialized  roles of major weapon sys-                 ,
tems acquisition  management into more or less historical

                                     74
military   commandstructure organizations.   Because of this,
there usually are a large number of organizational   units
not directly involved in the project which can only nega-
tively influence it.    In the Army and the Navy and to a
lesser extent in the Air Force, project managers are part of
organizations,  whose basic missions are considerably broader
than the managers* missions, with which organizations they
must compete for resources.
       As a matter of fact, each military  service alters tradi-
tional organization patterns when faced with managing major
programs. Although not recognized as a super program, in-
herent organizational   problems of the F-15 program were suc-
cessfully overcome by the program's having been placed in
the organization in such a way that the privileges     of sub-
stantial military   rank could be exercised as a means of by-
passing organizational    layers. The value of this reorgani-
zation is that the project manager has been given stature
and authority so as to be unencumbered by normal frustra-
tions produced by cooperation with the functional organiza-
tions.
      Each of the services has begun to upgrade the rank of
project managers. But military   rank alone will not accom-
plish what OSDand the military   services are trying to do.
        In our opinion, lessons learned from organizational
changes in structure for the super programs can aptly be ap-
plied to the whole subject of weapon systems acquisition.
It may be impractical to treat each of the large number of
projects now under way in the military     departments in a sim-
ilar manner. But, it occurs to us that, ideally, there
should be a direct relationship     between the way weapon sys-
tems requirements are categorized (strategic deterrent,
land warfare, ocean control, etc.) and the organizational
structure needed to acquire them. Such an arrangement would
facilitate    grouping related weapon systems in "packages" of
commonmission and would permit putting together an acquisi-
tion organization of appropriate size and stature to handle
the expanded concept. We believe that eventually program
management will evolve along mission lines.
     There are other alternatives, but whichever is chosen
must clearly provide for someone to be in charge, to have
                               75
clear authority    to make decisions,  and to have full respon-
 sibility  for the results.    The Deputy Secretary of Defense
recognizes    that correction  of this problem is fundamental   to
any real improvement and has stated that he plans to pursue
it aggressively.

      Recommendations --The Secretary       of Defense should place
      greater decisionmaking    authority      for each major acqui-
      sition  in a single organization,        within  the service
      concerned, with more direct      control     over the operations
      of weapon system programs and with sufficient            status to
      overcome organizational    conflict      between weapon system
      managers and the traditional       functional    organization.
G. System cost      experience

      Our analysis of the estimated   costs to develop 61 major
weapon systems which are prepared at various points in the
development cycle shows that the current estimates     through                    . _
program completion have grown 40 percent in comparison to
the planning cost estimates   for these programs.                                     ."

       Cost growth may result          from such things as unanticipated
development difficulties,          faulty    planning,     poor management,
bad estimating,       or deliberate       underestimating.        However, it
should be realized        that not all cost growth can be reasonably
prevented,      for instance,    cost growth resulting          from infla-
tion.     Further,    some cost growth may even be desirable,               for
instance,     incorporation     of   technological      changes    that  im-
prove the system effectiveness.

       Regarding our observations   made last year, we found
that DOD had made a good start toward developing      data that
specifically    identifies the variances   in program cost esti-
mates for systems reported under the SAR system.       We ob-
served, however, that on 15 Navy systems the causes for cost
change were either not provided or were incomplete.                                    .
      DOD also has acted to improve the format,   content, and
data in the S.ARs. Our review confirmed those improvements                             .
made during the last year.    We found,however, that some im-                     .
provements still  are needed.



                                      76
        Recommendations--The Secretary of Defense should ensure
        that the SARs (1) contain a summary statement regarding
        the overall acceptability  of the weapon for its mis-
        sion, (2) recognize the relationship  of other weapon
        systems complementary to the subject system, and (3) re-
        flect the current status of program accomplishment.




    .




.




                                77
APPENDIXES




 79
                                                                                                                                                APPENDIXI
                                                                                                                                                   Page 1

                                                                    SCHEDULE OF PROGRAM COST DATA
                                                           AS OF JUNE          30,      1970,    AND ARRANGED BY

                                                         ACQUISITION           PHASE AND MILITARY                  SERVICE



                                                                                                                                              Additional
                                                       Planning           Development            Cost          change           Current       procurement       Total
                                                      --estimate           estimate         . guantity               Other     estimate            -costs       __
                                                                                                                                                                costs

                                                                                                                                                                          .
CohTcEPTUAL       PHASE (9)        (note      d

.VALIDATION/RATIFICATION               (4):                                                                                       .
      ArOlJ.:
              None
     Imy   :
           DLGN 38                                         769.2               769.2              3,210.8          1,510.3      5,490.3                        5,490.3
           SSN-688                                      1,650.0             1.658.0               2.376.0             245.7     4.279.7                        4,279.7

     Air   Force:
           Ax                                           1,025.S       ’     1,025.S                                             1.025.5                        1.025.5
           OTH-B      (note   d) .                         100.9               100.9                                    -3.9       104.8                          104.8
ENGINEERING      AND/OR OPERATIONAL
  SYSTEMS DEvEu)PMENI        (57):
     Army :
          Cheyenne     (note   b)                 l        125.9               125.9                                    76.2          202.1                       202.1
          Shillelanh                                       357.4               357.4                 -18.1            156.5           495.8          i5.8         521.6
           SAFEGIJARli                                  4.185.0               *_                  1,365.0             389.0     5.939-o                        5.939.0
           DRAGON                                          382.2               404.2               -232.7              75.9        247.4             i7.4         284.8
           SAM-D (note      c)                          4.916.8             3.989.0             -1,791.4           1.215.9      3,413.5              82.4      3.495.9
           LANCE                                           586.7               652.9                                  108.2        761.1             90.4         851.5
           TOM                                             410.4               727.3               -300.1             248.1        675.3             33.3         708.6
           Improved      HAWK                              573.3          -    573.3                 -79.8            210.8        704.3            107.2         811.5
           M-60 AIE2                                       162.1               202.6                 -15.8            172.5        359.3             16.5         375.8
           MBT-70                                       2.126.5             2,091.4                -602.4             336.7     1,825.7             293.3      2,119.0
           Sheridan     Tank (note      f)                 422.5               375.6                 -13.1              93.4       455.9              31.6        487.5
                 Sheridan    Ammunition                    370.1               370.1               -125.2             105.9        350.8                          350.0
           cAMM4 GOAT                                        69.1              163.9                                    16.9       186.9              il.7        198.6
           CHAP/VULCAN                                      58.2                 58.2                38;:;              78.4       523.8            138.8         662.6
           TACFIRE                                         123.6               160.5                                    24.0       184.5                 3.8      188.3

     Navy :
     .    s-3!%                                         1,763.8             2.891.1                                     42.7    2.933.8              20.6      2,954.4
           F-14                                         6.166.0             6.166.0               2,0;6.1               77.0     8.279.1            294.4      8.573.5
           EA-6B                                           689.7               817.7                 -50.7            291.6     1,058.6              31.5      1.090.1
           P-X                                          1.294.2             1.294.2                  971.1            285.7      2,551-O             59.0      2.610.0
           A-7E                                         1.465.6             1.465.6                -385.3             494.4     lJ74.7               91.7      1.666.4
           AN/SQS-23                                       157.1               170.5                 -82.7            144.5         232.3            50.8         283.1
           AN/ SQS-26                                        95.7                88.8                                   30.8        119.6                         119.6
           AN/BQQ-2                                        126.9               179.0                                    86.2        265.2             i3.5        298.7
           DIFAR                                           178.5               414.1                    97.0            46.9        558.0           (e)           558.0
           VAST ANfUSM-335                                   49.8                57.5                -26.6              22.5         53.4                          54.8
           ;ASVS~xiUSM-247                                 241.1               312.0               -182.2             282-6         412.4            73.:         489.7
                                                           370.8               677.4                  642.3           181.2     1.500.9               11:o     1.511.9
           CONDOR                                          356.3               441.0               -220.9             131.3         351.4                         353.3
           POSEIDON                                                         4,568.7-               -243.6             790.2     5,1x.3           1,74::(:      6.855.5
           Standard   A?%                                  Go.3                241.6                 -10.4            -20.2         211.0             17.2        228.2
           Sparrow  E                                      687.2               740.7               -459.8               11.7        292.6             32.5        325.1
           Sparrow      F                                  139.8               453.6                 114.7            489.9     1,058.2               26.3     1.084.5
           Standard                                        313.2                                     --7.0              34.0        340.2           468.1         808.3
           Mark 48 Hod O&l                                 720.5               7i4.0                 488.9         2.554.3       3,757.Z              28.2     3.785.4
           LHA                                          1.380.3             1.380.3                                     47.5    1.427.8                8.2     1.436.0
           CVAN-68                                         427.5                                                      116.7         544.2                         544.2
           CYAN-69                                         519.0                                       -                            519.0                         519.0
           DE-1052                                      1.285.1             1.2g9.7                                   167.9     1.427.6             (2         1.427.6
           SSN-632                                                          2,515.8                                   397.3     2,913-l                        2,913.l
           SSN-ii85                                        l&.8                151.7                                    23.4       175.1                          175.1
           DIX modernization                               698.8               698.8                                  153.4           852.2                       852.2
                                                           324.4               328.5               -1G9.8             -12.6           186.1                       196.1
           DSRV                                            loo.2             '143.7                  -41.3            101.9        204.3                          234.9
           DD-963                                       1,704.4             2.581.2               1.595.4                       4.176.6
           AXIS   (note       9)                           388.0               427.6                                    13.0       440.6




                                                                                        81
APPENDIX I
    Page 2

                                                                                                                                                                             Additional
                                                                      Planning                 Development              Cost        change                 Current           procurement                      Total
                                                                      estimate
                                                                      --                        ~estimate            puJntity            --Other          estimate                costs                       costs
                                                                                                                                     (millions)

ENGINEERING     AND/OR OPERATIONAL
  SYSTEMS DEVELOPMENT (57) (continued):
     Air Force:
          B-l                        0.954.5                                                    10,107.a                                                  10.107.8                     392.9               10,500.7
          F-15    (note h)           6,039-l                                                      7.355.2                                                  7,356.4                     763.7                8,120.l
          C-5A                       3,423.0                                                      3,413.2                -756.2         1.63:::            4,308.6                     285.7                4,594.3
          F-111 AICIDfEtF            4,686.6                                                     5.505.0             -2,581.3           3,456.6            6.380.3                     960.3                7,340.6
          FB-111                     1.781.5                                                     1.781.5             -1.043.3              468.7           1.206.9                     231.6                1,438.5
               A-7D                                                                              1.379.1                 -282.6            303.1           1.399.6                     173.5                1,573.l
               AUACS                                                                              2.661.6                                                  2.661.6                     126.0                2Jg.f
               MAVERICK                                                 --257.9                      383.4                -;3.6                33.8           343.6                       8.0
               TITAN III                                                   932.2                     814.1                                   373.8         1,187.9                     (4                   1.18719
               SRAM                                                        167.1                     $36.6                 118.3             735.8         1.090.7                     590.9                1.681.6
               Minuteman  II                                            3,014.l                  4.254.9                      4.0            207.5         4.466.4                     583.2                5.049.6
               Minuteman  III                                           2,695.5                  4I673.0                  -37.7              999;1         5;635.2                     362.6                5.997.8
               777 COMSAT                                                  133.5                     138.0                                      5.1           143.1                                            143.1


"Comparative                cost     data       not         available'for                 systems        in   this       phase.

bThe Cheyenne costs  represented                                   research                 and development              costs       only.       The      production         contract            Gas       termi-
 nated on May 19, 1969.      Due                                to pending               litigations,       the          Army's        liability          was unknown.

c
    Army    officials              advised    us that,    while                        the      SAM-D had         gone     through        contract            definition,        contract           award
    had    been limited               to advance    development.

dCost       data      as of         August.31.                1970.         for    the         OTH-9.

%ata        were      not      available              for       inclusion.

fThe       DOD considered                this         as an annex                 to     the     Sheridan       vehicle           and not         a weapon         system    itself.

gResearch          and development                     costs          only.

%I e original                 Development      Concept   Paper No. 19 dated Sept.                                           28,      1968.        contained         a preliminary               planning
 estimate     for            lower    quantity     of F-15's   as $5,137 million.




                                                                                                             82
                                                                               APPENDIXII
                                                                                   Page 1

                      DIRECTOR    OF DEFENSE          RESEARCH    AND ENGINEERING
                                        WASHINGTON,       D C. 20301




                                                                         22 JAN 1971



Mr.     C. M. Bailey
Director,      Defense      Division
United    States    General       Accounting           Office
Washington,        D. C.        20548

Dear    Mr.    Bailey:

This letter    is in response     to the request     of Mr.  Hassell   Bell of
your organization      for informal     comments       on GAO draft   report
“The   First   Report   on the Continuing      Evaluation    of the Acquisition
of Major     Weapons    Systems”     (OSD Case #3219).

I know that you appreciate              the extremely          limited      time the DOD
had to review         this report.        However,       in recognition          of the equally
limited     time which       Mr.   Bell indicated          the GAO has to meet              its
 commitment         for submission         of the report         to the Congress,          we
have done our best to prepare                 a general       reaction       to it.   Because
 of the nature       and importance         of this subject,           we will want to
 examine      the final    report   further      in a more         thorough      and logical
fashion.        It would    be appreciated         if your report         to the Congress
 could indicate        the fact that the DOD has not had sufficient                     time
to make       such a review.

We have reviewed          the draft     report     and believe   that your recom-
mendations      address      important       aspects    of the weapon    system
process.       We agree      in particular       that we have not yet solved
some of the organizational             problems       and we will see that your
report     is made available         to the Services      and OSD offices      which
are working       on those problems,

We do appreciate      the recognition      that you give to the DOD efforts
to improve    its management        of weapon    systems       acquisition,      and
we know that you realize        we are giving      considerable          time and
attention  to further    improvements.




                                               83
APPENDIXII
    Page 2
 As you know,         we are carrying           on comprehensive           evaluations     of
 this management           problem      here      in the Department.           GAO reviews,
 such as this,       will be of benefit          to us, particularly        by giving    us an
 independent      review      and evaluation            of our options.      We are pleased
 to assist   you by providing           these       informal    comments        on the draft
 report.     We will forward           more       detailed    comments       after we have
 made    a more      thorough      evaluation          of the report,    if you feel that
 would    be helpful.

                                                              Sincerely,




                                                                                U.S.   GAO   Wash..   C

                                                84