oversight

Army Acquisition: Air Defense Antitank System Did Not Meet Operational Test Objectives

Published by the Government Accountability Office on 1990-12-10.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                              ~l - l _ ~l l - ~l . -




                                                                      A R M Y A C Q U IS IT IO N
                                                                     A i r D e fe n s e A n ti ta n k
                                                                     S y s te m D i d N o t M e e t
                                                                     O p e ra ti o n a l T e s t
                                                                     O b j e c ti v e s


                                                                                                                                               II Il l
                                                                                                                                                 142922




                                                       R E S T R IC I’E D --N o t         ta b e re l e a s e d d & s i d e             th e
                                                       G e n e ra l A c c o u n ti n g O fl ’S r ~ u n l e s s s p e c i fl c a l l y
                                                       a p p ro v e d b y th e O fW e a Y C o n g re s s i o n a l
                                                       R e l a ti o n s .


G A O /N S IA I)-9 1 -5   1
I”l _ (l _ -_.._.“. ..- .._ .- .. ..--..-- --_-__------   -.-..---
                          Uni t ed      States
                          General        Accounti n g   0ffi c e
                          Washi n gton,        D.C. 20548

                          Nati o nal         Securi t y               and
                          Internati o nal                 Affai r s         Di v i s i o n

                          B-240822

                          December               l o ,1990

                          The Honorabl e         Sam Nunn
                          Chai r man,      Commi t tee  on Armed                                Servi c es
                          Uni t ed    States Senate

                          The Honorabl e       Les Aspi n
                          Chai r man,    Commi t tee     on Armed                               Servi c es
                          House of Representati v es

                          As requi r ed           by the fi s cal year 1989 Nati o nal                                Defense Authori z ati o n                               Act
                          (P.L. l O O-456),            we eval u ated                the performance                     of the Army’s                     l i n e-of-si g ht
                          forward           heavy      air defense                system duri n g          operati o nal                   testi n g.             The act
                          requi r es        that the Secretary                     of Defense certi f y that the system meets or
                          exceeds          the Army’s          operati o nal              test performance                    cri t eri a           before the Sec-
                          retary of the Army obl i g ates                            procurement               funds after fi s cal year 1989.
                          Accordi n gl y ,          our obj e cti v e            was to determi n e                  whether              the system had
                          adequatel y            demonstrated                i t s operati o nal       sui t abi l i t y             and operati o nal
                          effecti v eness.            We are reporti n g                 on the cl a ssi f i e d            test resul t s              i n a sepa-
                          rate document.


                          Operati o nal         testi n g      di d not demonstrate                   that the l i n e-of-si g ht                           forward
Resul t s   i n Bri e f   heavy         air defense          system was operati o naIl y                       sui t abl e .      The                  system fel l far
                          short of i t s avai l a bi l i t y          requi r ements,             i n l a rge part because                               many system
                          components            were unrel i a bl e .            In addi t i o n,         other sui t abi l i t y                      measures
                          were ei t her not tested or not met.

                          The system di d not meet a number                                of effecti v eness                 requi r ements      duri n g
                          operati o nal          testi n g.   These i n cl u de           the i n di v i d ual          fire uni t cri t eri o n     for
                          destroyi n g         threat ai r craft             wi t hi n   a speci f i e d         engagement              area and detec-
                          ti o n requi r ements.            In addi t i o n,           we bel i e ve         that pl a toon-l e vel           and
                          response         ti m e requi r ements                  for the system-al t hough                           met-may      have
                          been set too l o w.

                          The Army has acknowl e dged                                    the system’s      rel i a bi l i t y                    probl e ms;          however,
                          it bel i e ves    that the system demonstrated                                 suffi c i e nt                       operati o nal           effec-
                          ti v eness     to support                 conti n uati o n          of the program.                              In August             1990, the
                          Army deci d ed             to defer producti o n                      of the system for 2 years. Duri n g                                           that
                          ti m e, the Army hopes to resol v e                                the system’s                 rel i a bi l i t y            probl e ms        and
                          demonstrate           i t s sui t abi l i t y              for combat.




                          Page      1                                                                  GAO/NSIADBl - 61               Ai r   Defense       Anti t ank     System
                                                B.240822




_.. I. __ _ ..-   ..__^ ^^_.._ _ _...._..__--
                                                Because the system has not demonstrated                       that it can meet establ i s hed
                                                requi r ements,        by l a w the Secretary         of the Army may not obl i g ate         any
                                                procurement          fundi n g    after fi s cal year 1989 unti l the system meets or
                                                exceeds         the Army’s     operati o nal     test performance      cri t eri a .


                                                In November       1987, the Army chose i t s current                     l i n e-of-si g ht   forward
Background                                      heavy ai r defense        system, trade-named             the “A i r Defense Anti t ank
                                                System” (ADATS), to provi d e          needed      ai r defense              to the maneuver           force
                                                (see fi g . 1). ADATS was sel e cted       partl y     because         it was i n producti o n             in
                                                Canada      and therefore       coul d move qui c kl y         i n to producti o n          to sati s fy
                                                Army requi r ements.          An earl y Army esti m ate               of the system’s            uni t cost
                                                was about $11.2 mi l i o n.




                                                Page       2                                             GAO/NSLADSl - 51       Ai r   Defense   Anti t ank   System
                                                         B.240822




Fi g ure   1: the   Ai r Defense   Anti t ank   System




                                                                                            : i‘ /’ ”




                                                         ADATS i s one            of fi v e components              i n the Army’s               pl a nned            Forward         Area
                                                         Ai r Defense System, whi c h together                                    are expected              to meet the Army’s
                                                         forward      ai r defense                  needs. ADATS' mi s si o n            i s to defend tanks and
                                                         i n fantry     fi g hti n g       vehi c l e s.    The Army expects ADATS to si g ni f i c antl y
                                                          enhance     i t s current              ai r defense      capabi l i t y         because              of the system’s
                                                         extended          ranges and i t s abi l i t y           to operate               i n adverse             envi r onments.          The
                                                          system consi s ts              of a l a uncher        and ei g ht mi s si l e s              that are gui d ed              by a
                                                          l a ser beam and mounted                        on a modi f i e d          Bradl e y        Fi g hti n g         Vehi c l e  chassi s .
                                                           The system must be posi t i o ned                      so that i t has a l i n e of si g ht to enemy


                                                         Page    3                                                            GAO/NSIAD-91-61              Ai r   Defense    Anti t ank    System
                                    B-249822




                                    ai r craft.    A shoot-on-the-move                                capabi l i t y           was not part                of the system’s
                                    requi r ement.

                                    The Army conducted                      two phases of operati o nal                 testi n g, whi c h were
                                    compl e ted            i n May 1990, In the fi r st, i n di v i d ual              ADATS uni t s         fi r ed l i v e mi s -
                                    si l e s agai n st         drone ai r craft   at Whi t e Sands Mi s si l e               Range, New Mexi c o.
                                    In the second phase, the Army conducted                                  a seri e s of tests that repl i c ated
                                    ground          battl e s at Fort Hunter-Li g gett,             Cal i f orni a ,       duri n g       whi c h    fi r i n gs
                                    by ai r and ground               weapon      systems were si m ul a ted                     by l a sers.


                                    Operati o nal           avai l a bi l i t y        measures     the proporti o n             of ti m e a system wi l
ADATS’ Operati o nal                be avai l a bl e        to successful l y              conduct     assi g ned     mi s si o ns.       Such avai l a bi l i t y
Avai l a bi l i t y    Is Not       depends            on the rel i a bi l i t y         of component           parts and the ti m e it takes to
Adequate            to Compl e te   mai n tai n
                                    broken
                                                        them, to acqui r e
                                                     component                  parts.
                                                                                         needed    spare parts, and to repai r or repl a ce
Its Mi s si o n
                                    The Army’s               operati o nal         tests showed                 that the ADATS system di d not meet
                                    i t s requi r ements              for overal l            operati o nal           avai l a bi l i t y                   or for the rel i a bi l i t y
                                    of i n di v i d ual        components.            Further,              we bel i e ve                   that the test resul t s over-
                                     state actual avai l a bi l i t y              because              the Army excl u ded                                     i m portant        data
                                    from i t s cal c ul a ti o ns.          In addi t i o n,             ADATS exceeded                           the mai n tenance                       ti m e
                                    al l o tted         for each hour of the system’s                           operati o n.                    Moreover,                    the Army
                                    di d not desi g n            these tests to col l e ct                  certai n           i n formati o n                         necessary        to
                                    ful l y eval u ate           the Army’s           abi l i t y        to mai n tai n                   or l o gi s ti c al l y             support
                                    ADATS.



Avai l a bi l i t y  and            The operati o nal    avai l a bi l i t y requi r ement          for fi e l d i n g                   ADATS i s 71 percent.
Reliability         Requi rements   For operati o nal   testi n g, the Army set an i n teri m                          avai l a bi l i t y         requi r e-
                                    ment of 66 percent            and reported        an adj u sted     avai l a bi l i t y                   of 39 percent
Were Not Met                        duri n g the tests.

                                     The system al s o fel l far short of i t s requi r ements                                                 for the rel i a bi l i t y            of
                                    i n di v i d ual           components.           Rel i a bi l i t y              i s measured           by the average                   ti m e
                                     between           operati o nal        mi s si o n                 fai l u res,      whi c h i n cl u de      crew performance-
                                     rel a ted       fai l u res,      and between                       fai l u res      sol e l y caused by equi p ment.                          In
                                    both cases, requi r ements                          were set and tested for the overal l                                           fi r e uni t and
                                     for the mi s si l e             weapon       subsystem.

                                    For the fi r e uni t , the average       ti m e requi r ed    between       operati o nal        mi s si o n
                                    fai l u res    i s 60 hours at the ti m e of fi e l d i n g and 38 hours for the opera-
                                    ti o nal    tests. However,    the system demonstrated              an abi l i t y      to operate           for


                                    Page       4                                                                       GAO/NSIAD-91-61             Ai r   Defense       Anti t ank      System
                                  8240822




                                  onl y about 9 hours before it fai l e d.                        For the weapon                   subsystem,     the
                                  fi e l d i n g requi r ement              i s 92 hours, and the i n teri m                   test requi r ement     is
                                  66 hours. The subsystem                           demonstrated              about 8 hours. Thi s performance
                                  al s o represents            a decl i n e        from 1987 tests i n whi c h the demonstrated
                                  average        ti m e between                operati o nal     mi s si o n       fai l u res    was 16 hours for the
                                  fi r e uni t and 17 hours for the weapon                                  subsystem.

                                  The system’s          performance,        as measured       by the average                ti m e between
                                  equi p ment   fai l u res,     has al s o worsened       despi t e        Army efforts to i m prove
                                  it. The demonstrated             average     ti m e between        equi p ment         fai l u res
                                   decreased   from 17 hours i n 1987 to about 12 hours for the fi r e uni t and
                                  to 11 hours for the weapon                subsystem      duri n g       the operati o nal          tests.


Army Resul t s        Overstate   In cal c ul a ti n g      an operati o nal                 avai l a bi l i t y       of 39 percent,              the Army
Avai l a bi l i t y               excl u ded           some cri t i c al   test data and made certai n                                  erroneous        assump-
                                  ti o ns. We di d not quanti f y                   the cumul a ti v e                   effect of al l the probl e ms             that
                                  we found i n the Army’s                    cal c ul a ti o n.                  However,           on the basi s of the effects
                                  that we di d cal c ul a te             and the di s crepanci e s                         outl i n ed      bel o w, we bel i e ve
                                  that the actual avai l a bi l i t y                  of ADATS i s l o wer than reported                              by the Army.

                                  For exampl e ,          the Army’s      cal c ul a ti o ns               di d not i n cl u de   any test data from
                                  one of two test phases-                the mi s si l e           fi r i n gs.        Thi s phase was cri t i c al
                                  because        it was the onl y ti m e duri n g                  testi n g that the enti r e system was
                                  actual l y      i n operati o n.    Our cal c ul a ti o ns,                  usi n g the methodol o gy          the Army
                                  appl i e d    for the force-on-force                   test resul t s, show an operati o nal                      avai l a -
                                  bi l i t y of 33 percent         for the mi s si l e -fi r i n g                 phase.

                                  In addi t i o n,          al t hough                 the system requi r ements                                             sti p ul a te         that al l
                                  unschedul e d                mai n tenance                              ti m e be i n cl u ded                   i n test cal c ul a ti o ns,                      the
                                  Army i n cl u ded                 onl y correcti v e                                  mai n tenance                 ti m e associ a ted                      wi t h opera-
                                  ti o nal      mi s si o n       fai l u res.                   For exampl e ,                          as a resul t of a mal f uncti o n                                wi t h
                                  the system’s                 radar-a                         mi s si o n-cri t i c al                    pi e ce of equi p ment-about                                      6
                                  hours were needed                            to repai r the subsystem                                              over a 4-day peri o d.                             Because
                                  the mal f uncti o n                    was not scored as an operati o nal                                                           mi s si o n     fai l u re,          how-
                                  ever, the mai n tenance                                     ti m e was not i n cl u ded                                i n the cal c ul a ti o n.                  Li k e-
                                  wi s e, unschedul e d                       mai n tenance                                ti m e needed               to repl a ce               a uni t that
                                  affects the turret’s                         movement                               was not i n cl u ded.                      In total , the Army per-
                                  formed            145 hours of unschedul e d                                                    mai n tenance               that were not i n cl u ded                         in
                                  the force-on-force                         avai l a bi l i t y                      cal c ul a ti o n.




                                  Page      6                                                                              GAO/NSLAD-91-61                     Ai r   Defense         Anti t ank        System
                                                                                                                                                                                                Y


                                          B-240822




M i s si o n-Essenti a l    Fai l u res   Army requi r ements                         documents                          defi n e the “m i s si o n-essenti a l ”                               functi o ns
Not Adequatel y            Defi n ed      that an ai r defense                      system must perform                                               to successful l y               defend         the
                                           heavy maneuver                        force. These i n cl u de,                                   for exampl e ,              the abi l i t y          to shoot,
                                           move, and communi c ate.                                      Identi f yi n g                  the component                   fai l u res        that resul t
                                          i n a l o ss of m i s si o n-essenti a l                                  functi o ns               i s cri t i c al      because            test cal c ul a -
                                           ti o ns of operati o nal                  avai l a bi l i t y                   i n cl u de             the fai l u re       of and l o gi s ti c al             sup-
                                           port needed                for onl y those component                                              fai l u res.         The Army di d not cl e arl y
                                          defi n e           these operati o nal                       m i s si o n           fai l u res             unti l tests were underway                            and
                                           after parti a l             scori n g       resul t s were al r eady                                        known. We bel i e ve                   that these
                                           defi n i t i o ns        shoul d        have been approved                                            before testi n g began. Wai t i n g
                                           unti l after some scores were known                                                          provi d es             the appearance                  of
                                           affecti n g           the outcome                of test resul t s by deci d i n g                                       the cri t eri a          to be used
                                           after the fact.

                                          We bel i e ve                      that the defi n i t i o ns                  used to eval u ate                        ADATS' operati o nal                avai l -
                                          abi l i t y     i n appropri a tel y                         el i m i n ated          the fai l u re           of some components                           that
                                          were essenti a l                         for ADATS to perform                           i t s m i s si o n.            For exampl e ,                 after the
                                          tests were underway,                                     the Army deci d ed                       not to consi d er                         the fai l u res    of
                                          the l a ser range fi n der to be operati o nal                                                   m i s si o n           fai l u res,           even though            (1)
                                          sol d i e rs         used i t as an i n tegral                               part of the test m i s si l e s’                        fi r i n g sequence;
                                          (2) the Army determ i n ed                                           that the range fi n der’s                            faul t y performance                  was
                                          responsi b l e                       for m i s sed targets duri n g                          the m i s si l e -fi r i n g               phase of opera-
                                          ti o nal       testi n g; and (3) test procedures                                             requi r ed              that the range fi n der, as
                                          a key component,                                   be operati o nal                 for the start of each m i s si l e -fi r i n g                            test.
                                          If these data had been i n cl u ded                                              i n the Army’s cal c ul a ti o ns,                                the number
                                          of operati o nal                         m i s si o n       fai l u res        woul d have been hi g her,                                    and ADATS' oper-
                                          ati o nal          avai l a bi l i t y                woul d have been l o wer.

                                          Fi n al l y , bui l t -i n             test equi p ment       messages,                 desi g ned       to al e rt the ADATS
                                          crew to component                           fai l u res,  were not al w ays                     treated     as i n di c ati o ns        of
                                          operati o nal              m i s si o n      fai l u res,  as i s requi r ed              by test procedures.                    Doi n g so
                                          woul d further                    reduce the system’s               avai l a bi l i t y            rate.


Li m i t ed    Mai n tenance              The ADATS schedul e                 cal l s for contractor       mai n tenance             support            above the
Capabi l i t y       Demonstrated         organi z ati o nal        l e vel , or that l e vel i m medi a tel y           above the crew’s mai n te-
                                          nance responsi b i l t y,                 unti l the system i s fi e l d ed      outsi d e     of the Uni t ed
Duri n g Testi n g                        States. Because there are l e gal prohi b i t i o ns                        on contractor          i n vol v ement      in




                                          Page      0                                                                         GAO/NSIALL91-61                    Ai r   Defense        Anti t ank        System
                                E-240822




                                operati o nal            testi n g,’ the Army deci d ed                     to eval u ate          the mai n tenance          pro-
                                vi d ed onl y at the organi z ati o nal                    l e vel duri n g            operati o nal          tests. The test
                                resul t s showed                 a l i m i t ed     Army capabi l i t y          to mai n tai n           ADATS at the orga-
                                ni z ati o nal      l e vel . Army mai n tenance                       personnel          demonstrated              thei r capa-
                                bi l i t y     to meet one of the two mai n tenance-rel a ted                                    requi r ements,          but fel l
                                far short of meeti n g                         the other.

                                The average              ti m e to correct equi p ment               fai l u res    at the organi z ati o nal
                                l e vel - or        the mean ti m e to repai r -was                    0.62 hours, a fi g ure that was
                                wel l wi t hi n        the requi r ement            of 1.5 hours. However,                        not al l organi z a-
                                ti o nal     mai n tenance                tasks were tested; cri t eri a          for the ti m e associ a ted
                                 wi t h each task have not been establ i s hed;                               and the al l o cati o n         of tasks
                                among vari o us                l e vel s of mai n tenance          was not based on fi r m requi r e-
                                ments. Therefore,                       the val u e of thi s i n formati o n         i s l i m i t ed.

                                The second measure                      of performance-peopl e                    and ti m e al l o tted      to orga-
                                ni z ati o nal  l e vel mai n tenance                per system operati n g             hour, or mai n tenance
                                rati o -was          set at 0.094 mai n tenance                  hours per system operati n g                   hour.
                                Thi s requi r ement                 was not met. The system demonstrated                             a rati o of 0.56,
                                or, i n other words, the mai n tenance                         support        needed       for each hour of
                                system operati o n                  was al m ost 500 percent               more than al l o wed           by the
                                requi r ement.                Thi s poor performance            resul t ed      i n part from the system’s
                                numerous         rel i a bi l i t y     fai l u res,

                                W i t hout     cl e arl y        defi n ed     tasks and associ a ted  repai r ti m es i d enti f i e d                                    at
                                al l mai n tenance               l e vel s , the Army wi l not have a cl e ar understandi n g                                                of
                                the mai n tenance                   needed      to support the system.


Logi s ti c s   Support   Not    The Army performed                             a qual i t ati v e             assessment              rather than a test of the
Tested                          l o gi s ti c s support           that wi l be requi r ed                             for ADATS because                  i t has not yet
                                 determi n ed              ADATS l o gi s ti c s            support            requi r ements.             The test cal c ul a ti o ns
                                 assumed         that wai t i n g               for spare parts, mai n tenance                                personnel ,         and trans-
                                portati o n       servi c es        i n vol v ed              i n correcti n g             operati o nal        mi s si o n     fai l u res
                                 woul d take an average                           of 14 hours. The actual ti m e may change                                               after
                                the Army conducts                       i t s l o gi s ti c s           support          anal y si s .    Any i n crease              i n ti m e
                                 spent wai t i n g            for parts, personnel ,                           or other servi c es             woul d decrease
                                 ADATS' avai l a bi l i t y.           In si m i l a r,             previ o us          testi n g of an ai r defense                     weapon,
                                the Sergeant                 York, a 23-hour                       del a y was assumed.

                                 C‘ ontractor    i n vol v ement duri n g operati o nal      testi n g     of a maj o r defense        acqui s i t i o n   system i s prohi b -
                                i t ed by statute unl e ss the contractor        i s to be i n vol v ed      i n the operati o n,      mai n tenance,         and support of
                                the system when i t i s depl o yed        i n combat.



                                Page    7                                                                 GAO/NSIAD-91-61             Ai r Defense          Anti t ank   System
                                      B240822                                                                                                                   +




                                      The Army reported               that ADATS demonstrated                superi o r      mi s si o n      perform-
ADATS’ Operati o nal                  ante for condi t i o ns        under whi c h i t was tested duri n g                 operati o nal           testi n g.
Effecti v eness  Is 5                 The system di d not, however,                  meet i t s requi r ements            for an i n di v i d ual
Questi o nabl e                       fi r e uni ts’
                                      boundary,
                                                      performance          agai n st
                                                       nor for the detecti o n
                                                                                      targets wi t hi n
                                                                                       of threat ai r craft.
                                                                                                               a certai n       engagement
                                                                                                                        We are reporti n g             on
                                      the speci f i c    resul t s i n a separate       cl a ssi f i e d document.

                                      In addi t i o n,     we questi o n         the system’s           effecti v eness           because

                                  . some requi r ements           mi g ht have been set too l o w and
                                  . the operati o nal        tests di d not demonstrate    al l of the requi r ed                                perform-
                                    ance capabi l i t i e s.

                                      The tests were based on two measures                                     of operati o nal          effecti v eness-the
                                      abi l i t y   of an i n di v i d ual      fi r e uni t to destroy                a target wi t hi n           certai n
                                      boundari e s         and the abi l i t y            of a pl a toon        of four fi r e uni t s to defeat enemy
                                      ai r attacks. The fi r e uni t cri t eri o n                     i s contai n ed        i n the Army’s            requi r ed
                                      operati o nal          capabi l i t y    statement             for the l i n e-of-si g ht         forward            heavy ai r
                                      defense         system, whi c h predates                     the sel e cti o n       of ADATS. Test offi c i a l s
                                      from the Army and the Offi c e of the Secretary                                           of Defense determi n ed
                                      that a more operati o nal l y                      accurate         measure        of effecti v eness            woul d be at
                                      the pl a toon        l e vel agai n st         al l attacki n g        targets regardl e ss             of boundari e s.
                                      Therefore,          they deri v ed           a second cri t eri o n              from the cost and operati o nal
                                      effecti v eness            anal y si s  that had been performed                             for the Forward               Area
                                      Ai r Defense System program.

                                      For both the fi r e uni t and pl a toon-l e vel                     cri t eri a ,     requi r ements         for the
                                      system’s         effecti v eness             were set agai n st     fi x ed-wi n g          and rotary-wi n g        ai r -
                                      craft. A number                of addi t i o nal        performance             measures,        some of whi c h had
                                      quanti t ati v e       requi r ements,            al s o were i d enti f i e d.


Operati o nal   Effecti v eness       Al t hough        ADATS met i t s overal l                 pl a toon-l e vel      effecti v eness        requi r ement,
Requi r ement    May Be               we bel i e ve      the requi r ement            mi g ht have been set too l o w to accuratel y
                                      refl e ct the condi t i o ns           expected            when the system i s fi e l d ed.               The cost and
Understated                           operati o nal        effecti v eness         anal y si s ,          from whi c h the cri t eri o n          was
                                      deri v ed,      contai n ed        several     assumpti o ns,                such as the fol l o wi n g,          that we
                                      bel i e ve    understate          the requi r ements                     for ADATS' effecti v eness:

                                  . The anal y si s           assumed       that      ADATS woul d   al w ays       be avai l a bl e ,     that i s , that
                                    i t s avai l a bi l i t y    woul d equal           100 percent.    Thi s avai l a bi l i t y      rate i s greater
                                    than the requi r ement                 of 71       percent   and much greater than ADATS’
                                     demonstrated               performance            of 39 percent     or l e ss.


                                      Page8                                                          GAO/NSIAD-91-51           Ai r   Defense   Anti t ank   System
                                                 0240822




                                             . The si m ul a ted         battl e on whi c h the mi n i m um              pl a toon           requi r ement        was
                                               based showed              that arti l e ry     made a si g ni f i c ant         contri b uti o n           to the out-
                                               come of the battl e , yet arti l e ry              was not used i n the operati o nal                           tests.
                                               Further,         the anal y si s      assumed     that ADATS had a gun, whi c h contri b uted
                                               to i t s performance             i n battl e . The Army currentl y                  has no fi r m pl a ns to
                                               produce         a gun for ADATS.
                                             . The anal y si s       di d not address          a number           of threats to ADATS.

                                                 Another        i n di c ati o n              that the ADATS pl a toon-l e vel               effecti v eness              requi r e-
                                                 ment may be too l o w i s that i t can destroy                                        as many or more fri e ndl y
                                                 ai r craft    than threat ai r craft                           and sti l meet the requi r ement.                     In addi t i o n,
                                                 the requi r ement                     agai n st        pri m ary          targets i s not onl y l o wer than the
                                                 requi r ement                agai n st          secondary            targets, but al s o si g ni f i c antl y           l o wer than
                                                 that requi r ed                 for an i n di v i d ual             fi r e uni t . There i s no di r ect correl a ti o n
                                                 between        the cri t eri a                 contai n ed        i n the requi r ed      operati o nal          capabi l i t y
                                                 documents               and used for fi r e uni t s and the l o wer cri t eri a                               devel o ped             for
                                                 operati o nal            testi n g for pl a toons.


Some Requi r ed                                 The Army had pl a nned                       to     use the operati o nal                tests to assess ADATS' per-
Performance     Capabi l i t i e s              formance         under a vari e ty                  of real i s ti c     condi t i o ns.     However,     because             of
                                                safety consi d erati o ns         and               other factors, the tests di d not suffi c i e ntl y
Were Not Demonstrated                           demonstrate             some capabi l i t i e s             cri t i c al  to performance            on the battl e fi e l d .
                                                Some capabi l i t i e s      were not                 tested, whi l e others were tested but di d not
                                                meet requi r ements,

                                                Shortcomi n gs        were especi a l y                    cri t i c al i n the mi s si l e -fi r i n g phase.                     The
                                                Army al l o cated        11 mi s si l e s               for the operati o nal             tests and actual l y                      fi r ed
                                                onl y 9. These l i m i t ed      tests were not adequate                            to demonstrate             al l
                                                requi r ed     performance             capabi l i t i e s.

                                                Test           and performance               probl e ms         are summari z ed                as fol l o ws:

                                         l  Duri n g     l i v e mi s si l e        fi r i n gs,       onl y one successful                      shot was fi r ed agai n st         a
                                            hoveri n g          hel i c opter.             Our anal y si s          of test data i n di c ates,               however,       that
                                           the hel i c opter                never hovered,                  fl e w too hi g h, and remai n ed                      exposed     too
                                           l o ng to trul y represent                             the threat.
                                     l      The Army di d not measure                                   the system’s            performance              agai n st      maneu-
                                            veri n g    targets duri n g                      ei t her the l i v e mi s si l e         fi r i n gs    or the ground         battl e
                                           testi n g, al t hough                 there i s a requi r ement                      for such a capabi l i t y.
                                         . Testi n g         under adverse                        weather        condi t i o ns    di d not take pl a ce.




                                                 Page      9                                                           GAO/NSLAD-91-51             Ai r   Defense     Anti t ank         System
                                         B-240022




                                    . Informati o n               on the system’s                            capabi l i t y              at ni g ht was not suffi c i e nt
                                      because              safety concerns                      prevented                      threat hel i c opters                       from usi n g real -
                                       i s ti c tacti c s.
                                    . Force-on-force                     tests of the l a ser range fi n der were not conducted                                                                         under
                                       real i s ti c       battl e condi t i o ns.                 However,                       data gathered                     duri n g          the mi s si l e -
                                        fi r i n g phase of testi n g showed                                      that the range fi n der’s                                poor performance
                                      was responsi b l e                     for one mi s sed fi r i n g and one mi s sed engagement.                                                                     In
                                       addi t i o n,         accordi n g             to Army offi c i a l s ,                       data on the range fi n der from the
                                       force-on-force                   tri a l s coul d not be anal y zed.
                                    . The ranges al l o wed                          duri n g         the ground                       battl e s for si m ul a ted                       mi s si l e       fi r -
                                      i n gs exceeded                  those that had been demonstrated                                                          under real i s ti c                 test con-
                                       di t i o ns.        Because the l a ser range fi n der coul d not be used duri n g                                                                            most of
                                       the force-on-force                          testi n g, target range data was someti m es                                                          unavai l a bl e
                                      to the ADATS crew. To compensate,                                                     ADATS was al l o wed                          to fi r e at ranges up
                                       to al m ost twi c e as far as those demonstrated                                                               i n the mi s si l e -fi r i n g                   phase
                                        and was gi v en credi t for these si m ul a ted                                                     ki l s . Army offi c i a l s                    bel i e ve,
                                        however,              that techni c al                 test fi r i n gs                 are suffi c i e nt                to demonstrate                        range
                                       capabi l i t i e s.
                                    . The requi r ement                          for fi r i n g a certai n                        number             of mi s si l e s              per ti m e peri o d
                                        was tested and demonstrated                                            i n onl y one l i v e mi s si l e                         fi r i n g.
                                    . The requi r ement                          that crew members                                 rel o ad         and rearm the system’s                                   mi s -
                                       si l e s was tested but not achi e ved.                                              Al t hough              sol d i e rs       met the ti m e
                                        requi r ement,              the l o aded               mi s si l e s              were not avai l a bl e                      for fi r i n g i n suffi -
                                       ci e nt quanti t i e s.


                                         The Army has deci d ed                           to defer ADATS producti o n,                                        has devel o ped            a new
The Army’s New                           schedul e ,       and i s devel o pi n g                          associ a ted           quanti t y               and cost data for the
Program Proposal              May        program.          Producti o n               i s bei n g postponed                          for at l e ast 2 years; quanti t i e s
Not Overcome                             wi l decrease;
                                         $3.6 mi l i o n
                                                                    and i n i t i a l             esti m ates
                                                               per uni t . The Army hopes to resol v e
                                                                                                                        of uni t costs show an i n crease
                                                                                                                                                            the system’s
                                                                                                                                                                                                of over
                                                                                                                                                                                rel i a bi l i t y
Probl e ms                               probl e ms        and demonstrate                             i t s operati o nal               sui t abi l i t y        through      a fol l o w-
                                         on eval u ati o n          before the next producti o n                                       deci s i o n,            However,      the Army
                                         has not yet determi n ed                            the ful l scope of testi n g i n the proposed                                            schedul e
                                         and may not conduct                            addi t i o nal              operati o nal              testi n g before the ful l - rate
                                         producti o n         deci s i o n.


                                         The Army                  recei v ed    $54.8 mi l i o n        i n research,  devel o pment,       test, and
Congressi o nal       Fundi n g          eval u ati o n             fundi n g    for fi s cal year 1990. It al s o requested,              and Congress
Acti o ns         ”                      appropri a ted,                   $208.9 mi l i o n      i n procurement      fundi n g      that cannot be obl i -
                                         gated unti l               the Secretary            of Defense certi f i e s  that ADA-E successful l y
                                         compl e ted               operati o nal      testi n g. For fi s cal year 1991, the Army requested


                                         Page       10                                                                         GAO/NSIAD-91-61                   Ai r   Defense         Anti t ank       System
   I                              B-240822




                                  $ 2 7 1 .8 m i l i o n        in procurement            fu n d i n g      a n d $ 9 .1 m i l i o n     i n research,
                                  d e v e l o p m e n t,       test, a n d eval u ati o n           fu n d s for A D A T S - r e l a te d p r o d u c t
                                  i m p r o v e m e n ts.

                                  S i n ce th e c o m p l e ti o n        o f o u r revi e w, th e c o n fe r e n c e                        c o m m i tte e s        on
                                  d e fe n s e a u thori z ati o n           a n d o n d e fe n s e appropri a ti o ns                             h a v e d e n i e d th e
                                  A rmy’s r e q u e s t for p r o c u r e m e n t                  fu n d i n g         i n fi s cal year 1 9 9 1 . T h e c o n fer-
                                  e n c e c o m m i tte e        o n d e fe n s e appropri a ti o ns                       al s o r e s c i n d e d        th o s e procure-
                                  m e n t fu n d s th a t h a d b e e n a p p r o p r i a t e d                    i n fi s cal year 1 9 9 0 .” H o w e v e r ,
                                  th e A rmy r e q u e s te d            a n d fu n d s w e r e a u th o r i z e d                   and appropriated                     i n th e
                                  a m o u n t o f $ 9 2 m il ion              for c o n ti n u e d           research, d e v e l o p m e n t,                     test, a n d
                                  eval u ati o n       o f A D A T S .B e c a u s e o f c o n ti n u i n g                 congressional                  concerns about
                                  A D A T Sperformance,                th e c o n fe r e n c e          c o m m i tte e           report o n d e fe n s e a p p r o -
                                  pri a ti o ns     restri c ts hal f o f th a t fu n d i n g                   u n ti l o th e r studi e s a r e c o m p l e te d .


                                  T h e D e p a r tm e n t     o f D e fe n s e ( D O D ) p r o v i d e d    o ffi c i a l      c o m m e n ts o n th e con-
A g e n cy C o m m e n ts   and   te n ts o f o u r draft report. ( S e e a p p . I.) It d i d n o t a d d r e s s th e speci f i c
O u r E val u ati o n             fi n d i n g s   i n th e report, b u t r e c o g n i z e d        th e seri o usness               o f th e m . D O D al s o
                                  p o i n te d o u t th a t further testi n g a n d D O D revi e ws o f th e restructured
                                  A D A T S p r o g r a m wi l ta k e p l a c e pri o r to a ful l - rate                  p r o d u c ti o n     deci s i o n.

                                  In l i g ht o f th e severi t y        o f rel i a bi l i t y,    avai l a bi l i t y,         a n d m a i n tai n abi l i t y
                                  probl e ms,        escal a ti n g   system costs, a n d th e c o m p r e s s e d                           2-year s c h e d u l e
                                  wi t hi n w h i c h th e A rmy p l a n s to i d e n ti f y                        a n d resol v e p r o b l e m s              a n d test
                                  fi x es, th e C h a i r m a n     o f th e H o u s e A r m e d Servi c es C o m m i tte e                                  has
                                  r e q u e s te d   th a t w e revi e w th e A rmy’s n e w A D A T Sp r o g r a m . In a d d i ti o n ,
                                  th e report o f th e H o u s e C o m m i tte e                 o n A p p r o p r i a ti o n s           o n th e fi s cal year
                                  1 9 9 1 d e fe n s e b u d g e t r e q u e s ts th a t w e eval u ate                      th e p l a n n e d       testi n g
                                  program.


                                  Appendix        II sets forth th e obj e cti v es,                    s c o p e , a n d m e th o d o l o g y           o f thi s
                                  revi e w. A s a r r a n g e d wi t h your o ffi c e, unl e ss y o u publ i c l y                                  announce       its
                                  c o n te n ts earl i e r, w e p l a n n o further di s tri b uti o n                           o f thi s report u n ti l 3 0
                                  d a y s from i t s i s sue d a te . A t th a t ti m e , w e wi l s e n d copi e s to th e
                                  Chairmen        o f th e S e n a te C o m m i tte e s              o n A p p r o p r i a ti o n s              and on Govern-
                                  m e n tal A ffai r s a n d th e H o u s e C o m m i tte e s                      o n A p p r o p r i a ti o n s       and on
                                  Government              O p e r a ti o n s , th e S e c r e tari e s o f D e fe n s e a n d th e A rmy, a n d


                                  2 T h e s e funds w e r e i d enti f i e d for potenti a l    reducti o ns   i n o u r report D e f e n s e B u d g e t : Potenti a l   Reduc-
                                  ti o ns to the A r m y a n d N a v y Mi s si l e P r o g r a m s ( G A O / N S I A D 8 0 3 0 m ,           Sept. 1 9 9 0 ) .



                                  Page     11                                                               G A O /NSIAD-91-61            A i r Defense      Anti t ank    System
B-240822                                                                                                            ,




the Di r ector         of the Offi c e of Management            and Budget.        Copi e s       wi l     al s o       be
made avai l a bl e         to other i n terested  parti e s      on request.

Pl e ase         cal l me at (202) 275-4141       if you or your staff have any questi o ns.
Maj o r         contri b utors  to the report     are l i s ted i n Appendi x III.




Ri c hard          Davi s
Di r ector,         Army    Issues




Page       12                                          GAQ/NSLAD-91-61     Ai r   Defense     Anti t ank       System
Page   13   GAO/NSIAD-91-51   Ai r   Defense   Anti t ank   System
                                                                                                                                                ,
Contents


Letter                                                                                                                                                 1

Appendi x  I                                                                                                                                      16
Comments     From        the
Department     of
Defense
Appendi x         II                                                                                                                              17
Obj e cti v es,     Scope, and
Methodol o gy
Appendi x      III
Maj o r Contri b utors         to
Thi s Report
Fi g ure                            Fi g ure            1: The   Ai r Defense   Anti t ank   System                                                   3




                                    Abbrevi a ti o ns

                                    ADATS                  Ai r Defense    Anti t ank System
                                    DOD                    Department      of Defense


                                    Page       14                                            GAO/NSIAD-91-51   Ai r   Defense   Anti t ank   System
.




    Page   16   GAO/NSIAD-91-61   Ai r   Dkfense   Anti t ank   System
Appendi x   I

Comments        From                    the Departxnent                                                                  of Defense’                                         I*



                                                                            OFFICE               OF   THE   DIRECTOR              OF
                                                                  DEFENSE RESEARCH AND ENGINEERING
                                                                                 WASHINGTON,                  DC      20301

                                                                                                                                           27 SEP 1990

                   Mr. Frank            C. Conahan
                   Assi s tant          Comptrol l e r                     General
                   Nati o nal          Securi t y          and
                       Internati o nal                Affai r s                 Di v i s i o n
                   U.S. General                 Accounti n g                    Offi c e
                   Washi n gton,            DC 20548
                   Dear            Mr.     Conahan:
                             Thi s   is the Department       of Defense                                               (DOD) response       to the
                   General         Accounti n g    Offi c e (GAO) draft                                              report,      "Army Acqui s i t i o n:
                   Operati o nal         Testi n g  of the Ai r Defense                                                Anti t ank    System Reveal s
                   Seri o us       Weaknesses",       dated  13 September                                                1990 (GAO Code 393345/
                   OSD Case 8461).
                                  The     DOD recogni z es           your concern                wi t h                          the seri o usness      of the
                   fi n di n gs            speci f i e d     i n the draft               report.                                It is not possi b l e        to
                   address               each fi n di n g        speci f i c al l y         because                               the fi n al      Army test
                   data            is    not yet avai l a bl e .                    Upon recei p t                              of that data we wi l
                   exami n e             it i n detai l .
                                       The Army has approved                    a restructured                    program          to correct
                    rel i a bi l i t y               probl e ms     wi t h     the system.                   Addi t i o nal          testi n g            is an
                   i n tegral                  part of the restructured                             program.                We have schedul e d
                   Conventi o nal                      Systems      Commi t tee                 (CSC) and Defense                  Acqui s i t i o n
                   Board (DAB) revi e ws                          on 31 October                     1990,     and 16 November                        1990,
                   respecti v el y ,                     to assess          the vi a bi l i t y            of the restructured                           Li n e
                   of Si g ht-Forward-Heavy                               program.                Further       testi n g          as wel l            as
                   cost,                  performance,          schedul e ,         and supportabi l i t y                       are to be
                   addressed                     i n that process.
                                 Assumi n g      Congressi o nal                         authori z ati o n                      and appropri a ti o n                of
                   the necessary                   funds,          authori t y                       to proceed                   i n to        ful l       rate
                   producti o n             wi l       be conti n gent                           upon approval                           by the Defense
                   Acqui s i t i o n           Board.            Based upon successful                                            compl e ti o n             of the BAM
                   Maturati o n             Phase,        whi c h             i n cl u des               a Fol l o w-On                    Eval u ati o n
                   (Operati o nal                Test),          certi f i c ati o n                       wi l        be         provi d ed              per the
                   fi s cal          year 1989 Defense                            Authori z ati o n                    Act             (P.L. 100-456)              that
                   the system               is operati o nal l y                           sui t abl e             and          effecti v e.

                                                                                                               Si n cerel y ,



                                                                                                              Frank       Kendal l
                                                                                                              Acti n g     Deputy        Di r ector
                                                                                                              (Tacti c al          Warfare        Programs)



                                  Page16                                                                           GAO/NSIALI-91.51             Ai r Defense   Anti t ank   System
Append@   II

obj e cti v es,   Scope, and Methodol o gy


                        As requi r ed       by the fi s cal year 1989 Nati o nal                  Defense Authori z ati o n            Act
                        (P.L. l O O-456),        we eval u ated    the performance                  of the Ai r Defense Anti -
                        tank System (ADATS), the Army’s                    l i n e-of-si g ht      forward        heavy ai r defense
                        system, duri n g         operati o nal  testi n g.’ Our obj e cti v e            was to determi n e
                        whether         the system had adequatel y                   demonstrated          i t s sui t abi l i t y and
                        effecti v eness       for combat.

                        Fi n al Army and Offi c e of the Secretary                       of Defense reports on the resul t s
                        of ADATS' operati o nal     testi n g were not avai l a bl e               when we compl e ted           our
                        revi e w. As a resul t , thi s report i s based on our anal y si s                   of i n teri m    test
                        resul t s and other program-rel a ted             documents,             our observati o ns        of the
                        operati o nal  tests, and our di s cussi o ns                wi t h Army and Offi c e of the Sec-
                        retary of Defense offi c i a l s .    Our cal c ul a ti o ns          are based on the programs
                        and the data base of the Army’s               Operati o nal            Test and Eval u ati o n
                        Agency.

                        Between January                and May 1990, we attended                   al l operati o nal        tests at
                        Whi t e Sands Mi s si l e          Range, New Mexi c o,      and the maj o ri t y               of tests and
                        post-tri a l        meeti n gs   at Fort Hunter-Li g gett,        Cal i f orni a .        We di s cussed     the
                        conduct           of the tests, thei r resul t s, and anal y si s              wi t h numerous          test
                        parti c i p ants.

                        We revi e wed           test pl a nni n g    documents             devel o ped          by the ADATS Program
                        Manager,           the Operati o nal        Test and Eval u ati o n              Agency, other Army
                        organi z ati o ns,        and the Di r ector        of Operati o nal             Test and Eval u ati o n                          in
                        the Offi c e of the Secretary                 of Defense. We di s cussed                             the ADATS program
                        and test-rel a ted           i s sues wi t h Army and defense                     i n tel l i g ence             offi c i a l s ;    the
                        Forward            Area Ai r Defense System Program Executi v e                                         Offi c er; the ADATS
                        Program            Manager;         Test and Experi m entati o n                Command                   test conductors;
                        and offi c i a l s      of the Ai r Defense Arti l e ry                  School and Center, the Opera-
                        ti o nal       Test and Eval u ati o n        Agency, the Logi s ti c s                 Eval u ati o n             Agency, the
                        Army Materi e l             Systems Anal y si s       Acti v i t y,         Headquarters                   of the Depart-
                        ment of the Army, and the Offi c e of the Secretary                                             of Defense.

                        We exami n ed              the Army’s    i n ternal    control s      for ensuri n g    the val i d i t y       of
                        test data by revi e wi n g            the i n strumentati o n          and audi o , vi d eo, and
                        manual        systems for tri a l data col l e cti o n,            The Army recogni z ed                 that the
                        data col l e cti o n         system woul d contai n           errors. However,         due to ti m e con-
                        strai n ts,      we di d not test those systems to quanti f y                      the magni t ude           of
                        potenti a l        errors.


                        T‘he   act al s o   requi r es   that the Di r ector,   Operati o nal     Test and Eval u ati o n,          perform   a si m i l a r   revi e w.



                        Page    17                                                              GAO/NSIAD=91-51              Ai r   Defense     Anti t ank         System
Appendi x              II
Obj e cti v es,             Scope,   and   Methodol o gy




We al s o revi e wed                    i n ternal           control s        for ensuri n g            accuracy           i n data anal -
 ysi s and conducted                          a l i m i t ed     test of those control s .                    On the basi s of our
observati o ns                 duri n g        tri a l s and attendance                      at post-tri a l        meeti n gs,             we
i d enti f i e d     anomal i e s               i n tri a l condi t i o ns            and conduct.             We traced sel e cted
anomal i e s          through                the Army’s                data val i d ati o n           processes           to determi n e
 how Army eval u ators                                 had treated anomal i e s.                   Due to the compl e xi t y                        and
vol u me         of generated                     data and the numerous                          anomal i e s        i d enti f i e d        duri n g
the tri a l s , we di d not quanti f y                               the i m pact           of tri a l anomal i e s                  on the
Army’s           concl u si o ns.              Further,           we di d not veri f y                 the Operati o nal                 Test and
Eval u ati o n           Agency’s                 programs.              Therefore,           our current           anal y si s           accepted
the Army-devel o ped                              data as val i d .

We conducted     our work from                                 December    1989 to August         1990                              i n accor-
dance wi t h general l y accepted                               government     audi t i n g standards.




Page              18                                                         GAO/NSIAD-91-61                Ai r   Defense      Anti t ank      System
Appendi x         III

Maj o r                 Contri b utors                     to Thi s                           Report


                                         Henry             L. Hi n ton, Associ a te                 Di r ector
Nati o nal      Securi t y and           Raymond                    Dunham,          Assi s tant          Di r ector
Internati o nal         Affai r s        Katheri n e                 V. Schi n asi ,       Eval u ator-i n -Charge
Di v i s i o n,         Washi n gton,    Robert             Shi e l d s,   Eval u ator
                                         Jai Eun             Lee, Computer                   Programmer              Anal y st
D.C.                                     Eugene              Thompson,               Consul t ant


                                         Fl o yd       Ortega, Regi o nal     Assi g nment                          Manager
San Franci s co                          Chri s ti n e      Frye, Eval u ator
Regi o nal    Offi c e
                                         Robert             Thames,           Regi o nal        Assi g nment            Manager
Denver              Regi o nal           Wi l i a m           Wri g ht,       Eval u ator
Offi c e




                                          Page        19                                                             GAO/NSi A D-91-51   Ai r   Defense   Anti t ank   System
*.
     l
Ordc*ri n g         Informati o n

The first fi v e copi e s          of each GAO report                are free. Addi t i o nal    copi e s
8rti $2 each. Ordt!rs             shoul d       be sent t,o thp fol l o wi n g     address,     accom-
pani t d   by a check          or money           order      made   out to the Superi n tendent
of IIocuments,            when      necessary.          Orders    for 100 or more copi e s         to be
mai l c bd to a si n gl e      address         are di s counted        25 percent.

1J.S. General   Accounti n g                      Offi c e
P.O. Box 60 15
Gai t hersburg,   MI) 20877

Orders        may        al s o     be pl a ced       by cal l i n g   (202)   2766241.