oversight

Airborne Self-Protection Jammer

Published by the Government Accountability Office on 1997-01-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

      United States
GAO   General Accounting
      Washington,
                            Office
                    D.C. 20548

      National Security and
      International Affairs Division



      B-275469

      January 29, 1997

      The Honorable William V. Roth, Jr.
      United States Senate

      Dear Senator Roth:

      As you requested, we have reviewed the testing and data now available concerning
      the operational effectiveness and suitability of the Airborne Self-Protection Jammer
      (ASPJ) to determine whether the results justify restarting production, Additionally,
      we are providing information on whether alternative electronic countermeasure
      systems are available to the Department of the Navy to perform the same missions.

      BACKGROUND

      The ASPJ was designed to help protect Navy and Air Force fighter aircraft from
      radar-guided weapons. During the 1980s and early 1990s the Navy and Air Force
      planned to spend about $4.8 billion developing and producing it. At one time, they
      planned to buy as many as 2,400 units. In 1992, however, poor test results led
      Congress to deny funds for further production. At the time of contract termination,
      the Navy had placed orders for 136 systems and spare components for a contract
      value of $547.9 million. The 95 systems that had been delivered were placed in
      storage and production was stopped. During its 1992 Operational Test and
      Evaluation on the F/A-18 aircraft, the ASPJ did not meet alI operational
      effectiveness or suitability’ thresholds, and failed to demonstrate the threshold
      improvement required over the jammer it was intended to replace, the ALQ-126B.

      In June 1995, Air Force Captain Scott O’Grady’s F-16 aircraft was shot down over
      the Balkans by a surface to air missile.’ Marine Corps F/A-l% equipped with the
      AL&-126B were also flying in the Balkans theater of operations without active
      self-protection against that threat. The 1992 ASPJ Operational Test and Evaluation
      had shown in a laboratory simulation using actual threat hardware that the ASPJ,


      ‘Operational effectiveness refers to the ability of a system to complete its assigned
      mission. Operational suitability refers to a system’s ability to be used successfully
      taking into account factors such as reliability and maintainability.
      ‘Discussion of specific surface to air missile threats in relation to jammer
      performance is considered classified by the Navy.
                                                        GAOAWAD-B74R   Airborne   Self-ProMon   Jammer


                                     /uO~
B-275469



despite its other problems, was three times more effective than the requirement for
that particular threat. Realizing this, the Navy requested and received authority in
June 1995 from the Secretary of Defense to take ASPJ systems out of storage and
deploy them in the Bosnian theater of operations on Marine Corps F/A-18s for
contingency purposes. The 1992 test results also indicated ASPJ was only slightly
better or comparable to the ALQ-126B against two of the other three types of threat
missiles in the Bosnian theater of operations. Neither ASPJ nor the ALQ-126B was
effective against the third threat.

RESULTS IN BRIEF

Test results and operational data now available do not support restarting Airborne
Self-Protection Jammer production. Reasons for this are the limitations in the
scope of recent testing and a lack of demonstrated improvements in test results
since the 1992 operational evaluation. Additionally, although ASPJ systems were
deployed on aircraft operating over Bosnia, no quantifiable effectiveness data could
be gathered during those operations. Furthermore, according to the Office of the
Director of Operational Test and Evaluation and the Navy’s Operational Test and
Evaluation Force, neither the recent test results nor operational performance
support restarting production. Moreover, the Navy’s long-term plan is to acquire the
Integrated Defensive Electronic Countermeasure (IDECM) system instead of
restarting ASPJ production.

According to the Navy, no readily available self-protection alternative to the ASPJ
was available to counter one particular threat missile system when ASPJ was
deployed for operations over Bosnia in June 1995. However, recognizing the Navy
has plans to procure the IDECM system, Congress has directed the Navy to report
on electronic warfare alternatives not later than February 15, 1997. As part of a
separate, ongoing review, we are also looking at potential alternative electronic
warfare systems and expect to report the results of our work in early spring 1997.

RECENT TEST RESULTS HAVE NOT
DEMONSTRATED IMPROVEMENT

After ASPJ production was terminated in 1992, the Navy found itself with no
jammer for its F-14D aircraft. Unlike its other F-14 and F/A-18 fighters, the F-14D
was configured so it could carry only the ASPJ. Subsequently, the Navy proposed
taking some ASPJs out of storage and placing them on F-14Ds, if they could pass




Page 2                                           GAO/NSIAD-9746R   Airborne   Self-Protection   Jammer
B-275469



new tests. Instead of the measures of effectiveness used during the 1992 tests,3
however, ASPJ was to be tested to determine only whether the F-14D was more
survivable with the ASPJ than without it. In 1995 and 1996, as part of a previously
planned operational evaluation of the F-14D aircraft, the ASPJ testing was
conducted. A preliminary test report from the Navy concluded that the F-14D is
more survivable with the ASPJ than without it.

After Captain O’Grady was shot down, the Navy requested authority to deploy
ASPJs on F/A-18 aircraft operating over the Balkans. Subsequent testing was done
on the F/A-18 during the June-July 1995 time frame at China Lake, California, to
determine whether changes in the aircraft avionics/weapon systems and the
operational flight program software since the 1992 ASPJ operational evaluation had
degraded the integration of ASPJ with the aircraft. Results of the integration tests
showed that ASPJ conflicted with another system on the aircraft4 According to
Navy officials, this was corrected in a subsequent software upgrade.

According to DOD, the 1995 testing was also done to demonstrate that ASPJ
performance against three specific threats was consistent with 1992 operational
testing results and comparable to the ALQ-126B. Performance results were
consistent with 1992 results. However, the number of test runs against the three
threat radars was very limited. According to the Director of Operational Test and
Evaluation (DOT&E) and the Navy’s Operational Test and Evaluation Force
(OPTEVFOR), the limited tests were adequate for determining consistency with
prior operational test results but did not provide sufficient data to determine
effectiveness against a broad mix of threats for purposes of making production
decisions. In addition, the 1995 testing at China Lake did not include performance
against the type of missile that hit Captain O’Grady’s aircraft in June 1995.

After the Secretary of Defense approved the decision to deploy the ASPJ to the
Balkans, DOT&E took the initiative to ensure the Navy tested the ASPJ on the
F/A-18 against the threat that downed Captain O’Grady’s aircraft to confirm the 1992
lab results. Test flights were conducted at Eglin Air Force Base, Florida, in August
1995. Both DOT&E and OPTEVFOR concluded from the test results that ASPJ
would work against this threat.

However, the recent F-14D, China Lake and Eghn tests (1) did not demonstrate any
improvement in effectiveness since ASPJ’s 1992 operational evaluation and (2) were


3The 1992 testing was designed to determine whether ASPJ could (1) improve
overall survivability by 30 percent against a required mix of threats compared with
an aircraft with no jammer and (2)provide an increase in survivability equal to or
greater than the jammer it was intended to replace, the AL&-126B.
4The specific system affected is considered classified by the Navy.
Page 3                                           GAO/NSIAD-97-46R Airborne Self-Protection   Jammer
B-275469



too limited to conclude that ASPJ could be operationally effective against the
required mix of threats faced in the 1992 tests. Additionally, ASPJ effectiveness
could not be determined from the Balkans data because there was no means in
place of recording data involving threat engagements with F/A-l&. According to
DOT&E and OPTEVFOR, neither the test results nor the operational performance
support a conclusion that ASPJ is operationally effective relative to the 1992 test
criteria. Also, both of these organizations concluded that these limited test results
do not justify restarting ASPJ production.

INSUFFICIENT OPERATIONAL DATA
WAS COLLECTED IN THE BALKANS
THEATER OF OPERATIONS

Representatives of DOT&E and OPTEVFOR traveled to the Balkans theater of
operations to convince the Marine Corps F/A-18 squadron deployed there to
implement a process established for collecting ASPJ suitability data. An official of
DOT&E credits your interest in the program with contributing to DOT&E’s resolve
to ensure that ASPJ suitability data was gathered and analyzed by the Navy.

In assessing suitability, OPTEVFOR identified significant limitations in the data
gathered in the Balkans theater of operations. For example, mean time to repair
data was not available. Further, the data gathered did not include sufficient detail
to assess whether threat radar signals were stimulating the ASPJ to transmit during
flights in the Balkans. (The jammer should only transmit a jamming signal when it
detects an actual threat. Without confirmation that the jammer was being
stimulated, whether it was working properly remains unknown.) Test officials
emphasized that these are inherent limitations in trying to assess suitability based
on the collection of operational field data with the resources that were available.

In addition, though there were multiple built-in-test failure indications on the ground
as well as in flight,5 aircrews continued their missions regardless of indications that
ASPJ might not be working. In flight, if necessary, they made repeated attempts to
clear built-in-test failure indications and achieve a “go” status from the ASPJ.
According to Navy officials, this is standard operating procedure in the fleet for
built-in-test systems. Built-in-test indications were counted as failures of ASPJ only
if they were not cleared. Nevertheless, this is si,@ificant because all built-in-test
failure indications were scored as mission critical failures during the 1992
operational test and this resulted in ASPJ’s inability to meet suitability thresholds.
Such deficiencies cause questions to remain regarding ASPJ suitability for the
F/A-18.



5The built-in-test subsystem is supposed to let users know whether or not ASPJ is
working.
Page 4                                             GAO/NSIAD-B746R   Airborne Self-Protection   Jammer
B-275469



Consequently, although suitability data collected during the Balkan mission
appeared to show some improvement in ASPJ compared with the 1992 operational
test, OPTEVFOR has concluded that it could not use that data to reach a conclusion
regarding improvement in the built-in-test function. Moreover, suitability
measurement was statistically skewed in favor of ASPJ success because no
missions were terminated after the built-in-test system indicated system failure.
Hence, the total number of flight hours continued to accumulate, whether or not
ASPJ was working. Navy officials maintain that because the faihrre indications
were cleared in flight or could not be duplicated during post-flight maintenance, the
results were not skewed. However, our review indicated that the built-in-test
subsystem was not able to provide unequivocal information as to the status of
ASPJ. This was a concern that contributed to ASPJ being rated unsuitable during
the 1992 Operational Test and Evaluation and it continues to exist.

In Bosnia, the mean time between operational mission failures was computed as
45.1 flight hours, exceeding the 1992 test requirement threshold of 33.3 hours.
However, this was calculated not in accordance with the 1992 test methodology
counting all built-in-test failure indications as failures, but in accordance with what
the Navy maintains is a standard fleet procedure in which aircrews continue to try
to recycle the built-in-test subsystem until the failure indication is cleared. If
meantime between operational mission failure was computed in accordance with
the 1992 test methodology, it would have been 31.1 hours, below the requirement
threshold.

CONGRESS DIRECTS STUDY
OF ALTERNATIVE SYSTEMS

According to the Navy, it had no readily available self-protection alternative to the
ASPJ for one particular threat missile system in June 1995 when ASPJ was deployed
for operations over the Balkans. Navy officials indicated that Congress supported
the Navy’s plans to field ASPJ systems delivered prior to the 1992 ASPJ production
termination. Looking to the future, however, the Navy’s long-term plan for fighter
electronic self-protection is the joint Navy/Air Force IDECM system, which
combines an electronic techniques generator to deceive radar with a towed decoy
system. (A system that incorporates a towed decoy is envisioned to have greater
capability against certain types of threats than an onboard jammer alone.) Navy
officials maintain, however, that due to difficulty they may have retrofitting IDECM,
they must investigate restarting ASPJ production for older aircraft.

Congress recently provided $47.9 million to the Navy to procure 36 ASPJ systems
for 3 deployed F/A-l8 C/D squadrons. While providing these funds for ASPJ,
according to the Conference Report on the National Defense Authorization Act for
Fiscal Year 1997, the conferees stated that their decision did not reflect a
commitment for additional procurement of ASPJ systems or to restart production
for U.S. government customers at this time. The congressional conferees further
Page 5                                             GAO/NSIAD-97-46R Airborne   SeEProtection   Jammer
B-275469



recognized that the Navy is expecting the IDECM to serve as the long-term
electronic countermeasure system for the F/A-18 E/F and they want the Navy to
explore long-term electronic countermeasure solutions for the F/A-18 C/D.

In addition, the Senate Appropriations Committee has directed the Navy to provide
a report that delineates the costs (including operations and support costs), systems
availability, and operational advantages and disadvantages of adapting IDECM
components for the F/A-18 C/D fighter compared to alternative electronic warfare
systems. The Committee directed the Navy to submit this report to the
congressional defense committees no later than February 15, 1997. As part of a
separate, ongoing review, we are also looking at the IDECM and ALE-50 towed
decoy systems and expect to report on this issue in early spring 1997.

AGENCY COMMENTS

In commenting on a draft of this letter, the Department of Defense (DOD) generally
concurred with our findings. Writing for DOD, DOT&E acknowledged that the 1995
and 1996 testing and deployed ASPJ operations were not adequate to support a
conclusion that ASPJ is effective and suitable relative to the requirements against
which ASPJ was tested in 1992. However, DOD reiterated that (1) testing
conducted to date has been adequate to find ASPJ effective against the selected
threats of interest for the Balkans contingency operations, (2) the ASPJ
effectiveness against selected Bosnian threats is important in these contingency
operations, and (3) the suitability of ASPJ has been adequate for the requirements
placed on it during contingency operations. However, DOD stated that many
questions remain unanswered.

DOD also provided a number of technical comments designed to enhance the
clarity, accuracy, and completeness of the report. We have incorporated them in
the report where appropriate.

SCOPE AND METHODOLOGY

We performed work at DOT&E in Washington, D.C., and OPTEVFOR in Norfolk, VA.
To determine whether test results supported restarting ASPJ production, we
compared 1995 and 1996 testing and test results and data from flight operations
over the Balkans with data from ASPJ’s 1992 operational evaluation that led to
ASPJ production termination. We discussed this data with representatives of
DOT&E and OPTEVFOR and drew on their conclusions as well. In reviewing
alternative systems, we performed work at the Naval Air Systems Command,
McDonnell-Douglas Aircraft, and Lockheed/Martin contractor locations. We
interviewed responsible agency officials and reviewed applicable documents. We
conducted our review between August and December 1996 in accordance with
bGenerally accepted government auditing standards.

Page 6                                            GAOLWAD-97-46R   Airborne   Self-Protection   Jammer
B-275469




We are sending copies of this letter to interested congressional committees, the
Secretary of Defense, the Secretary of the Navy, and the Director of the Office of
Management and Budget. We will also provide copies to others upon request.

If you or your staff have questions, please contact me at (202) 512-4841. Major
contributors to this assignment were Paul Latta, Robert Coleman, Henry Arzadon,
Terry Parker and Charles Ward.

Sincerely yours,




Louis J. Rodrigues
Director, Defense Acquisition Issues




(707187)
Page 7                                           GAOINSIAD-97-46R Airborne   Self-Protection   Jammer
 .
-,.:-
Ordering      Information

The first copy of each GAO report and testimony is free.
Additional   copies are $2 each. Orders should be sent to the
following  address, accompanied by a check or money order
made out to the Superintendent     of Documents, when
necessary. VISA and MasterCard      credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted    25 percent.

Orders      by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner         of 4th and G Sts. NW)
U.S. General Accounting          Office
Washington,  DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 2534066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony.   To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information         on
how to obtain these lists.

For information         on how to access GAO reports on the INTERNET,
send an e-mail         message with “info” in the body to:

info@www.gao.gov

or visit    GAO’s World      Wide Web Eome Page at:

http#www.gao.gov
United States
General Accounting  Office
Washington, D.C. 20548-0001


Official   Business
Penalty    for Private   Use $300    .

Address    Correction    Requested