United States GAO General Accounting Washington, Office D.C. 20548 National Security and International Affairs Division B-275469 January 29, 1997 The Honorable William V. Roth, Jr. United States Senate Dear Senator Roth: As you requested, we have reviewed the testing and data now available concerning the operational effectiveness and suitability of the Airborne Self-Protection Jammer (ASPJ) to determine whether the results justify restarting production, Additionally, we are providing information on whether alternative electronic countermeasure systems are available to the Department of the Navy to perform the same missions. BACKGROUND The ASPJ was designed to help protect Navy and Air Force fighter aircraft from radar-guided weapons. During the 1980s and early 1990s the Navy and Air Force planned to spend about $4.8 billion developing and producing it. At one time, they planned to buy as many as 2,400 units. In 1992, however, poor test results led Congress to deny funds for further production. At the time of contract termination, the Navy had placed orders for 136 systems and spare components for a contract value of $547.9 million. The 95 systems that had been delivered were placed in storage and production was stopped. During its 1992 Operational Test and Evaluation on the F/A-18 aircraft, the ASPJ did not meet alI operational effectiveness or suitability’ thresholds, and failed to demonstrate the threshold improvement required over the jammer it was intended to replace, the ALQ-126B. In June 1995, Air Force Captain Scott O’Grady’s F-16 aircraft was shot down over the Balkans by a surface to air missile.’ Marine Corps F/A-l% equipped with the AL&-126B were also flying in the Balkans theater of operations without active self-protection against that threat. The 1992 ASPJ Operational Test and Evaluation had shown in a laboratory simulation using actual threat hardware that the ASPJ, ‘Operational effectiveness refers to the ability of a system to complete its assigned mission. Operational suitability refers to a system’s ability to be used successfully taking into account factors such as reliability and maintainability. ‘Discussion of specific surface to air missile threats in relation to jammer performance is considered classified by the Navy. GAOAWAD-B74R Airborne Self-ProMon Jammer /uO~ B-275469 despite its other problems, was three times more effective than the requirement for that particular threat. Realizing this, the Navy requested and received authority in June 1995 from the Secretary of Defense to take ASPJ systems out of storage and deploy them in the Bosnian theater of operations on Marine Corps F/A-18s for contingency purposes. The 1992 test results also indicated ASPJ was only slightly better or comparable to the ALQ-126B against two of the other three types of threat missiles in the Bosnian theater of operations. Neither ASPJ nor the ALQ-126B was effective against the third threat. RESULTS IN BRIEF Test results and operational data now available do not support restarting Airborne Self-Protection Jammer production. Reasons for this are the limitations in the scope of recent testing and a lack of demonstrated improvements in test results since the 1992 operational evaluation. Additionally, although ASPJ systems were deployed on aircraft operating over Bosnia, no quantifiable effectiveness data could be gathered during those operations. Furthermore, according to the Office of the Director of Operational Test and Evaluation and the Navy’s Operational Test and Evaluation Force, neither the recent test results nor operational performance support restarting production. Moreover, the Navy’s long-term plan is to acquire the Integrated Defensive Electronic Countermeasure (IDECM) system instead of restarting ASPJ production. According to the Navy, no readily available self-protection alternative to the ASPJ was available to counter one particular threat missile system when ASPJ was deployed for operations over Bosnia in June 1995. However, recognizing the Navy has plans to procure the IDECM system, Congress has directed the Navy to report on electronic warfare alternatives not later than February 15, 1997. As part of a separate, ongoing review, we are also looking at potential alternative electronic warfare systems and expect to report the results of our work in early spring 1997. RECENT TEST RESULTS HAVE NOT DEMONSTRATED IMPROVEMENT After ASPJ production was terminated in 1992, the Navy found itself with no jammer for its F-14D aircraft. Unlike its other F-14 and F/A-18 fighters, the F-14D was configured so it could carry only the ASPJ. Subsequently, the Navy proposed taking some ASPJs out of storage and placing them on F-14Ds, if they could pass Page 2 GAO/NSIAD-9746R Airborne Self-Protection Jammer B-275469 new tests. Instead of the measures of effectiveness used during the 1992 tests,3 however, ASPJ was to be tested to determine only whether the F-14D was more survivable with the ASPJ than without it. In 1995 and 1996, as part of a previously planned operational evaluation of the F-14D aircraft, the ASPJ testing was conducted. A preliminary test report from the Navy concluded that the F-14D is more survivable with the ASPJ than without it. After Captain O’Grady was shot down, the Navy requested authority to deploy ASPJs on F/A-18 aircraft operating over the Balkans. Subsequent testing was done on the F/A-18 during the June-July 1995 time frame at China Lake, California, to determine whether changes in the aircraft avionics/weapon systems and the operational flight program software since the 1992 ASPJ operational evaluation had degraded the integration of ASPJ with the aircraft. Results of the integration tests showed that ASPJ conflicted with another system on the aircraft4 According to Navy officials, this was corrected in a subsequent software upgrade. According to DOD, the 1995 testing was also done to demonstrate that ASPJ performance against three specific threats was consistent with 1992 operational testing results and comparable to the ALQ-126B. Performance results were consistent with 1992 results. However, the number of test runs against the three threat radars was very limited. According to the Director of Operational Test and Evaluation (DOT&E) and the Navy’s Operational Test and Evaluation Force (OPTEVFOR), the limited tests were adequate for determining consistency with prior operational test results but did not provide sufficient data to determine effectiveness against a broad mix of threats for purposes of making production decisions. In addition, the 1995 testing at China Lake did not include performance against the type of missile that hit Captain O’Grady’s aircraft in June 1995. After the Secretary of Defense approved the decision to deploy the ASPJ to the Balkans, DOT&E took the initiative to ensure the Navy tested the ASPJ on the F/A-18 against the threat that downed Captain O’Grady’s aircraft to confirm the 1992 lab results. Test flights were conducted at Eglin Air Force Base, Florida, in August 1995. Both DOT&E and OPTEVFOR concluded from the test results that ASPJ would work against this threat. However, the recent F-14D, China Lake and Eghn tests (1) did not demonstrate any improvement in effectiveness since ASPJ’s 1992 operational evaluation and (2) were 3The 1992 testing was designed to determine whether ASPJ could (1) improve overall survivability by 30 percent against a required mix of threats compared with an aircraft with no jammer and (2)provide an increase in survivability equal to or greater than the jammer it was intended to replace, the AL&-126B. 4The specific system affected is considered classified by the Navy. Page 3 GAO/NSIAD-97-46R Airborne Self-Protection Jammer B-275469 too limited to conclude that ASPJ could be operationally effective against the required mix of threats faced in the 1992 tests. Additionally, ASPJ effectiveness could not be determined from the Balkans data because there was no means in place of recording data involving threat engagements with F/A-l&. According to DOT&E and OPTEVFOR, neither the test results nor the operational performance support a conclusion that ASPJ is operationally effective relative to the 1992 test criteria. Also, both of these organizations concluded that these limited test results do not justify restarting ASPJ production. INSUFFICIENT OPERATIONAL DATA WAS COLLECTED IN THE BALKANS THEATER OF OPERATIONS Representatives of DOT&E and OPTEVFOR traveled to the Balkans theater of operations to convince the Marine Corps F/A-18 squadron deployed there to implement a process established for collecting ASPJ suitability data. An official of DOT&E credits your interest in the program with contributing to DOT&E’s resolve to ensure that ASPJ suitability data was gathered and analyzed by the Navy. In assessing suitability, OPTEVFOR identified significant limitations in the data gathered in the Balkans theater of operations. For example, mean time to repair data was not available. Further, the data gathered did not include sufficient detail to assess whether threat radar signals were stimulating the ASPJ to transmit during flights in the Balkans. (The jammer should only transmit a jamming signal when it detects an actual threat. Without confirmation that the jammer was being stimulated, whether it was working properly remains unknown.) Test officials emphasized that these are inherent limitations in trying to assess suitability based on the collection of operational field data with the resources that were available. In addition, though there were multiple built-in-test failure indications on the ground as well as in flight,5 aircrews continued their missions regardless of indications that ASPJ might not be working. In flight, if necessary, they made repeated attempts to clear built-in-test failure indications and achieve a “go” status from the ASPJ. According to Navy officials, this is standard operating procedure in the fleet for built-in-test systems. Built-in-test indications were counted as failures of ASPJ only if they were not cleared. Nevertheless, this is si,@ificant because all built-in-test failure indications were scored as mission critical failures during the 1992 operational test and this resulted in ASPJ’s inability to meet suitability thresholds. Such deficiencies cause questions to remain regarding ASPJ suitability for the F/A-18. 5The built-in-test subsystem is supposed to let users know whether or not ASPJ is working. Page 4 GAO/NSIAD-B746R Airborne Self-Protection Jammer B-275469 Consequently, although suitability data collected during the Balkan mission appeared to show some improvement in ASPJ compared with the 1992 operational test, OPTEVFOR has concluded that it could not use that data to reach a conclusion regarding improvement in the built-in-test function. Moreover, suitability measurement was statistically skewed in favor of ASPJ success because no missions were terminated after the built-in-test system indicated system failure. Hence, the total number of flight hours continued to accumulate, whether or not ASPJ was working. Navy officials maintain that because the faihrre indications were cleared in flight or could not be duplicated during post-flight maintenance, the results were not skewed. However, our review indicated that the built-in-test subsystem was not able to provide unequivocal information as to the status of ASPJ. This was a concern that contributed to ASPJ being rated unsuitable during the 1992 Operational Test and Evaluation and it continues to exist. In Bosnia, the mean time between operational mission failures was computed as 45.1 flight hours, exceeding the 1992 test requirement threshold of 33.3 hours. However, this was calculated not in accordance with the 1992 test methodology counting all built-in-test failure indications as failures, but in accordance with what the Navy maintains is a standard fleet procedure in which aircrews continue to try to recycle the built-in-test subsystem until the failure indication is cleared. If meantime between operational mission failure was computed in accordance with the 1992 test methodology, it would have been 31.1 hours, below the requirement threshold. CONGRESS DIRECTS STUDY OF ALTERNATIVE SYSTEMS According to the Navy, it had no readily available self-protection alternative to the ASPJ for one particular threat missile system in June 1995 when ASPJ was deployed for operations over the Balkans. Navy officials indicated that Congress supported the Navy’s plans to field ASPJ systems delivered prior to the 1992 ASPJ production termination. Looking to the future, however, the Navy’s long-term plan for fighter electronic self-protection is the joint Navy/Air Force IDECM system, which combines an electronic techniques generator to deceive radar with a towed decoy system. (A system that incorporates a towed decoy is envisioned to have greater capability against certain types of threats than an onboard jammer alone.) Navy officials maintain, however, that due to difficulty they may have retrofitting IDECM, they must investigate restarting ASPJ production for older aircraft. Congress recently provided $47.9 million to the Navy to procure 36 ASPJ systems for 3 deployed F/A-l8 C/D squadrons. While providing these funds for ASPJ, according to the Conference Report on the National Defense Authorization Act for Fiscal Year 1997, the conferees stated that their decision did not reflect a commitment for additional procurement of ASPJ systems or to restart production for U.S. government customers at this time. The congressional conferees further Page 5 GAO/NSIAD-97-46R Airborne SeEProtection Jammer B-275469 recognized that the Navy is expecting the IDECM to serve as the long-term electronic countermeasure system for the F/A-18 E/F and they want the Navy to explore long-term electronic countermeasure solutions for the F/A-18 C/D. In addition, the Senate Appropriations Committee has directed the Navy to provide a report that delineates the costs (including operations and support costs), systems availability, and operational advantages and disadvantages of adapting IDECM components for the F/A-18 C/D fighter compared to alternative electronic warfare systems. The Committee directed the Navy to submit this report to the congressional defense committees no later than February 15, 1997. As part of a separate, ongoing review, we are also looking at the IDECM and ALE-50 towed decoy systems and expect to report on this issue in early spring 1997. AGENCY COMMENTS In commenting on a draft of this letter, the Department of Defense (DOD) generally concurred with our findings. Writing for DOD, DOT&E acknowledged that the 1995 and 1996 testing and deployed ASPJ operations were not adequate to support a conclusion that ASPJ is effective and suitable relative to the requirements against which ASPJ was tested in 1992. However, DOD reiterated that (1) testing conducted to date has been adequate to find ASPJ effective against the selected threats of interest for the Balkans contingency operations, (2) the ASPJ effectiveness against selected Bosnian threats is important in these contingency operations, and (3) the suitability of ASPJ has been adequate for the requirements placed on it during contingency operations. However, DOD stated that many questions remain unanswered. DOD also provided a number of technical comments designed to enhance the clarity, accuracy, and completeness of the report. We have incorporated them in the report where appropriate. SCOPE AND METHODOLOGY We performed work at DOT&E in Washington, D.C., and OPTEVFOR in Norfolk, VA. To determine whether test results supported restarting ASPJ production, we compared 1995 and 1996 testing and test results and data from flight operations over the Balkans with data from ASPJ’s 1992 operational evaluation that led to ASPJ production termination. We discussed this data with representatives of DOT&E and OPTEVFOR and drew on their conclusions as well. In reviewing alternative systems, we performed work at the Naval Air Systems Command, McDonnell-Douglas Aircraft, and Lockheed/Martin contractor locations. We interviewed responsible agency officials and reviewed applicable documents. We conducted our review between August and December 1996 in accordance with bGenerally accepted government auditing standards. Page 6 GAOLWAD-97-46R Airborne Self-Protection Jammer B-275469 We are sending copies of this letter to interested congressional committees, the Secretary of Defense, the Secretary of the Navy, and the Director of the Office of Management and Budget. We will also provide copies to others upon request. If you or your staff have questions, please contact me at (202) 512-4841. Major contributors to this assignment were Paul Latta, Robert Coleman, Henry Arzadon, Terry Parker and Charles Ward. Sincerely yours, Louis J. Rodrigues Director, Defense Acquisition Issues (707187) Page 7 GAOINSIAD-97-46R Airborne Self-Protection Jammer . -,.:- Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 2534066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with “info” in the body to: email@example.com or visit GAO’s World Wide Web Eome Page at: http#www.gao.gov United States General Accounting Office Washington, D.C. 20548-0001 Official Business Penalty for Private Use $300 . Address Correction Requested
Airborne Self-Protection Jammer
Published by the Government Accountability Office on 1997-01-29.
Below is a raw (and likely hideous) rendition of the original report. (PDF)