oversight

Tactical Aircraft: DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue

Published by the Government Accountability Office on 2003-03-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to Congressional Committees




March 2003
             TACTICAL AIRCRAFT

             DOD Should
             Reconsider Decision
             to Increase F/A-22
             Production Rates
             While Development
             Risks Continue




GAO-03-431
                                               March 2003


                                               TACTICAL AIRCRAFT

                                               DOD Should Reconsider Decision
Highlights of GAO-03-431, a report to          to Increase F/A-22 Production Rates
Congressional Committees
                                               While Development Risks Continue



The Air Force is developing the                The F/A-22 development program did not meet key performance, schedule,
F/A-22 aircraft to fly at higher               and cost goals in fiscal year 2002, and delays in the flight test program have
speeds for longer distances, be less           led to an increase in the development cost estimate of $876 million. In
detectable, and improve the pilot’s            response to this increase, DOD restructured the development program and
awareness of the surrounding                   reduced production aircraft by 27. If additional delays occur, further changes
situation. The F/A-22 will replace
the Air Force’s existing fleet of
                                               may be required. The program also continues to address technical problems
F-15 aircraft. Over the past several           that have limited the performance of test aircraft, including violent
years the program has experienced              movement or “buffeting” of the vertical fins, overheating in portions of the
significant cost overruns and                  aircraft, weakening of materials in the horizontal tail, and instability of
schedule delays. Congress                      avionics software. Air Force officials cannot predict when they will resolve
mandated that GAO assess the                   these problems. These technical problems, along with the late delivery of
development program and                        aircraft to the flight test center, have delayed the development program.
determine whether the Air Force                Based on F/A-22 flight test accomplishment data and current flight test
is meeting key performance,                    plans, we believe that operational testing will likely be delayed several
schedule, and cost goals. GAO also             months beyond the planned August 2003 start date.
assessed the implications of the
progress of the development
program on production.
                                               The F/A-22 program is in its final stages of development, and low-rate initial
                                               production has begun. Since fiscal year 1997, funds have been appropriated
                                               to acquire production aircraft, and the F/A-22 acquisition plan calls for
                                               steadily increasing annual production rates. However, GAO considers the
To help minimize the risks of                  Air Force’s acquisition strategy at high risk for increases in production costs.
producing large quantities of                  In past reports, GAO has reported that acquiring aircraft while significant
aircraft that may require costly               technical challenges remain does not allow for adequate testing of the
modifications, GAO recommends                  aircraft. The uncertainties regarding performance capabilities of the F/A-22
that the Secretary of Defense                  aircraft and its development schedule will persist until technical problems
(1) reconsider the decision to                 have been addressed, including testing of modifications or fixes necessary to
increase the annual production rate            potentially alleviate these problems. In light of those uncertainties, steadily
beyond 16 aircraft until greater
                                               increasing annual production rates could result in the Air Force having to
knowledge on any need for
modifications is established                   modify a larger quantity of aircraft after they are built.
through completion of operational
testing and (2) update the 2002
risk assessment and certification
with sufficient detail to allow
verification of the conclusions.
In comments on a draft of this
report, the Department of Defense
(DOD) stated that it agreed, for the
most part, with our description of
the current state of the F/A-22
program’s content, schedule and
cost. However, DOD did not concur
with our recommendation.
www.gao.gov/cgi-bin/getrpt?GAO-03-431.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Allen Li at
(202) 512-4841 or lia@gao.gov.
Contents


Letter                                                                                           1
                       Results in Brief                                                         1
                       Background                                                               3
                       F/A-22 Technical Problems Continue to Affect Performance                 4
                       Flight Test Schedules Have Been Extended and May Slip Further            8
                       Development and Modernization Costs Have Increased                      10
                       Risks in the F/A-22 Acquisition Plan                                    12
                       Conclusions                                                             14
                       Recommendations for Executive Action                                    14
                       Agency Comments and Our Evaluation                                      15
                       Scope and Methodology                                                   17

Appendix I             Estimates of Performance for Key Parameters                             20



Appendix II            Comments from the Department of Defense                                 21



Appendix III           GAO Staff Acknowledgments                                               25



Related GAO Products                                                                           26



Table
                       Table 1: Schedule Changes for Key F/A-22 Test Program Events              9


Figures
                       Figure 1: F/A-22 Flight Test Schedule Changes                             9
                       Figure 2: Planned Modernization Funding Increases, President’s
                                Budgets for Fiscal Years 2001-2004                             11
                       Figure 3: Number of Production Aircraft on Contract Prior to
                                Completion of Operational Testing                              13




                       Page i                                           GAO-03-431 F/A-22 Aircraft
Abbreviations

DIOT&E            Dedicated Initial Operational Test and Evaluation
DOD               Department of Defense
OSD               Office of the Secretary of Defense



This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                                       GAO-03-431 F/A-22 Aircraft
United States General Accounting Office
Washington, DC 20548




                                   March 14, 2003

                                   Congressional Committees

                                   The Air Force is developing the F/A-221 aircraft to replace its fleet of F-15
                                   air superiority aircraft. The F/A-22 is designed to be superior to the
                                   F-15 because it is capable of flying at higher speeds for longer distances,
                                   more difficult to detect, and able to provide the pilot with substantially
                                   improved awareness of the surrounding situation. The Air Force began the
                                   F/A-22 development program in 1991. During the past several years, the
                                   program has experienced repeated and significant cost overruns and
                                   schedule delays. Congressional concern about the aircraft’s development
                                   program cost and progress is long-standing, and it continues.

                                   The National Defense Authorization Act for Fiscal Year 19982 requires
                                   us to assess the Air Force’s F/A-22 development program annually and
                                   determine whether the Air Force is meeting key performance, schedule,
                                   and cost goals.3 This is our fifth report. Specifically, we determined (1) the
                                   progress of F/A-22 development in terms of performance, schedule, and
                                   cost and (2) implications of this progress on the Department of Defense’s
                                   (DOD) acquisition plans.


Results in Brief                   The F/A-22 development program did not meet its key performance,
                                   schedule, and cost goals for fiscal year 2002. The program continues to
                                   address technical problems that have limited the performance of test
                                   aircraft. These problems include unexpected shutdowns of the aviation
                                   electronics (avionics) and excessive movement of the vertical tails. Air
                                   Force officials stated they do not yet understand the problems associated
                                   with the avionics instability well enough to predict when they would be
                                   able to resolve them. Aircraft also have been unable to meet maintenance
                                   requirements and are spending more time than planned on the ground
                                   undergoing maintenance.


                                   1
                                    “F/A” stands for fighter/attack aircraft. The Air Force changed the designation from F-22
                                   to F/A-22 in September 2002 to reflect the aircraft’s air-to-surface attack capability.
                                   2
                                       P.L. 105-85, Section 217, Nov. 18, 1997.
                                   3
                                    Section 217 of the act also requires us to assess whether we had access to sufficient
                                   information to make informed judgments on matters covered by our report.



                                   Page 1                                                         GAO-03-431 F/A-22 Aircraft
In addition, the program has experienced schedule delays. These delays
are the result of technical problems and the late delivery of developmental
aircraft to the flight test center. Many tasks originally scheduled for 2002
have been rescheduled for 2003, and the Air Force now plans to conduct
more developmental flight testing concurrent with operational testing.4
Moreover, we believe it is unlikely the Air Force will complete all
necessary flight testing prior to the planned start of operational testing.
Therefore, the start of operational testing may need to be delayed
several months beyond the planned August 2003 start date.

Delays in the flight test program have significant consequences. Most
recently, they have led to an increase of $876 million in the development
cost estimate. In December 2002, in response to this increase, DOD
restructured the program using funds from production and modernization
upgrades to cover the cost increases. As a result, DOD reduced the
number of production aircraft by 27, which decreased the total number of
aircraft to be acquired from 303 to 276.5 If additional delays occur, further
changes may be required.

Despite continuing development problems and challenges, the Air Force
plans to continue to acquire aircraft during low-rate production at
increasing yearly rates. For example, the Air Force plans to acquire
20 aircraft in 2003, rather than the maximum of 16 Congress allowed
without DOD submittal of a risk assessment and certification. However, as
we have previously reported, acquiring aircraft before adequate testing is a
high-risk strategy that could serve to further increase production costs.
The performance capabilities of the F/A-22 and the aircraft’s development
schedule will remain uncertain until technical problems have been
addressed, including testing of modifications or fixes necessary to
potentially alleviate these problems.

We are providing recommendations aimed at reducing the risk of
increasing the production rate of F/A-22 aircraft before technical
challenges have been addressed through operational testing. In its



4
 Short of war, operational testing is the most realistic way of assessing weapon system
performance. It puts a weapon through the rigors of combat conditions to determine its
operational effectiveness and suitability.
5
 The 27 aircraft are a reduction from the approved program quantity of 303 aircraft,
however, the Air Force had hoped to acquire as many as 339 aircraft by achieving
cost reductions.




Page 2                                                        GAO-03-431 F/A-22 Aircraft
             comments on a draft of this report, DOD indicated it did not concur with
             our recommendation that it not exceed an annual production rate of
             16 aircraft until operational testing is complete. DOD stated that the
             acquisition of more than 16 aircraft in fiscal year 2003 involves lower risk
             and lower total program cost than staying at 16.


             The F/A-22 is to be an air superiority and ground attack aircraft with
Background   advanced features to make it less detectable to adversaries (stealth
             characteristics) and capable of high speeds for long ranges.6 It is designed
             to have integrated avionics that greatly improve pilots’ awareness of the
             situation surrounding them. The objectives of the F/A-22 development
             program are to (1) design, fabricate, test, and deliver nine F/A-22
             development test aircraft, two non-flying structural test aircraft, six
             production representative test aircraft, and 37 flight-qualified engines;
             (2) design, fabricate, integrate, and test the avionics; and (3) design,
             develop, and test the support and training systems. The F/A-22 is being
             developed under contracts with Lockheed Martin Corporation, the prime
             contractor (for the aircraft), and Pratt & Whitney Corporation (for
             the engine).

             Following a history of increasing cost estimates to complete the
             development phase of the F/A-22 program, the National Defense
             Authorization Act for Fiscal Year 1998 established a cost limitation for
             both the development and production.7 Subsequently, the National
             Defense Authorization Act of 2002 eliminated the cost limitation for the
             development, but left the cost limit for production cost in place.8 The
             production program is now limited to $36.8 billion.9 The current cost
             estimate of the development program is $21.9 billion.

             Currently, the F/A-22 program is in both development and production.
             Development is in its final stages, and low rate initial production has
             begun. Since fiscal year 1997, funds have been appropriated to acquire




             6
               Air superiority is the degree of air dominance that allows the conduct of operations by
             land, sea, and air forces without prohibitive interference by the enemy.
             7
                 P.L. 105-85, Section 217, Nov. 18, 1997.
             8
                 P.L. 107-107, Section 213, Dec. 28, 2001.
             9
                 The cost limitation, before adjustment under the act’s provisions, was $43.4 billon.




             Page 3                                                            GAO-03-431 F/A-22 Aircraft
                       production aircraft, and the F/A-22 acquisition plan calls for steadily
                       increasing annual production rates.

                       The aircraft’s development problems and schedule delays have caused
                       congressional concerns, particularly in light of DOD’s planned increase in
                       production rates. The National Defense Appropriations Act for Fiscal Year
                       200310 prohibited the obligation of funds for the acquisition of more than
                       16 production aircraft in fiscal year 2003, until the Under Secretary of
                       Defense for Acquisition, Technology, and Logistics submits the following
                       to the congressional defense committees: (1) a formal risk assessment that
                       identifies and characterizes the potential cost, technical, schedule, or
                       other significant risks resulting from increasing the F/A-22 production
                       quantities prior to the Dedicated Initial Operational Test and Evaluation
                       (DIOT&E)11 of the aircraft and (2) either a certification that increasing the
                       F/A-22 production quantity for fiscal year 2003 beyond 16 aircraft involves
                       lower risk and lower total program cost than staying at that quantity or
                       implementing a revised production plan, funding, and test schedule. In
                       December 2002, DOD submitted the risk assessment and certification
                       to Congress.


                       The F/A-22 developmental program did not meet key performance goals
F/A-22 Technical       established for fiscal year 2002 and continues to confront numerous
Problems Continue to   technical challenges. Major technical problems include instability of the
                       avionics software, violent movement, or “buffeting,” of vertical fins,
Affect Performance     overheating in portions of the aircraft, weakening of materials in the
                       horizontal tail, and the inability to meet airlift support and maintenance
                       requirements. Modifications are being made to some test aircraft to
                       address some of these problems in preparation for operational testing.
                       Nevertheless, these problems continue to restrict the performance and
                       testing of the F/A-22.


Avionics Instability   Software instability has hampered efforts to integrate advanced avionics
                       capabilities into the F/A-22 system. Avionics control and integrated
                       airborne electronics and sensors provide an increased awareness of the
                       situation around the pilot. The Air Force told us that the avionics have
                       failed or shut down during numerous tests of F/A-22 aircraft due to


                       10
                            P.L. 107-248, Section 8119, Oct. 23, 2002.
                       11
                            Short of war, operational testing is the most realistic way of assessing a weapon system.




                       Page 4                                                            GAO-03-431 F/A-22 Aircraft
                         software problems. The shutdowns occur when the pilot attempts to use
                         the radar, communication, navigation, identification, and electronic
                         warfare systems concurrently. Although the plane can still be flown after
                         the avionics have failed, the pilot is unable to successfully demonstrate the
                         performance of the avionics. Therefore, the Air Force has had to extend
                         the test program schedule.

                         The Air Force recognized that the avionics problems pose a high technical
                         risk to the F/A-22 program, and in June 2002 the Air Force convened a
                         special team to address the problem. According to the team, the
                         unpredictable nature of the shutdowns was not surprising considering the
                         complexity of the avionics system. The team recommended that the
                         software be stabilized in the laboratory before releasing it to flight testing.
                         The team further recommended conducting a stress test on the software
                         system architecture to reduce problems and ensure that it is operating
                         properly. The Air Force implemented these recommendations. Further, the
                         Air Force extended the avionics schedule to accommodate avionics
                         stability testing and now plans to complete avionics testing in the first
                         quarter of 2005. However, Air Force officials stated that they do not yet
                         understand the problems associated with the instability of the avionics
                         software well enough to predict when they will be able to resolve
                         this problem.


Vertical Fin Buffeting   Under some circumstances, the F/A-22 experiences violent movement, or
                         buffeting, of the vertical fins in the tail section of the aircraft. This occurs
                         as air, moving first over the body and the wings of the aircraft, places
                         unequal pressures on the vertical fins and rudders. Unless the violent
                         movement is resolved or the fins strengthened, the vertical fins will break
                         over time because the pressures experienced exceed the strength limits of
                         the fins. In addition, the buffeting problem has restricted the testing of
                         aerial maneuvers of the aircraft.

                         Lockheed Martin has developed several modifications to strengthen the
                         vertical fins and has performed an analysis to test the structural strength
                         of the aircraft. It concluded that no flight restrictions above 10,000 feet are
                         necessary as a result of buffeting. Currently, the Air Force has not begun
                         testing to verify flight operations at or below 10,000 feet; operational
                         limitations at altitudes below 10,000 feet remain in effect, with testing
                         scheduled to begin in June 2003.




                         Page 5                                                 GAO-03-431 F/A-22 Aircraft
Overheating Concerns       Overheating in the rear portions of the aircraft has significantly restricted
                           the duration of high-speed flight testing. As the F/A-22 flies, heat builds up
                           inside several areas in of the rear of the aircraft. Continued exposure to
                           high temperatures would weaken these parts of the aircraft. For example,
                           a portion of the airframe that sits between the engines’ exhausts12
                           experiences the highest temperatures. This intense heat could weaken or
                           damage the airframe. To prevent this heat buildup during flight testing, the
                           aircraft is restricted to flying just over 500 miles per hour, about the same
                           speed as a modern jet liner, and significantly below the supercruise13
                           requirement. Currently, the F/A-22 flies with temperature sensors in
                           those areas of the aircraft, and it slows down whenever the temperature
                           approaches a certain level. The Air Force may add copper sheets to the
                           rear of the aircraft to alleviate the problem. The Air Force began these
                           modifications in January 2003 and plans to complete them by July 2003.


Horizontal Tail Material   F/A-22 aircraft have experienced separations of materials in the horizontal
Separations                tail and the shaft, which allows the tail to pivot. Because the separations
                           reduce tail strength, the Air Force restricted flight testing of some aircraft
                           until it determined that this problem would not affect flight safety
                           during testing. The Air Force and the contractor initially believed that
                           improvements to the aircraft’s manufacturing process would solve this
                           problem. However, the Air Force has determined that it could only solve
                           this problem by redesigning the tail of the aircraft. The Air Force plans to
                           conduct flight testing of the redesigned tail between February 2004 and
                           April 2004.


Meeting Airlift Support    The Air Force estimates it will not meet the F/A-22 airlift support
Requirements               requirement despite last year’s estimate that it would meet all identified
                           key performance parameters.14 (Appendix I contains a list of key
                           performance parameters.) The airlift support requirement is that 8 C-141
                           aircraft or their equivalents would be sufficient to deploy a squadron of



                           12
                                The technical term for this section of the airframe is called the “stinger.”
                           13
                              Supercruise is the aircraft’s ability to travel at high speeds for long ranges. The F/A-22’s
                           supercruise requirement is approximately 1,000 miles per hour.
                           14
                             U.S. General Accounting Office, Tactical Aircraft: F-22 Delays Indicate Initial
                           Production Rates Should Be Lower to Reduce Risks, GAO-02-298 (Washington, D.C.:
                           Mar. 5, 2002).




                           Page 6                                                              GAO-03-431 F/A-22 Aircraft
                           24 F/A-22s for thirty days without resupply. Today the Air Force estimates
                           that 8.8 C-141 equivalents will be necessary.


Impact of Maintenance      The F/A-22’s performance may also be affected by maintenance needs
Needs on Performance       that exceed established objectives. The Air Force estimates that the
                           F/A-22 should, at this point in its development, be able to complete
                           1.67 flying hours between maintenance actions and 1.95 flying hours by
                           the end of development. However, aircraft are requiring five times the
                           maintenance actions expected at this point in development. As of
                           November 2002, the development test aircraft have been completing
                           only .29 flying hours between maintenance actions. Therefore, the
                           development test aircraft are spending more time than planned on the
                           ground undergoing maintenance.

                           In addition, the F/A-22 program has not completed the testing required to
                           prove the aircraft can be maintained worldwide without unique support
                           equipment. For example, the Air Force planned to fly the F/A-22 a
                           minimum of 650 hours prior to the start of operational testing to establish
                           that special support equipment is not necessary to maintain the materials
                           on the exterior of the aircraft. These materials are critical to the aircraft’s
                           low observable, or stealthy, nature. However, as of December 2002, the
                           program has only accomplished 191.6 hours. According to the Air Force,
                           the program will not complete testing for this requirement until the
                           completion of the development program, currently planned for July 2004.


Modifications to Improve   In 2002, the F/A-22 development program implemented several
Performance                modifications to development aircraft to improve performance. The
                           majority of modifications were related to installing the necessary upgrades
                           to complete operational testing. The last three development test aircraft
                           have required an average of 63 modifications. The first two production
                           aircraft have required an average of 50 of these upgrades.

                           In addition, the program repaired problems in the aircraft’s arresting gear
                           system that were discovered during development testing. Further, the Air
                           Force has scheduled modifications to address the previously cited
                           problems found with the vertical tail of the aircraft (fin-buffeting). The Air
                           Force included these repairs in its 2002 modification schedule, but did not
                           begin them in 2002. The modifications will begin during fiscal year 2003.




                           Page 7                                                GAO-03-431 F/A-22 Aircraft
                        Progress in F/A-22 flight testing was slower than expected in 2002 in all
Flight Test Schedules   test areas, according to Office of the Secretary of Defense (OSD) testing
Have Been Extended      officials. Consequently, the Air Force extended flight test schedules and
                        reduced the number of flight tests. Many tasks originally planned for 2002
and May Slip Further    were rescheduled for 2003. Further, the Air Force now plans to conduct
                        more developmental flight testing concurrently with operational testing.

                        Continuing technical problems were the primary reasons for the delays
                        in flight testing. In addition, late delivery of development aircraft to the
                        flight test center was a contributing problem; three developmental aircraft
                        were delivered from 9 to 12 months late. Late deliveries were due not
                        only to technical problems, but also to continuing problems associated
                        with the manufacture and assembly of development aircraft by the
                        prime contractor.

                        With the new schedule, the Air Force delayed the beginning of operational
                        testing for 4 months, until the portion of developmental testing required to
                        begin operational testing could be completed. Operational testing is now
                        planned to begin in August 2003. Figure 1 and table 1 show the changes in
                        the FA/-22 flight test schedules.




                        Page 8                                              GAO-03-431 F/A-22 Aircraft
Figure 1: F/A-22 Flight Test Schedule Changes




Table 1: Schedule Changes for Key F/A-22 Test Program Events

Key Events                                              Prior schedule   Revised schedule      Change in months
Completion of development flight testing
necessary prior to operational testing                      April 2003        August 2003                       4
Start of operational testing                                April 2003        August 2003                       4
Completion of operational testing                       December 2003           July 2004                       7
High-rate production decision                              March 2004         March 2004                        0
Source: U.S. Air Force.




                                           Page 9                                      GAO-03-431 F/A-22 Aircraft
                      However, according to OSD officials involved in operational testing, there
                      is a high risk of not completing an adequate amount of development flight
                      testing before operational testing is scheduled to begin. Indeed, we believe
                      that it is unlikely that the Air Force will be able to complete all necessary
                      avionics flight testing prior to the planned start of operational testing.
                      Based on F/A-22 flight test accomplishment data and current flight test
                      plans, we project that the start of operational testing might be delayed
                      until January 2004. As a result, operational testing could be delayed by
                      several months beyond the current planned date of August 2003.


                      In December 2002, the Air Force estimated that development costs had
Development and       increased by $876 million, bringing total development costs to
Modernization Costs   $21.9 billion. This increase was due to the technical problems and
                      schedule delays discussed earlier.
Have Increased
                      In addition, since fiscal year 2001, there have been dramatic increases
                      in planned funding for modernization upgrades15 that enhance the
                      operational capabilities of the F/A-22, as shown in figure 2. Currently, the
                      Air Force has almost $3.0 billion in funding for modernization projects,
                      which it plans to spend through fiscal year 2009. Most of the recent
                      increase in modernization funding is necessary to provide increased
                      ground attack capability. Other modernization projects include upgrading
                      avionics software, adding an improved short-range missile capability,
                      upgrading instrumentation for testing, and incorporating a
                      classified project.




                      15
                        The Air Force considers modernizations outside the scope of the development and
                      production programs.




                      Page 10                                                    GAO-03-431 F/A-22 Aircraft
Figure 2: Planned Modernization Funding Increases, President’s Budgets for
Fiscal Years 2001-2004




In December 2002, in response to the increase in development costs, the
Under Secretary of Defense, Comptroller, approved the restructuring of
the F/A-22 program. According to the Comptroller, the cost increase will
not require increased funds from Congress. Rather, the estimated
$876 million increase for development will be met by a $763 million
decrease in production funding and a transfer of $113 million from
modernization funds. This restructure eliminates 27 aircraft from the
current production program, reducing the total number of aircraft to be
acquired from 303 to 276.16




16
  The 27 aircraft are a reduction from the approved program quantity of 303 aircraft;
however, the Air Force had hoped to acquire as many as 339 aircraft by achieving
cost reductions.




Page 11                                                       GAO-03-431 F/A-22 Aircraft
                      Despite continuing development problems and challenges, the Air Force
Risks in the F/A-22   plans to continue acquiring production aircraft at increasing annual rates.
Acquisition Plan      This is a very risky strategy, because, as we have previously reported,
                      the Air Force may encounter higher production costs as a result of
                      acquiring significant quantities of aircraft before adequate testing. Late
                      testing could identify problems that require costly modifications in order
                      to achieve satisfactory performance.

                      For example, as shown in figure 3, the Air Force plans to acquire
                      20 aircraft during 2003, rather than the maximum of 16 Congress allowed
                      without DOD’s submittal of a risk assessment and certification. DOD
                      justified this strategy in the December 2002 risk assessment and
                      certification it submitted to Congress.17 In this document, DOD certified
                      that acquiring more than 16 aircraft involved lower risk and lower total
                      program cost than acquiring only 16. DOD identified the costs associated
                      with acquiring more than 16 aircraft per year as between $7 million and
                      $221 million, depending on the number of aircraft in excess of 16. DOD
                      concluded that this additional cost would be less than the potential cost of
                      modifying production aircraft once operational testing has been
                      completed. Figure 3 shows the Air Force’s acquisition plan.




                      17
                        The National Defense Appropriations Act for Fiscal Year 2003 (P.L. 107-248, Oct. 23,
                      2002) required DOD to submit a certification to Congress to justify the acquisition of
                      more than 16 aircraft in fiscal year 2003. In December 2002, DOD submitted the
                      certification to Congress.




                      Page 12                                                       GAO-03-431 F/A-22 Aircraft
Figure 3: Number of Production Aircraft on Contract Prior to Completion of Operational Testing




                                         a
                                         Operational testing is scheduled to be completed in July 2004.


                                         However, DOD’s risk assessment may be overly optimistic because it
                                         is grounded in the conclusion that there is a low risk that remaining
                                         development and operational testing will identify needs for expensive
                                         modifications. The performance capabilities of the F/A-22 and its schedule
                                         will remain uncertain until technical problems have been addressed,
                                         including testing of modifications or fixes necessary to potentially
                                         alleviate these problems. Furthermore, we believe that the amount of
                                         development and operational testing and the remaining uncertainties
                                         increase the possibility that modifications considered unlikely in DOD’s
                                         analysis will, indeed, need to be made. For example, the Air Force has still
                                         not completely defined the fin-buffet problem described earlier in this
                                         report. The remaining 15 percent of flight testing to help characterize the
                                         problem is not scheduled to begin until June 2003. Consequently, there is
                                         still the possibility that additional modifications and costs may be
                                         necessary to correct this problem on production aircraft. DOD’s risk
                                         assessment acknowledges that additional fin buffet testing is needed, but
                                         concludes that modifications are not expected.

                                         The optimism of DOD’s risk assessment is reflected in the Air Force’s
                                         general acquisition strategy. As also shown by figure 3, the Air Force
                                         is currently committed to acquiring 73 production aircraft (26 percent)
                                         before operational and development testing is complete. We believe that—
                                         like the fiscal year 2003 decision to acquire more than 16 aircraft—this is
                                         an overly optimistic strategy given the remaining F/A-22 technical
                                         problems and the current status of testing. As we have noted, acquiring
                                         aircraft before completing adequate testing to resolve significant technical


                                         Page 13                                                          GAO-03-431 F/A-22 Aircraft
                      problems increases the risk of costly modifications later. If F/A-22 testing
                      schedules slip further—as we believe is likely—even more aircraft will be
                      acquired before development and operational testing is complete, and the
                      risk of costly modifications will increase still more.


                      Continuing the acquisition of aircraft in increasing quantities when
Conclusions           significant development testing and technical problems remain is an
                      acquisition strategy that relies on overly optimistic assumptions regarding
                      the outcome and timing of the remaining testing events. By employing
                      such a strategy, major problems are more likely to be discovered after
                      production has begun when it is either too late or very costly to correct
                      them. At the very least, key decisions are being made without adequate
                      information about the weapon system’s demonstrated operational test
                      results. In its certification, DOD quantified the estimated costs associated
                      with a higher production rate. However, the potential advantage was
                      predicated on the assumption that the risks of modifications are low.
                      As we stated last year, by limiting F/A-22 production quantities and
                      completing development testing, the Air Force could gain information that
                      would reduce uncertainties and the risks of increased costs and delays
                      before committing to additional production aircraft. As we discussed
                      earlier in this report, DOD recently decided to reduce production
                      quantities as part of a program restructure to address F/A-22 development
                      problems and associated cost increases. Based on uncertainties about the
                      resolution of problems found in the past year, we continue to maintain the
                      position that production quantities should be limited.


                      In light of continued uncertainties regarding the resolution of problems
Recommendations for   found in the past year and notwithstanding the December 2, 2002
Executive Action      certification provided by DOD, we recommend that the Secretary of
                      Defense

                  •   reconsider the Department’s decision to increase the annual
                      production rate beyond 16 aircraft until greater knowledge on any
                      need for modifications is established through completion of
                      operational testing, and
                  •   update the 2002 risk assessment and certification with sufficient detail to
                      allow for verification of the conclusions following the completion of
                      operational testing.




                      Page 14                                             GAO-03-431 F/A-22 Aircraft
                     In written comments on a draft of this report, DOD stated that it agreed,
Agency Comments      for the most part, with our description of the current state of the F/A-22
and Our Evaluation   program’s content, schedule, and cost. However, DOD did not concur with
                     our recommendation that it not increase its production rate beyond the
                     maximum of 16 aircraft Congress allowed without DOD submitting a risk
                     assessment and certification. DOD said that our recommendation does not
                     sufficiently account for the costs of termination associated with the
                     approval given to funding long-lead items, the manufacturing inefficiencies
                     associated with a reduction in aircraft quantities, or the effects of inflation
                     on the cost of acquiring aircraft at a lower rate. DOD also noted that we
                     had not provided a quantitative assessment to justify limiting production,
                     and it reiterated its reliance on the risk assessment and certification it
                     submitted to Congress in December 2002. DOD also asserted, incorrectly,
                     that our report concludes that minimal cost risk would be realized by
                     slowing production.

                     Following review of DOD’s comments, we clarified the recommendation
                     in our draft report by establishing two recommendations. These
                     recommendations are based on the current state of the program—
                     including the challenges and risks it faces—and on our examination of
                     DOD’s risk assessment and certification. DOD acknowledges the
                     challenges faced by the program but believes the risk of modification is
                     low. As we discussed in this report, until testing has been completed and
                     technical problems have been addressed, the performance capabilities
                     of the F/A-22 and its schedule will remain uncertain; thus, it is not possible
                     to predict that expensive modifications will not be required. For example,
                     as we stated earlier in this report, DOD’s risk assessment concludes that
                     significant costs associated with a more extensive modification to resolve
                     the fin buffet problem may be required, but the probability is low. DOD
                     arrives at this conclusion even though the last phase of testing to help
                     characterize the fin buffet problem has not yet begun. Furthermore, we
                     continue to believe there is still significant risk that the F/A-22 program
                     will not be able to begin operational testing as scheduled in August 2003.
                     Subsequent to our providing the draft of this report to DOD for comment,
                     OSD’s operational test and evaluation office issued a report stating that
                     F/A-22 technical and schedule risk are still high, as is the risk that
                     operational testing will be further delayed.

                     While DOD’s December 2002 risk assessment and certification did provide
                     an indication that manufacturing inefficiencies and inflation as a result of
                     lower production rates would increase costs, sufficient detail was not
                     provided in its risk assessment for us to verify DOD’s conclusion. We
                     requested additional detailed information to help us evaluate and verify


                     Page 15                                               GAO-03-431 F/A-22 Aircraft
the conclusions. However, the information provided to us was not
adequate to verify the conclusions contained in the risk assessment.
Regardless, even with such verification, still needing to be resolved are the
uncertainties to date regarding when development problems can be fixed
and the possibility of finding additional problems prior to the completion
of operational testing. As a result, we have little confidence that existing
problems can be quickly resolved and will not result in further delays. Our
work has shown that continuing the acquisition of aircraft in increasing
quantities when significant development testing and technical problems
remain is risky. By employing such a strategy, major problems are more
likely to be discovered after the program has begun production when it is
either too late or very costly to correct them.

DOD also provided various technical comments, which we have
incorporated as appropriate. One of these comments related to the total
number of production aircraft to be acquired. The projected number of
production aircraft the Air Force plans to or can actually acquire has
historically been fluid and elusive. For example, the President’s budget
for fiscal year 2003 reflected plans to acquire 333 production aircraft,
even though the approved program at the time called for acquiring
295 production aircraft. In its technical comments, DOD stated that the
approved program plan is to acquire 295 aircraft. As a result of the recent
F/A-22 restructuring to cover development cost increases, the Air Force
says that it now plans to acquire 276 aircraft. However, DOD estimates
that the cost of production to acquire these 276 aircraft will be
$42.2 billion, which exceeds the current production cost limit by
$5.4 billion.18 Consequently, unless the production cost limit is raised or
substantial cost reduction plans are achieved, it appears that the number
of aircraft that can actually be purchased will have to be lowered from
the 276 planned. This is particularly true if production or development
costs—or both—continue to rise and no additional funds are provided by
the Congress.

Last month, we recommended in another report that DOD provide
Congress with documentation reflecting the quantity of aircraft that DOD
believes can be procured within the existing production cost limit.19 DOD’s


18
     The current production cap, as adjusted, is $36.8 billion.
19
  U.S. General Accounting Office, Tactical Aircraft: DOD Needs to Better Inform Congress
about Implications of Continuing F/A-22 Cost Growth, GAO-03-280 (Washington, D.C.:
Feb. 28, 2003).




Page 16                                                           GAO-03-431 F/A-22 Aircraft
              explanation in its technical comments to a draft of this report identifies
              the likelihood that F/A-22 aircraft quantities will continue to fluctuate. This
              makes our recent recommendation that much more compelling.


              To determine whether the development program is likely to meet
Scope and     performance goals, we analyzed information on the status of key
Methodology   performance parameters. We compared performance goals established by
              the Under Secretary of Defense for Acquisition, Technology, and Logistics
              with the Air Force’s estimates of performance for completion of
              development made in December 2002.

              To identify the status of F/A-22 modifications, we collected updated
              information on the status of existing aircraft structural problems that
              have required aircraft modifications. To determine whether the program
              is expected to meet schedule goals, we reviewed program and avionics
              schedules and discussed potential changes to these schedules with F/A-22
              program officials. We tracked progress in the flight test program and
              evaluated schedule variances in the contractors’ performance
              management system and compared planned milestone accomplishment
              dates with actual dates. We tracked technical problems in manufacturing
              and assembling the development test aircraft.

              To determine whether the program is likely to meet the cost goal, we
              examined (1) the extent to which the development program is likely to be
              completed within the current cost estimate, (2) the Air Force’s plans to
              fund the program for fiscal year 2003, and (3) the program’s funding plan
              compared to the current cost estimate.

              In examining DOD’s risk assessment, we discussed the various DOD
              assumptions and approaches used in the assessment with a program
              official who conducted the assessment. We then analyzed the various DOD
              assumptions and approaches used to make the assessment conclusions.

              In making these determinations, assessments, and identifications, we
              required access to current information about test results, performance
              estimates, schedule achievements and revisions, costs being incurred,
              aircraft modifications, and the program’s plans for continued development
              and initial production. The Air Force and contractors gave us access to
              sufficient information to make informed judgments on the matters covered
              in this report.




              Page 17                                              GAO-03-431 F/A-22 Aircraft
In performing our work, we obtained information or interviewed officials
from the Office of the Secretary of Defense, Washington, D.C., and the
F/A-22 System Program Office, Wright-Patterson Air Force Base, Ohio. We
performed our work from September 2002 through December 2002 in
accordance with generally accepted government auditing standards.


We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretary of the Air Force; and
the Director, Office of Management and Budget. We will also make copies
available to others upon request. In addition, the report will be available at
no charge on the GAO Web site at http://www.gao.gov. Please contact me
at (202) 512-4841 or Catherine Baltzell at (202) 512-8001 if you or your staff
have any questions concerning this report. Major contributors to this
report are listed in appendix III.




Allen Li
Director
Acquisition and Sourcing Management




Page 18                                              GAO-03-431 F/A-22 Aircraft
List of Congressional Committees

The Honorable John Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate

The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate

The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives

The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives




Page 19                            GAO-03-431 F/A-22 Aircraft
                                             Appendix I: Estimates of Performance for Key
Appendix I: Estimates of Performance for     Parameters



Key Parameters


 Key performance parameter                                   Requirement                   Current Estimate                Margin
 Radar cross section                                         Classified                    Favorable                       Favorable
 Supercruise                                                 1.5 Mach                      1.68 Mach                       12% favorable
 Acceleration (<100% is favorable)a                          54 seconds                    52.3 seconds                    3% favorable
 Maneuverability                                             3.7 g                         3.7 g                           0
 Payload (missiles)                                          Four medium-range,            Six medium range,               NA
                                                             two short-range               two short-range
 Combat radiusb sub + super                                  260 + 100                     315 + 100                       15% favorable
 Radar detection range                                       Classified                    105%                            5% favorable
 Independent airlift support (C-141 equivalents)             8                             8.8                             (0.8) unfavorable
 Sortie generation rate                                      Classified                    100%                            0
 Average flight test hours between maintenance               3.0                           3.0                             0
 Interoperability                                            100% of IERsc                 100% of IERsc                   0
Source: U.S. Air Force.
                                             a
                                              The acceleration parameter is a measure of the time it takes the aircraft to increase speed to a
                                             certain level. If the aircraft is able to increase speed to a certain level in less time than expected,
                                             this is considered favorable. Therefore, a measure of less than 100 percent is favorable.
                                             b
                                                 Subsonic is below speed of sound, and supersonic is above speed of sound.
                                             c
                                                 IERs are information exchange requirements.




                                             Page 20                                                                   GAO-03-431 F/A-22 Aircraft
             Appendix II: Comments from the Department of Defense
Appendix II: Comments from the Department
of Defense




             Page 21                                                GAO-03-431 F/A-22 Aircraft
Appendix II: Comments from the Department of Defense




Page 22                                                GAO-03-431 F/A-22 Aircraft
Appendix II: Comments from the Department of Defense




Page 23                                                GAO-03-431 F/A-22 Aircraft
Appendix II: Comments from the Department of Defense




Page 24                                                GAO-03-431 F/A-22 Aircraft
                  Appendix III: GAO Staff Acknowledgments
Appendix III: GAO Staff Acknowledgments


                  Catherine Baltzell, Marvin E. Bonner, Edward Browning, Gary Middleton,
Acknowledgments   Sameena Nooruddin, Madhav Panwar, Karen A. Richey, Don M.
                  Springman, and Ralph White made key contributions to this report.




                  Page 25                                         GAO-03-431 F/A-22 Aircraft
             Related GAO Products
Related GAO Products


             Tactical Aircraft: F-22 Delays Indicate Initial Production Rates Should
             Be Lower to Reduce Risks. GAO-02-298. Washington, D.C.: March 5, 2002.

             Tactical Aircraft: Continuing Difficulty Keeping F-22 Production Costs
             Within the Congressional Limitation. GAO-01-782. Washington, D.C.:
             July 16, 2001.

             Tactical Aircraft: F-22 Development and Testing Delays Indicate Need
             for Limit on Low-Rate Production. GAO-01-310. Washington, D.C.:
             March 15, 2001.

             Defense Acquisitions: Recent F-22 Production Cost Estimates
             Exceeded Congressional Limitation. GAO/NSIAD-00-178.
             Washington, D.C.: August 15, 2000.

             Defense Acquisitions: Use of Cost Reduction Plans in Estimating
             F-22 Total Production Costs. GAO/T-NSIAD-00-200. Washington, D.C.:
             June 15, 2000.

             Budget Issues: Budgetary Implications of Selected GAO Work for
             Fiscal Year 2001. GAO/OCG-00-8. Washington, D.C.: March 31, 2000.

             F-22 Aircraft: Development Cost Goal Achievable If Major Problems Are
             Avoided. GAO/NSIAD-00-68. Washington, D.C.: March 14, 2000.

             Defense Acquisitions: Progress in Meeting F-22 Cost and Schedule Goals.
             GAO/T-NSIAD-00-58. Washington, D.C.: December 7, 1999.

             Fiscal Year 2000 Budget: DOD’s Production and RDT&E Programs.
             GAO/NSIAD-99-233R. Washington, D.C.: September 23, 1999.

             Budget Issues: Budgetary Implications of Selected GAO Work for
             Fiscal Year 2000. GAO/OCG-99-26. Washington, D.C.: April 16, 1999.

             Defense Acquisitions: Progress of the F-22 and F/A-18E/F Engineering
             and Manufacturing Development Programs. GAO/T-NSIAD-99-113.
             Washington, D.C.: March 17, 1999.

             F-22 Aircraft: Issues in Achieving Engineering and Manufacturing
             Development Goals. GAO/NSIAD-99-55. Washington, D.C.: March 15, 1999.




             Page 26                                          GAO-03-431 F/A-22 Aircraft
           Related GAO Products




           F-22 Aircraft: Progress of the Engineering and Manufacturing
           Development Program. GAO/T-NSIAD-98-137. Washington, D.C.:
           March 25, 1998.

           F-22 Aircraft: Progress in Achieving Engineering and Manufacturing
           Development Goals. GAO/NSIAD-98-67. Washington, D.C.: March 10, 1998.

           Tactical Aircraft: Restructuring of the Air Force F-22 Fighter Program.
           GAO/NSIAD-97-156. Washington, D.C.: June 4, 1997.

           Defense Aircraft Investments: Major Program Commitments Based on
           Optimistic Budget Projections. GAO/T-NSIAD-97-103. Washington, D.C.:
           March 5, 1997.

           F-22 Restructuring. GAO/NSIAD-97-100BR. Washington, D.C.:
           February 28, 1997.

           Tactical Aircraft: Concurrency in Development and Production of
           F-22 Aircraft Should Be Reduced. GAO/NSIAD-95-59. Washington, D.C.:
           April 19, 1995.

           Tactical Aircraft: F-15 Replacement Issues. GAO/T-NSIAD-94-176.
           Washington, D.C.: May 5, 1994.

           Tactical Aircraft: F-15 Replacement Is Premature as Currently Planned.
           GAO/NSIAD-94-118. Washington, D.C.: March 25, 1994.




(120179)
           Page 27                                          GAO-03-431 F/A-22 Aircraft
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548