oversight

Military Aircraft: Institute for Defense Analyses Purchase Price Estimate for the Air Force's Aerial Refueling Aircraft Leasing Proposal

Published by the Government Accountability Office on 2003-10-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States General Accounting Office
Washington, DC 20548




                                   October 14, 2003 


                                   The Honorable John Warner 

                                   Chairman 

                                   The Honorable Carl Levin 

                                   Ranking Member 

                                   Committee on Armed Services 

                                   United States Senate 


                                   Subject: Military Aircraft: Institute for Defense Analyses Purchase Price
                                   Estimate for the Air Force’s Aerial Refueling Aircraft Leasing Proposal

                                   On September 4, 2003, we provided the Senate Committee on Armed
                                   Services our observations on the Air Force’s proposed lease of 100 Boeing
                                   767 aircraft modified for aerial refueling, to be known as the KC-767A.1 At
                                   the hearing, the Committee heard testimony from the Institute for Defense
                                   Analyses (IDA) concerning the results of a study it did on the estimated
                                   acquisition cost of each aircraft.2 IDA concluded that $120.7 million was a
                                   reasonable price for a KC-767A aerial refueling aircraft. At the time of the
                                   hearing, we had just obtained access to the IDA study and were not in
                                   position to comment on it. On September 5, 2003, you asked us to analyze
                                   the IDA study and provide you our assessment. This letter responds to that
                                   request. Our objectives were to assess the reasonableness of IDA’s
                                   approach to the issue of pricing the KC-767A aircraft and to provide any
                                   comments that we had on its methodology used to estimate the base
                                   prices and costs for each aircraft.

                                   To analyze the IDA report, we reviewed the report submitted to the Office
                                   of the Under Secretary of Defense (Acquisition Technology and Logistics)
                                   and the Office of the Director, Program Analysis and Evaluation,
                                   Department of Defense (DOD).3 We met with the IDA (Cost Analysis and
                                   Research Division) and Air Force acquisition officials from the Office of
                                   the Assistant Secretary (Acquisition), Directorate of Global Reach. We also


                                   1
                                    Military Aircraft: Observations on the Proposed Lease of Aerial Refueling Aircraft by
                                   the Air Force. GAO-03-923T. Washington, D.C.: September 4, 2003.
                                   2
                                    Purchase Price Estimate for the KC-767A Tanker Aircraft. IDA Paper P-3800.
                                   Alexandria, Virginia.: July 2003.
                                   3
                                       These offices sponsored the IDA study.



                                   Page 1                                                       GAO-04-164R Military Aircraft
             met with Office of Management and Budget officials to discuss the
             methodology, conclusions, and data used in IDA’s analysis. In addition, we
             examined the Air Force’s draft lease (which is still in negotiation and is
             subject to change) and reviewed documents and briefings from the Office
             of the Assistant Secretary of the Air Force for Acquisitions, Air Mobility
             Programs, to identify issues and costs that are related to the IDA study.
             Finally, we used data gathered for our review of the DOD response to your
             suggestion that the Air Force lease 25 aircraft and purchase 754 and
             additional data gathered for our on-going review of tanker requirements
             being conducted for the House Armed Services Committee’s
             Subcommittee on Readiness.


             We believe the IDA has provided a reasonable and comprehensive
Summary      estimate within the stated objective and assumptions given by the study
             authors. The objective of the IDA study was to establish a reasonable
             direct purchase price for 100 KC-767A aircraft. (Refer to the enclosure for
             a description of the aircraft specifications as assessed by the IDA.) The
             IDA was not asked to and did not address or make any assumptions about
             provisions of the proposed lease, financing, suitability of leasing, or any
             other acquisition alternatives to the proposal. Further, the IDA did not
             attempt to reconcile its study results to coincide with the aircraft
             configuration currently being negotiated between the Air Force
             and Boeing.


             The Department of Defense Appropriations Act for fiscal year 20025
Background   included a provision allowing the Air Force to establish a multiyear pilot
             program for leasing Boeing 767 commercial aircraft to be used as aerial
             refueling aircraft. Aerial refueling provides a key capability in enhancing
             the mobility of U.S. forces and the Air Force is in the process of planning
             for the replacement of its aging aircraft fleet. As you know, the Air Force is
             in the final stages of negotiating a lease agreement with Boeing for 100
             new 767 aircraft that will be modified for use as refueling aircraft.

             While recognizing that aerial refueling is a key capability that is essential
             to the mobility of U.S. forces, the Senate Committee on Armed Services


             4
              Military Aircraft: Observations on DOD’s Aerial Refueling Aircraft Acquisition
             Options. GAO-04-169R. Washington, D.C.: October 14, 2003.
             5
                 Pub. L. No. 107-117, § 8159, 115 Stat. 2230, 2284-85.




             Page 2                                                      GAO-04-164R Military Aircraft
                                       has raised many issues concerning the proposed Air Force lease, including
                                       questions about the validity of an independent DOD-commissioned study
                                       performed by the Institute for Defense Analyses, a federally funded
                                       research and development center. The July 2003 IDA study entitled
                                       Purchase Price Estimate for the KC-767A Tanker Aircraft concluded that
                                       $120.7 million was a “conservative, robust” estimate of a reasonable
                                       purchase price for the proposed KC-767A aircraft.


                                       To determine the cost of acquiring 100 KC-767A aircraft and to formulate
The IDA Study and                      the associated assessment methodology and data sources, IDA first
Our Assessment                         developed specific categories to describe the aircraft elements being
                                       priced. These categories,6 reflecting the most significant financial
                                       investments to the proposed aircraft purchase, were (a) basic B767-200ER
                                       aircraft, (b) enhanced B767-200ER features, (c) combination (so-called
                                       “combi”) configuration modifications, (d) auxiliary fuel tanks, (e) tanker
                                       and other modifications, and (f) development costs. The Table provides a
                                       detailed summary of IDA’s purchase price analysis.

Table 1: Summary of Institute for Defense Analyses’ KC-767A Tanker/Combi Purchase Price Analysis

 FY02 Dollars in millions
                                        IDA unit price
 Category / element                          estimate          Primary analysis technique        Primary data sources
 Basic B767-200ER                                   72.1       Commercial pricing                Consultants, Department of
                                                                                                 Transportation data
 Enhanced B767-200ER features                         1.6      Commercial pricing                Consultants, Boeing, Air Force
                                                                                                 data, IDA models, vendor quotes
 ‘Combi’ modifications                                9.5      Commercial pricing                Consultants, public data
 Auxiliary fuel tanks                                 6.3      Cost analysis                     Vendor quotes, IDA models
 Tanker and other modifications                     20.3       Cost analysis                     IDA models, Air Force, Boeing
 Development costs                                  10.9       Cost analysis                     Air Force, IDA models
 Total                                             120.7
Source: IDA and Air Force.

                                       IDA separately estimated each of the above categories using a variety of
                                       estimating techniques including cost estimating relationships, cost models,
                                       and analogous data. In most cases, they used several techniques to
                                       generate multiple cost estimates of a single item. The IDA took an average



                                       6
                                           A detailed description of these categories may be found in the IDA report page S-2.




                                       Page 3                                                            GAO-04-164R Military Aircraft
     of these estimates as the final estimate value in each category. Likewise,
     the IDA used commercial pricing techniques and marketing analysis where
     possible and traditional cost analysis techniques, as dictated by the
     content of the aircraft, where IDA determined that no significant
     commercial market existed.7 In addition, IDA’s analysis relied on data from
     a variety of public sources, including other government sources, the
     analyses of consultant organizations hired by the IDA, data supplied by
     Boeing, other aerospace suppliers, and the Air Force. The specific
     categories and our conclusions about the reasonableness of the IDA
     analysis of each category follow:

•	  Basic B767-200ER: This element represents the cost to acquire the
    baseline aircraft, the commercial aircraft upon which the KC-767A is
    based. The necessary modifications and design changes would be applied
    to this aircraft. To determine the cost, the IDA gathered pricing
    information from five different commercial and government sources.
    These prices were then averaged to obtain the price used in the study. We
    believe this was a reasonable approach. In analytical terms, the coefficient
    of variance for the five data points used is very small (4 percent), which
    shows the various estimates are close together and provides confidence in
    the estimate.8 We also note that data provided by the Air Force in October
    2003 showed that they used six different methods to estimate the cost of
    the basic B767-200ER aircraft. The prices developed under the approaches
    ranged from $60.0 million to $80.5 million per aircraft for an average of
    $71.1 million. Air Force officials ultimately used a weighted average and
    concluded that a reasonable price for the basic aircraft would be about
    $79.0 million per aircraft.
• 	 Enhanced B767-200ER features: This category includes features added
    to the basic aircraft to create the KC-767A. These items are not normally
    available on the 200ER variant of the B767 but have been specified by the
    Air Force for the KC-767A. However, Boeing has still not released its
    updated version of the system specifications negotiated for the
    B767-200ER aircraft variant. Each feature (e.g. enhanced cockpit, landing
    gear, and engines; upgraded power supply, etc.) was estimated separately
    using either commercially available information or IDA cost models.
    Again, given the complexity and assumptions associated with these


     7
       The Boeing Company has asserted that a commercial market does exist for its KC-767A
     aircraft.
     8
      Coefficient of variation is a measure of dispersion. It is the ratio of the standard deviation
     to the arithmetic mean expressed as a percent. The smaller the number, the less the
     variation in the distribution and therefore the closer observations are to the mean.




     Page 4                                                            GAO-04-164R Military Aircraft
     engineering features, we believe the IDA’s approach to be reasonable and
     comprehensive for estimating this element’s cost.
•	   ‘Combi’ modifications: This element contains modifications to the
     baseline B767-200ER to allow the aircraft to transport passengers or
     freight, or a combination of the two simultaneously. To calculate this
     estimate, IDA performed three separate estimates and then took the
     average. Because this is a well-accepted and standard means to
     conducting analysis of this type, we believe this methodology represents a
     sound approach. Also, the coefficient of variation is very small (6 percent).
     This shows that the various estimates are close together and provides
     further confidence in the estimate.
•	   Auxiliary fuel tanks: This category provides for extra fuel capacity
     (i.e., the lower fuselage fuel tanks, pumps, and installation materials
     required to give the KC-767A additional fuel capacity). IDA solicited quotes
     from two different vendors to generate the estimate. The two quotes were
     averaged for the final result. Although, for assessment purposes, the
     quotes were further apart with a coefficient of variation of 34 percent, we
     believe they are only slightly above the rule of thumb for the range of
     reasonableness of about a 30 percent coefficient of variation.
•	   Tanker and other Air Force-unique modifications: This sub-grouping
     includes modifications associated with the refueling, fuel-receiving and
     military-unique capabilities. These modifications are associated with the
     ability to receive and offload fuel, including the installation of the
     centerline boom, the hose/drogue unit, the fuel receiving receptacle, the
     remote aerial refueling operator, and the plumbing and electrical changes
     associated with these items.9 Avionics and miscellaneous Air Force
     requirements are also included in this category. The IDA separately
     estimated each modification that would occur. A cost estimating
     relationship was used to develop the final estimates, which were based on
     historical data collected by the IDA.10 We believe this to be a reasonable
     methodology and analytical tool for pricing this element.
•	   Development costs: This element, broken down into three sub-
     categories, captures any non-recurring investment costs needed to design
     the KC-767A and represents the investment Boeing would make to create
     the KC-767A configuration. The IDA accepted Air Force data for most of
     the costs and used a cost estimating relationship to estimate the



     9
       “Hose and drogue” or “probe and drogue” and “centerline boom and receptacle” refer to
     different types of refueling equipment systems used to refuel different types of aircraft
     in flight.
     10
       A cost-estimating relationship is a cost function whose arguments are variables related to
     the performance of the items or to specific features of their designs.




     Page 5                                                         GAO-04-164R Military Aircraft
                            remainder. The IDA assumed that a portion of these development costs 

                            would be borne by the impending foreign sales to Italy and Japan and

                            adjusted their estimate accordingly. We believe this is a reasonable 

                            approach for pricing this element. 

                            In addition, as a further check on its assessment approach, the IDA 

                            enlisted an expert panel of former and/or retired distinguished Air Force 

                            officials to review its analytical approach and study methodology as well 

                            as its positions on a range of reasonable prices for the KC-767A. IDA 

                            officials told us that the panel concurred with IDA’s final report 

                            assessment and reporting results. 


                            In its analysis, IDA used a December 19, 2002, system specification 

                            document identifying the KC-767A aircraft configuration. The notional 

                            aircraft is configured as a combination aerial refueling, cargo, and 

                            passenger aircraft designed to permit both freight hauling and passenger 

                            transport in the same mission. We provide a detailed description of the 

                            notional aircraft in appendix I. To obtain a lower negotiated price on the 

                            KC-767A aircraft, the Air Force has changed the specifications to eliminate 

                            the requirement that the aircraft be capable of passenger transport and 

                            cargo hauling on the same flight while retaining the other planned combi 

                            modifications. 



Air Force and Office of 
   Both the Air Force and Office of Management and Budget acknowledge
Management and Budget 
     that the IDA study was a useful and valuable tool used in negotiations with
Comments on the 
           Boeing and believe it assisted the Air Force in negotiating a lower price for
                            the 100 KC-767A aircraft leasing package. However, the Air Force believes
IDA Study 
                 that the IDA purchase price estimate did not have the “fidelity” or
                            accuracy of its negotiation position with Boeing. Air Force officials told us
                            that any line-by-line comparison of what individual items should cost
                            between the IDA study results and those negotiated by the Air Force and
                            Boeing is of limited value because the contract negotiation price was for
                            100 aircraft and each party would undoubtedly divide the pieces
                            differently. However, the Air Force did not provide sufficient evidence to
                            explain how negotiating for 100 aircraft would necessarily lead to a higher
                            price than the average unit price for 100 aircraft given that IDA did not
                            assume that a volume discount would be given.

                            In addition, according to the Air Force, even if the IDA estimate proved
                            correct and the price per aircraft were lower than the negotiated price, the
                            government would be protected. Air Force officials pointed out that the
                            contract as currently proposed includes a best price guarantee and a
                            return–on-sales cap that enables the government to receive an equitable


                            Page 6                                               GAO-04-164R Military Aircraft
                       adjustment if Boeing’s profit exceeds the cap. While the draft contract
                       does include a “best price guarantee,” this provision only guarantees that
                       The Boeing Company will not sell comparable KC-767A aircraft for less
                       than the Air Force would pay but it does not address the question of
                       whether the Air Force could have obtained a lower price. Moreover, as we
                       testified before the Senate Committees on Armed Services, and
                       Commerce, Science and Transportation,11 it is not clear to us why the sales
                       cap is 15 percent as negotiated, when a financial analysis has concluded
                       that Boeing’s profit on commercial 767 aircraft is in the range of 6
                       percent.12

                       The Air Force also raised several other concerns with the IDA estimate
                       surrounding different acquisition strategies used, different bases for
                       estimates, etc. However, after review, we considered most of the
                       differences to be inherent in the nature of IDA’s tasking and not
                       attributable to significant discrepancies or voids in information. For
                       example, the IDA was directed to consider only the cost of a direct
                       purchase price, not a leasing arrangement, so the IDA did assume a
                       different acquisition strategy than the Air Force. However, the Air Force
                       did not provide sufficient evidence to explain how a different acquisition
                       strategy would necessarily lead to a different purchase price for the
                       aircraft. Moreover, the cost estimates used by IDA to develop its purchase
                       prices are applicable and also used by DOD cost estimators for both firm
                       fixed price and cost contracts, and the planned lease of KC-767A aircraft is
                       to be done under a firm fixed price contract.


                       In oral comments on a draft of this correspondence, representatives from
Agency Comments 
      the Air Force did not disagree with our analysis or our conclusions.
and Our Evaluation 
   However, these officials believe that they negotiated a reasonable price for
                       the aircraft as planned to be configured by the Air Force and in the
                       quantity to be delivered, and that the IDA estimate of $120.7 million per
                       aircraft was not achievable. We do not know whether such a price is
                       achievable because such analysis was outside the scope of our work.




                       11
                          Military Aircraft: Observations on the Air Force’s Plan to Lease Aerial Refueling
                       Aircraft. GAO-03-1143T. Washington, D.C.: September 3, 2003; and Military Aircraft:
                       Observations on the Proposed Lease of Aerial Refueling Aircraft by the Air Force.
                       GAO-03-923T. Washington, D.C.: September 4, 2003.
                       12
                            Morgan Stanley, Does 767 Tanker Equate to 700+ Comml Orders?, (May 30, 2003).




                       Page 7                                                        GAO-04-164R Military Aircraft
In addition, officials from the Office of the Secretary of Defense and IDA
generally concurred with our analysis and our report.


We conducted this work from September to October 2003 in accordance 

with generally accepted government auditing standards. 


Unless you announce its contents earlier, we plan no further distribution 

of this letter until 10 days from its issue date. At that time, we will send 

copies of this letter to the chairman and ranking member of the Committee 

on Armed Services, House of Representatives, and the defense 

subcommittees of the Senate and House Committees on Appropriations. 

We will send a copy to the Chairman, Subcommittee on Readiness, House 

Committee on Armed Services, for whom we are conducting a broader 

body of work in this area. We will also send copies to the Secretary of 

Defense, the Director of the Office of Management and Budget, and the

President of the Institute for Defense Analyses. We will also make copies 

available to other interested parties upon request. In addition, the letter 

will be available at no charge on the GAO Web site at http://www.gao.gov. 


We appreciate this opportunity to be of assistance. If you or your 

staffs have any questions regarding this letter, please contact me at 

(202) 512-4914 or Brian J. Lepore, Assistant Director, at (202) 512-4523. 

Other key contributors to this review were Ann M. Dubois, Joseph J. Faley,

Jennifer K. Echard, Kenneth W. Newell, Madhav S. Panwar, Charles W. 

Perdue, Kenneth E. Patton, and Tim F. Stone. 





Neal P. Curtin, Director 

Defense Capabilities and Management 


Enclosure 





Page 8                                              GAO-04-164R Military Aircraft
Enclosure I: Description of Proposed Aircraft
as Assessed by IDA

               The KC-767A tanker/combi aircraft (the aircraft can serve as an air-
               refueling tanker, can carry freight or passengers, or can combine freight
               and passengers in the same mission) is to be based on the commercial
               B767-200ER.1 Modifications would include the addition of features
               available on other Boeing 767 models, as well as changes required for the
               military application. In the tanker role, total fuel capacity is to be just over
               200,000 pounds, including up to 41,000 pounds carried in added auxiliary
               fuel tanks. The KC-767A would have the capability to perform refueling by
               the hose/drogue and boom methods from the aircraft centerline and would
               also be able to receive fuel from other refueling aircraft. The cabin of the
               KC-767A would be convertible to three configurations. In the passenger
               configuration, the KC-767A would accommodate up to 190 passengers and
               10 crewmembers. The freight configuration would accommodate 19 cargo
               pallets and 10 crewmembers. The combination (so-called “Combi”)
               configuration would have the capacity for 10 pallets, 10 crewmembers,
               and 70 passengers, although this configuration has been dropped by the
               Air Force to reduce the price of the proposed aircraft.




               1
(350437) 	      System specification for Air Force KC-767A tanker transport aircraft, Boeing Integrated
               Defense Systems, December 19, 2002.




               Page 9                                                        GAO-04-164R Military Aircraft
This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.
                           The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission              Congress, exists to support Congress in meeting its constitutional responsibilities
                           and to help improve the performance and accountability of the federal
                           government for the American people. GAO examines the use of public funds;
                           evaluates federal programs and policies; and provides analyses,
                           recommendations, and other assistance to help Congress make informed
                           oversight, policy, and funding decisions. GAO’s commitment to good government
                           is reflected in its core values of accountability, integrity, and reliability.


                           The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of        through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and            text files of current reports and testimony and an expanding archive of older
                           products. The Web site features a search engine to help you locate documents
Testimony                  using key words and phrases. You can print these documents in their entirety,
                           including charts and other graphics.
                           Each day, GAO issues a list of newly released reports, testimony, and
                           correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                           daily. The list contains links to the full-text document files. To have GAO e-mail
                           this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail
                           alerts” under the “Order GAO Products” heading.


Order by Mail or Phone 	   The first copy of each printed report is free. Additional copies are $2 each. A
                           check or money order should be made out to the Superintendent of Documents.
                           GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                           single address are discounted 25 percent. Orders should be sent to:
                           U.S. General Accounting Office
                           441 G Street NW, Room LM
                           Washington, D.C. 20548
                           To order by Phone: 	 Voice:      (202) 512-6000
                                                TDD:        (202) 512-2537
                                                Fax:        (202) 512-6061


                           Contact:
To Report Fraud,
                           Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in        E-mail: fraudnet@gao.gov
Federal Programs           Automated answering system: (800) 424-5454 or (202) 512-7470


                           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs 	           U.S. General Accounting Office, 441 G Street NW, Room 7149
                           Washington, D.C. 20548