oversight

Unmanned Aerial Vehicles: DOD's Demonstration Approach Has Improved Project Outcomes

Published by the Government Accountability Office on 1999-08-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Secretary of Defense




August 1999
                  UNMANNED AERIAL
                  VEHICLES

                  DOD’s Demonstration
                  Approach Has
                  Improved Project
                  Outcomes




GAO/NSIAD-99-33
United States General Accounting Office                                                               National Security and
Washington, D.C. 20548                                                                         International Affairs Division



                                    B-276812                                                                                     Letter

                                    August 30, 1999

                                    The Honorable William S. Cohen
                                    The Secretary of Defense

                                    Dear Mr. Secretary:

                                    The Department of Defense (DOD) needs Unmanned Aerial Vehicles (UAV)
                                    for surveillance and reconnaissance missions. Since the end of the Vietnam
                                    War, DOD began at least nine UAV acquisition programs that were later
                                    canceled, spending $4 billion in the process.1 (See app. I.) In 1994, as part
                                    of its acquisition reform efforts, DOD adopted an Advanced Concept
                                    Technology Demonstration (ACTD) strategy for assessing UAVs.2 We
                                    reviewed current UAV projects to determine whether DOD’s strategy of
                                    conducting ACTDs before developing and producing UAVs provides an
                                    improved knowledge base for making acquisition decisions. DOD has
                                    completed ACTD projects for the Predator and Outrider UAV systems and
                                    has an ongoing ACTD for the Global Hawk UAV. DOD terminated a fourth
                                    UAV project, DarkStar, before its ACTD was completed.



Results in Brief                    The ACTD strategy of focusing on mature technology and proving military
                                    utility before committing to a UAV has expanded DOD’s knowledge base,
                                    allowing it to make some well informed acquisition decisions. For
                                    example, when DOD began the Predator ACTD in 1994, the Predator was
                                    considered technologically mature because its design was based on an
                                    existing UAV, the Gnat 750. Nevertheless, DOD still required that the
                                    Predator’s performance be demonstrated. Prototypes of the Predator were
                                    deployed in Bosnia in 1995 and 1996, allowing users to determine whether
                                    the UAV would meet their needs. Only after this performance data was
                                    gathered and analyzed in 1997 was DOD willing to formally commit to the
                                    UAV’s acquisition.3 In another case, the ACTD for the DarkStar UAV, DOD


                                    1
                                     The canceled programs were Compass Cope, Compass Dwell, Aquila, Amber, Condor, Hunter, Raptor, a
                                    classified program, and the Medium Range UAV.
                                    2
                                    ACTDs are carried out to demonstrate, within 2 to 4 years, that a technologically mature system has
                                    military utility before DOD formally commits to develop and produce it.
                                    3
                                     Defense Acquisition: Advanced Concept Technology Demonstration Program Can Be Improved
                                    (GAO/NSIAD-99-4, Oct. 15, 1998).




                   Leter            Page 1                                              GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                     B-276812




                     gained knowledge early on that led to its decision not to acquire that
                     system. Likewise, for the same reason, DOD decided not to acquire the
                     joint-service Outrider UAV on a sole-source basis. DOD’s ACTD approach
                     to UAV acquisition is consistent with the best practices of leading
                     commercial developers, which require proof of technological maturity and
                     performance before they will develop or produce a product.4

                     On the other hand, DOD’s formal acquisition process, used during its earlier
                     UAV efforts, allowed programs to proceed with much less knowledge (and
                     thus higher risk) of technologies, design, and potential production
                     problems. Problems with development and production, along with the
                     associated cost and schedule increases, were a predictable consequence of
                     proceeding on such limited knowledge. For example, when DOD
                     committed to the Aquila UAV in 1979, the system was not technologically
                     mature. Several of Aquila’s key planned subsystems—such as a
                     miniaturized jam-resistant data link and a day-night sensor with laser
                     designator—did not even exist at the time. As a result, by 1982, in large
                     part due to numerous problems in developing subsystem technologies,
                     Aquila development costs had almost quintupled, and the schedule had
                     slipped 27 months.5 Nevertheless, DOD continued the program until 1987,
                     when, after spending more than $1 billion, it terminated Aquila.6



Background           UAVs are pilotless aircraft used in reconnaissance and surveillance and in
                     the identification, location, and designation of targets. A UAV system
                     includes one or more aircraft, a launch and recovery system, and a ground
                     station for flight control. During the Vietnam War, target drones were
                     modified to carry cameras and were used extensively for intelligence
                     gathering missions, avoiding risk to manned aircraft.7 After the war, DOD
                     began several UAV programs to capitalize on these demonstrated


                     4
                      Before proceeding into product development, leading commercial firms require that technology
                     development be complete. They also place a premium on demonstrated performance. See Best
                     Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD’s Environment
                     (GAO/NSIAD-98-56, Feb. 24, 1998).
                     5
                     Results of Forthcoming Critical Tests Are Needed to Confirm Army Remotely Piloted Vehicle’s
                     Readiness for Production (GAO/NSIAD-84-72, Apr. 4, 1984).
                     6
                      Aquila Remotely Piloted Vehicle: Its Potential Battlefield Contribution Still in Doubt
                     (GAO/NSIAD-88-19, Oct. 26, 1987).
                     7
                      The terms “drone” and “unmanned aerial vehicle” can be used interchangeably to refer to remotely
                     controlled aircraft.




             Leter   Page 2                                                 GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                           B-276812




                           capabilities. However, nearly all these UAV programs were terminated
                           before reaching completion.8

                           DOD initiated its ACTD program in 1994 as an acquisition reform initiative
                           to lower system costs and reduce acquisition time, and DOD modified its
                           UAV acquisition strategy to incorporate the ACTD approach. The approach
                           responded to the recommendations of the 1986 Packard Commission,
                           which was created to review defense acquisitions and determine how
                           weapon systems could be made faster and at lower cost. The Commission
                           recommended, among other things, that prototypes be built and tested to
                           assess military utility and provide a basis for realistic cost estimates before
                           a commitment to acquisition is made. We reported in October 1998 that the
                           ACTD approach can potentially cut a weapon system’s development and
                           acquisition time. 9



ACTD Results Provide       ACTD results provide DOD with a better basis for making UAV acquisition
                           decisions. DOD has completed ACTD projects for the Predator and
Better Basis for           Outrider UAVs. On the basis of the knowledge it gained during these
Decisions                  demonstrations, DOD committed to acquiring Predator UAVs and chose not
                           to acquire joint-service Outrider UAVs on a sole-source basis. Additionally,
                           because of performance and cost concerns, DOD terminated the ACTD
                           project for the DarkStar UAV before its demonstration was completed.


Predator Demonstrated in   When DOD began the ACTD for the Predator UAV in 1994, its technologies
Bosnia Before Commitment   were considered mature because the aircraft was based on an existing UAV,
                           the Gnat 750, which had been developed previously for the Central
to Production              Intelligence Agency. DOD nevertheless required that the Predator’s
                           performance be demonstrated to ensure it would meet user needs before
                           DOD committed itself to acquiring the system. Predator prototypes were
                           deployed in Bosnia in 1995 and 1996 as part of the ACTD. The performance




                           8
                            The one exception, the Navy and the Marine Corps’ Pioneer, was not acquired through the formal DOD
                           process but was procured directly from a joint venture of Israeli and U.S. firms. When it deployed
                           Pioneer on Navy ships, DOD had to spend considerable time and money resolving a number of
                           significant problems. See Unmanned Aerial Vehicles: Realistic Testing Needed Before Production of
                           Short-Range System (GAO/NSIAD-90-234, Sept. 28, 1990).
                           9
                            Defense Acquisition: Advanced Concept Technology Demonstration Program Can Be Improved
                           (GAO/NSIAD-99-4, Oct. 15, 1998).




                           Page 3                                             GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                        B-276812




                        data gathered there convinced military users that Predator was worth
                        acquiring.

                        The Predator effort began with a 30-month ACTD contract awarded in
                        January 1994 for 3 systems and 10 air vehicles. Predator’s mission is to
                        provide long-range (500 nautical miles), long endurance (more than
                        20 hours), near real-time imagery to satisfy reconnaissance, surveillance,
                        and target acquisition requirements. These capabilities were demonstrated
                        in Bosnia. The demonstration also identified some problems such as the
                        UAV’s inability to see through cloud cover and icing of its wings in cold
                        weather. However, the contractor incorporated solutions, including a
                        synthetic aperture radar and a wing de-icing system. Predator systems are
                        now in production, and the Air Force has had Predator UAVs deployed in
                        two reconnaissance squadrons: one in Hungary supporting operations in
                        Bosnia and one in Nevada.


Outrider ACTD Allowed   In our 1997 report on the joint-service Outrider ACTD, we concluded that
DOD to Avoid Unwise     DOD had underestimated the time and effort needed to successfully
                        integrate nondevelopmental items into the Outrider prototype.10 In 1998,
Commitment              after a 2-year effort, the Outrider ACTD was completed. The ACTD
                        demonstrated that the Outrider prototype did not exhibit the necessary
                        military utility for its expressed objective of meeting the combined tactical
                        UAV requirements of the Army, the Navy, and the Marine Corps. The Navy
                        then withdrew from the project in favor of pursuing a vertical take-off and
                        landing UAV system, although on the basis of the knowledge gained during
                        the ACTD, the Army determined that Outrider had sufficient military utility
                        to continue as a competitor for the Army’s tactical UAV solution. However,
                        DOD determined there was not enough justification to continue in a sole-
                        source arrangement with the Outrider system. Rather, DOD directed the
                        Army to conduct a full and open competition for a tactical UAV system.

                        The objectives of the joint Outrider ACTD included determining whether
                        the UAV could (1) operate for 3 to 4 hours at a range of 200 kilometers,
                        (2) be transported on one C-130 cargo aircraft, (3) operate on automotive
                        gasoline, and (4) be used aboard ships. These objectives were not met. The
                        demonstration showed that Outrider (1) had a 200-kilometer range for only
                        2 hours, (2) needed two C-130 aircraft to transport it, and (3) required


                        10
                         Unmanned Aerial Vehicles: Outrider Demonstrations Will Be Inadequate to Justify Further Production
                        (GAO/NSIAD-97-153, Sept. 23, 1997).




                        Page 4                                             GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                            B-276812




                            aviation fuel. The demonstration of shipboard operations was not
                            attempted. Moreover, the demonstration results showed no evidence that
                            Outrider could achieve two longer-term objectives: replacing its analog
                            data link with a digital one and installing a heavy fuel engine.


Better Knowledge Base       DOD terminated the DarkStar ACTD in January 1999, well before the
Allowed DOD to Terminate    planned completion date, after it was determined that DarkStar was not
                            aerodynamically stable and not meeting cost and performance objectives.
DarkStar ACTD Early         The ACTD required the contractor to demonstrate that the UAV had
                            military utility and that future production versions could be built for
                            $10 million each. By requiring that DarkStar demonstrate its ability to meet
                            military requirements before a commitment to acquisition was made, DOD
                            was able to acquire knowledge about cost and performance relatively
                            quickly compared with the time usually needed under the formal
                            acquisition process.

                            DarkStar was meant to be a stealthy, high-altitude reconnaissance UAV to
                            be used in high threat environments. It was expected to reach an altitude
                            of 50,000 feet, have a range of 500 nautical miles, and be able to operate for
                            8 hours. The first prototype crashed on its second flight in 1996.
                            Correcting the design problems that had caused the crash became
                            expensive and time consuming. In December 1998, we reported that DOD
                            projections showed that the unit price for future production versions of
                            DarkStar would be about $13.7 million, well above the $10 million goal.11


Global Hawk High-Altitude   The Global Hawk high-altitude endurance ACTD is progressing toward an
Endurance ACTD Is           October 2000 program decision point. At that time, DOD will have
                            completed a military utility assessment of Global Hawk and hopes to have a
Progressing                 sound knowledge base for deciding whether to convert the UAV to a formal
                            acquisition program. Global Hawks have flown more than 135 hours and
                            have reached altitudes in excess of 66,000 feet. By the end of our review,
                            four prototype Global Hawks had been built. The Air Force was scheduled
                            to formally begin assessing the Global Hawk’s military utility in April 1999
                            for the ACTD sponsor, the U.S. Atlantic Command, but this assessment was
                            delayed by a crash.



                            11
                              Unmanned Aerial Vehicles: Progress in Meeting High Altitude Endurance Aircraft Price Goals
                            (GAO/NSIAD-99-29, Dec. 15, 1998).




                            Page 5                                             GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                           B-276812




                           DOD pursued several programs for a high-altitude, long-endurance UAV
                           through the 1960s and 1970s without successfully fielding such an aircraft.
                           The Global Hawk is considered technologically mature because many of its
                           components are adapted from other proven aircraft. It is intended to reach
                           altitudes of 65,000 feet, have a range of 3,000 nautical miles, and conduct
                           reconnaissance missions for more than 24 hours. According to DOD’s plans
                           for airborne reconnaissance, high-altitude UAVs such as Global Hawk
                           would initially augment and eventually could replace manned aircraft in
                           performing high-altitude intelligence gathering missions.


ACTD Approach Consistent   DOD’s approach to acquiring UAVs using the ACTD process is consistent
With Commercial Best       with the focus on mature technology and proving performance that we
                           found in our 1998 review of leading commercial development efforts.
Practices                  Commercial firms make a distinction between technology development
                           and product development, and they demand proof of performance before
                           committing to production.

                           The purpose of technology development is to foster technological advances
                           for potential application to a product. Product development in commercial
                           ventures is a clearly defined undertaking aimed to design and manufacture
                           an item that the customer needs and wants. The process of discovery—the
                           accumulation of knowledge and the elimination of unknowns—is
                           completed for the best commercial programs well ahead of commitment to
                           development of a product. Immature or undeveloped technology is kept
                           out of commercial product development programs.



Prior UAV Acquisitions     In contrast to the ACTD approach, DOD’s formal acquisition process allows
                           technology development to continue after product development has begun.
Accepted Significant       As a result, the distinction between technology development and
Unknowns                   development of an individual product is much less clear. Consequently,
                           DOD may have much less knowledge about technologies, design, and
                           potential production problems when it commits to development or
                           acquisition of a product. Problems with development and production,
                           along with the associated cost and schedule increases, are a predictable
                           consequence of such limited knowledge.

                           Under previous programs, accurate data of a UAV’s performance, military
                           utility, and cost was not generally available when DOD made a commitment




                           Page 6                                GAO/NSIAD-99-33 Unmanned Aerial Vehicles
          B-276812




          to acquisition.12 As a result, DOD spent far more than it anticipated and
          obtained far less than it expected in terms of capabilities. Examples
          include DOD’s acquisition efforts for the Aquila, Pioneer, and Hunter UAV
          systems.


Aquila    Several planned key subsystems of the Aquila UAV did not exist when DOD
          committed to developing the Aquila in 1979. These included a modular
          integrated communication and navigation system, an anti-jam data link,
          and a forward-looking infrared payload and laser designator. After about
          4 years, Aquila development costs had grown from $123 million to
          $590 million, and numerous problems with subsystem development had
          stretched the development schedule from 43 months to 70 months.13
          Nevertheless, DOD continued until 1987, when operational testing revealed
          that the Aquila did not meet requirements. Ultimately, after spending
          8 years and $1 billion in development funds, the Army terminated the
          program.


Pioneer   DOD also committed to acquire the Pioneer UAV in the face of significant
          unknowns. Subsequently—but only after the Navy and the Marine Corps
          had begun taking delivery of Pioneer systems—numerous problems
          emerged, requiring extensive investment of resources to solve.

          Before deciding to acquire Pioneer UAVs, the Navy acquired several Mastiff
          UAVs from Israel for Naval Gunfire Support while expediting a
          procurement program for Pioneer. The Navy, however, wanted Pioneer to
          be adapted so it could take off from and land on ships. No performance
          data on how the Pioneer design would accommodate shipboard
          requirements was collected before an acquisition decision was made.
          Thus, Pioneer immediately began to encounter problems. Recoveries
          aboard ship and electromagnetic interference from other shipboard
          systems led to a significant number of crashes. The Pioneer also suffered
          from numerous other shortcomings. Ultimately, the Navy was forced to
          spend $50 million in research and development to bring nine Pioneer
          prototypes up to a level of “minimal essential capability.” Although



          12
           Unmanned Aerial Vehicles: DOD’s Acquisition Efforts (GAO/T-NSIAD-97-138, Apr. 9, 1997).
          13
           Results of Forthcoming Critical Tests Are Needed to Confirm Army Remotely Piloted Vehicle’s
          Readiness for Production (GAO/NSIAD-84-72, Apr. 4, 1984).




          Page 7                                             GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                  B-276812




                  Pioneer’s performance never met its original requirements, the Navy and
                  the Marine Corps were able to use the improved Pioneer systems in the
                  Persian Gulf War and more recently in Somalia, Bosnia, and Yugoslavia.


Hunter            The Hunter UAV program entered production in January 1993, before its
                  performance had been tested under realistic conditions.14 Seven Hunter
                  systems, at a cost of $171 million, were already being built when testing
                  disclosed serious problems with engines, software, equipment, and logistic
                  support.15 After spending an additional 3 years and a total of about
                  $700 million on the program, DOD chose not to contract for further
                  production of Hunter. The seven existing Hunter systems are now
                  deployed in contingencies such as Yugoslavia, are being used for testing
                  and training, or are in storage.



Conclusions       By taking an ACTD approach that focuses on mature technology and
                  proves performance and military utility before acquisition, DOD is basing
                  its UAV acquisition decisions on an improved body of knowledge. The
                  approach is also consistent with best commercial practices, which place a
                  premium on demonstrated performance when deciding whether to develop
                  a new product. As we have previously reported, DOD’s standard
                  acquisition process did not provide the same level of data and knowledge
                  under previous UAV programs.



Agency Comments   DOD partially concurred with a draft of this report. DOD stated that
                  although the draft report was favorable to the UAV ACTD process, it did not
                  outline critical factors that led to DOD’s decision not to pursue Outrider
                  UAV acquisition. We have made changes in the body of this report to reflect
                  DOD’s concerns. DOD’s comments are reprinted in appendix II. Additional
                  technical comments from DOD have been incorporated as appropriate.




                  14
                   Unmanned Aerial Vehicles: Performance of Short Range System Still in Question (GAO/NSIAD-94-65,
                  Dec. 15, 1993).
                  15
                     Unmanned Aerial Vehicles: No More Hunter Systems Should Be Bought Until Problems Are Fixed
                  (GAO/NSIAD-95-52, Mar. 1, 1995).




                  Page 8                                            GAO/NSIAD-99-33 Unmanned Aerial Vehicles
              B-276812




Scope and     To determine whether the ACTD approach provides an improved
              knowledge base for making UAV acquisition decisions, we reviewed and
Methodology   analyzed the history of DOD’s previous UAV acquisition efforts and the
              results of two completed UAV prototype demonstrations, for Predator and
              Outrider. We also reviewed the DarkStar High Altitude Endurance UAV’s
              history and the status of the ongoing Global Hawk High Altitude Endurance
              UAV ACTD project. We observed flight tests and reviewed test reports. We
              interviewed DOD, Air Force, Army, Marine Corps, and Navy requirements,
              acquisition, and testing officials; service user representatives; and
              contractor officials. In addition, we leveraged from our past and ongoing
              work in the area of best practices.

              We performed our work at the offices of the Secretary of Defense, the Air
              Force, the Army, the Marine Corps, and the Navy in Washington, D.C.; the
              UAV Joint Projects Office, Patuxent River, Maryland; the Air Force Air
              Combat Command, Langley Air Force Base, Virginia; the U.S. Atlantic
              Command, Norfolk, Virginia; the Army Training and Doctrine Command,
              Fort Monroe, Virginia; the Air Force Aeronautical Systems Center,
              Wright-Patterson Air Force Base, Ohio; the U.S. Army, 15th Military
              Intelligence Battalion, Fort Hood, Texas; the 304 th Military Intelligence
              Battalion, Fort Huachuca, Arizona; the U.S. Air Force, 11th Reconnaissance
              Squadron, Indian Springs, Nevada; the U.S. Marine Corps, Pioneer
              Company at 29 Palms, California; and at various contractor facilities.

              We performed our review from April 1997 to May 1999 in accordance with
              generally accepted government auditing standards.


              We are sending copies of this report to Senator John Warner, Chairman, and
              Senator Carl Levin, Ranking Minority Member, Senate Committee on
              Armed Services and Representative Floyd Spence, Chairman, and
              Representative Ike Skelton, Ranking Minority Member, House Committee
              on Armed Services. We are also sending copies to the Honorable Louis
              Caldera, Secretary of the Army; the Honorable Richard Danzig, Secretary of
              the Navy; the Honorable F. Whitten Peters, Acting Secretary of the Air
              Force; and the Honorable Jacob Lew, Director, Office of Management and
              Budget. Copies will also be made available to others upon request




              Page 9                                GAO/NSIAD-99-33 Unmanned Aerial Vehicles
B-276812




Please contact me on (202) 512-4841 if you or your staff have any questions
concerning this report. Major contributors to this report were Michael
Aiken, Terrell Bishop, Terry Parker, and Charles Ward.

Sincerely yours,




Louis J. Rodrigues
Director, Defense Acquisitions Issues




Page 10                                 GAO/NSIAD-99-33 Unmanned Aerial Vehicles
Page 11   GAO/NSIAD-99-33 Unmanned Aerial Vehicles
Appendix I

Unmanned Aerial Vehicles Program
Cancellations Through 1996                                                                              Appenx
                                                                                                             Idi




              Program                                           Dollars spent (approximate)
              Compass Cope                                      $200 million
              Compass Dwell                                      200 million
              Aquila                                             1 billion
              Amber                                              200 million
              Condor                                             400 million
              Medium Range                                       210 million
              Special Program                                    1 billion
              Raptor                                             200 million
              Hunter                                             700 million
              Total                                             $4.1 billion
              Source: Defense Airborne Reconnaissance Office.




              Page 12                                            GAO/NSIAD-99-33 Unmanned Aerial Vehicles
Appendix II

Comments From the Department of Defense                                       Appe
                                                                                 nIx
                                                                                   Idi




Note: GAO comment
supplementing those in the
report text appear at the
end of this appendix.




See comment 1.




See comment 1.




                             Page 13   GAO/NSIAD-99-33 Unmanned Aerial Vehicles
Appendix II
Comments From the Department of Defense




Page 14                                   GAO/NSIAD-99-33 Unmanned Aerial Vehicles
                  Appendix II
                  Comments From the Department of Defense




                  The following is GAO’s comment on the Department of Defense’s (DOD)
                  letter.



GAO Comment       1. We have made changes in the body of this report to reflect DOD’s
                  concerns.




(707245)   Lte
             rt   Page 15                                   GAO/NSIAD-99-33 Unmanned Aerial Vehicles
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested