oversight

Missile Defense: Opportunity Exists to Strengthen Acquisitions by Reducing Concurrency

Published by the Government Accountability Office on 2012-04-20.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States Government Accountability Office

GAO          Report to Congressional Committees




April 2012
             MISSILE DEFENSE

             Opportunity Exists to
             Strengthen
             Acquisitions by
             Reducing
             Concurrency




GAO-12-486
                                                 April 2012

                                                 MISSILE DEFENSE
                                                 Opportunity Exists to Strengthen Acquisitions by
                                                 Reducing Concurrency
Highlights of GAO-12-486, a report to
congressional committees




Why GAO Did This Study                           What GAO Found
MDA has spent more than $80 billion              In fiscal year 2011, the Missile Defense Agency (MDA) experienced mixed
since its initiation in 2002 and plans to        results in executing its development goals and Ballistic Missile Defense System
spend $44 billion more by 2016 to                (BMDS) tests. For the first time in 5 years, GAO found that all of the targets used
develop, produce, and field a complex            in this year’s tests were delivered and performed as expected. None of the
integrated system of land-, sea-, and            programs GAO assessed were able to fully accomplish their asset delivery and
space-based sensors, interceptors,               capability goals for the year. Flight test failures, an anomaly, and delays
and battle management, known as the              disrupted the development of several components and models and simulations
BMDS.                                            challenges remain. Flight test failures forced MDA to suspend or slow production
                                                 of three out of four interceptors currently being manufactured while failure review
Since 2002, National Defense
                                                 boards investigated their test problems.
Authorization Acts have mandated that
GAO prepare annual assessments of                To meet the presidential 2002 direction to initially rapidly field and update missile
MDA’s ongoing cost, schedule, testing,           defense capabilities as well as the 2009 announcement to deploy missile
and performance progress. This report            defenses in Europe, MDA has undertaken and continues to undertake highly
assesses that progress in fiscal year            concurrent acquisitions. Concurrency is broadly defined as the overlap between
2011. To do this, GAO examined the               technology development and product development or between product
accomplishments of the BMDS                      development and production. While some concurrency is understandable,
elements and supporting efforts and              committing to product development before requirements are understood and
reviewed individual element responses
                                                 technologies mature or committing to production and fielding before development
to GAO data collection instruments.
                                                 is complete is a high-risk strategy that often results in performance shortfalls,
GAO also reviewed pertinent
Department of Defense (DOD) policies             unexpected cost increases, schedule delays, and test problems. It can also
and reports, and interviewed a wide              create pressure to keep producing to avoid work stoppages. In contrast, as
range of DOD, MDA, and BMDS                      shown in the notional graphic below, successful programs that deliver promised
officials.                                       capabilities for the estimated cost and schedule use a disciplined knowledge-
                                                 based approach.
What GAO Recommends
GAO makes seven recommendations
to the Secretary of Defense to reduce
concurrency and strengthen MDA’s
near- and long-term acquisition
prospects. DOD concurred with six
recommendations and partially
concurred with one related to reporting
on the cause of the Aegis BMD
Standard Missile-3 Block IB test failure
before committing to additional
purchases. DOD did not agree to tie
additional purchases to reporting the
cause of the failure. DOD’s stated               High levels of concurrency were present in MDA’s initial efforts and are present
actions were generally responsive to             in current efforts, though the agency has begun emphasizing the need to follow
problems already at hand, but did not            knowledge-based development practices. During 2011, the Ground-based
consistently address implications for            Midcourse Defense, the Aegis Standard Missile 3 Block IB, and the Terminal
concurrency in the future, as discussed          High Altitude Area Defense experienced significant ill effects from concurrency.
more fully in the report.                        For example, MDA’s discovery of a design problem in a new variant of the
                                                 Ground-based Midcourse Defense program’s interceptors while production was
View GAO-12-486. For more information,           underway increased costs, may require retrofit of fielded equipment, and delayed
contact Cristina Chaplain at (202) 512-4841 or   delivery. Flight test cost to confirm its capability has increased from $236 million
chaplainc@gao.gov.                               to about $1 billion. Because MDA continues to employ concurrent strategies, it is
                                                 likely that it will continue to experience these kinds of acquisition problems.
                                                                                          United States Government Accountability Office
Contents


Letter                                                                                    1
               Background                                                                 3
               Mixed Progress in Development and Delivery Efforts                         6
               Limited Progress in Developing Models and Simulations; Much
                 More Remains to Be Done                                                11
               MDA’s Highly Concurrent Acquisition Strategy Magnifies the
                 Effects of Tests and Other Problems                                    13
               Conclusions                                                              26
               Recommendations for Executive Action                                     27
               Agency Comments and Our Evaluation                                       29

Appendix I     Scope and Methodology                                                    33



Appendix II    Comments from the Department of Defense                                  36



Appendix III   BMDS Models and Simulations Progress                                     40



Appendix IV    Aegis Ballistic Missile Defense (Aegis BMD) with Standard Missile-3
               (SM-3) Block IA and Block IB                                             46



Appendix V     Aegis Ballistic Missile Defense (Aegis BMD) Standard Missile-3
               (SM-3) Block IIA                                                         53



Appendix VI    Aegis Ballistic Missile Defense (Aegis BMD) Standard Missile-3
               (SM-3) Block IIB                                                         58



Appendix VII   Aegis Ashore                                                             65




               Page i                                             GAO-12-486 Missile Defense
Appendix VIII   Ground-based Midcourse Defense (GMD)                                  73



Appendix IX     Precision Tracking Space System (PTSS)                                81



Appendix X      Targets and Countermeasures                                           87



Appendix XI     Terminal High Altitude Area Defense (THAAD)                           93



Appendix XII    GAO Contact and Staff Acknowledgments                                100



Tables
                Table 1: Description of MDA’s BMDS Elements                             4
                Table 2: European Phased Adaptive Approach Plans and Delivery
                         Time Frames as of 2009                                         6
                Table 3: BMDS Fiscal Year 2011 Selected Accomplishments                 7
                Table 4: Flight Test and Failure Review Cost to Assess CE-II
                         Capabilitya                                                  75
                Table 5: Fiscal Year 2011 Acquisition Events by Target Class          90


Figures
                Figure 1: Concurrency Compared to the Knowledge-Based
                         Approach                                                     16
                Figure 2: GMD Concurrent Schedule                                     17
                Figure 3: SM-3 Block IB Concurrent Schedule                           19
                Figure 4: THAAD Concurrent Schedule                                   21
                Figure 5: SM-3 Block IIB Concurrent Schedule                          23
                Figure 6: Aegis Ashore Concurrent Schedule                            24
                Figure 7: PTSS Concurrent Schedule                                    26
                Figure 8: SM-3 Block IB Schedule                                      48
                Figure 9: SM-3 Block IIB Schedule                                     61
                Figure 10: Aegis Ashore Schedule                                      67
                Figure 11: GMD Concurrent Schedule                                    77


                Page ii                                         GAO-12-486 Missile Defense
Figure 12: PTSS Concurrent Schedule                          83
Figure 13: THAAD Concurrent Schedule                         95




Page iii                               GAO-12-486 Missile Defense
Abbreviations

Aegis BMD         Aegis Ballistic Missile Defense
APL               Applied Physics Laboratory
APUC              average procurement unit cost
BAR               BMDS Accountability Report
BMDS              Ballistic Missile Defense System
CDR               critical design review
CE-I              Capability Enhancement-I
CE-II             Capability Enhancement -II
DACS              divert and attitude control system
DOD               Department of Defense
EKV               exoatmospheric kill vehicle
eMRBM             Extended Medium-Range Ballistic Missile
FTG               Flight Test GMD
FTM               Flight Test Missile of Aegis
FTT               THAAD Flight Test
FTX               Flight Test “Other”
GBI               ground-based Interceptor
GMD               Ground-based Midcourse Defense
ICBM              intercontinental ballistic missile
IMTP              Integrated Master Test Plan
IRBM              intermediate-range ballistic missile
LV                launch vehicle
MDA               Missile Defense Agency
MRBM              medium-range ballistic missile
OSF               Objective Simulation Framework
OTA               Operational Test Agency
PAA               Phased Adaptive Approach
PDR               preliminary design review
PTSS              Precision Tracking Space System
SM-3              Standard Missile-3
SPY-1             Army/Navy Water (Shipboard) Radar Surveillance
STSS              Space Tracking and Surveillance System
TDACS             throttleable divert and attitude control system
THAAD             Terminal High Altitude Area Defense
TSRM              third-stage rocket motor
VLS               vertical launching system


This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page iv                                                        GAO-12-486 Missile Defense
United States Government Accountability Office
Washington, DC 20548




                                   April 20, 2012

                                   Congressional Committees

                                   The Missile Defense Agency (MDA) has developed and deployed an
                                   initial integrated and layered Ballistic Missile Defense System (BMDS) to
                                   defend the United States, our deployed forces, allies, and friends, and is
                                   continuing development and production of additional capabilities. Since
                                   2002, MDA has spent more than $80 billion and plans to spend an
                                   additional $44 billion through 2016 to develop a highly complex system of
                                   systems—land-, sea-, and space-based sensors, interceptors and battle
                                   management. To rapidly field missile defense capabilities, MDA has
                                   concurrently developed, produced, and fielded a variety of systems. While
                                   this approach has helped MDA to rapidly deploy initial capabilities, it also
                                   has meant that it has fielded some assets whose capabilities are
                                   uncertain or impaired. In addition, there has been limited transparency
                                   and accountability for the acquisitions, particularly limited understanding
                                   of the costs, schedules, requirements, and system effectiveness. As the
                                   United States government seeks to address growing fiscal pressures,
                                   investments in new weapon systems, like the BMDS, will face increasing
                                   scrutiny to ensure that they are providing the best value for the
                                   increasingly limited resources available.

                                   Since 2002, National Defense Authorization Acts have mandated that we
                                   prepare annual assessments of MDA’s ongoing cost, schedule, testing,
                                   and performance progress. 1 To date, we have delivered assessments of
                                   MDA’s progress covering fiscal years 2003 through 2010 and are
                                   currently mandated to continue delivering assessments through fiscal




                                   1
                                     National Defense Authorization Act for Fiscal Year 2002, Pub. L. No. 107-107, § 232(g)
                                   (2001); Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub.
                                   L. No. 108-375, § 233 (2004); National Defense Authorization Act for Fiscal Year 2006,
                                   Pub. L. No. 109-163, § 232; John Warner National Defense Authorization Act for Fiscal
                                   Year 2007, Pub. L. No. 109-364, § 224 (2006); and National Defense Authorization Act for
                                   Fiscal Year 2008, Pub. L. No. 110-181, § 225. See also National Defense Authorization
                                   Act for Fiscal Year 2012, Pub. L. No. 112-81, § 232 (2011).




                                   Page 1                                                       GAO-12-486 Missile Defense
year 2016. 2 This report provides our assessment of MDA’s progress in
fiscal year 2011. Specifically, it highlights (1) progress and challenges in
delivery of assets for key programs, or elements; (2) progress in
developing models and simulations needed to assess BMDS
performance; and (3) challenges related to MDA’s use of highly
concurrent acquisition strategies. In terms of costs, we report on the cost
effects MDA experienced or is likely to experience for individual systems
as a result of its past, ongoing, and planned acquisition practices. We will
not be able to assess aggregate cost reporting until corrective actions we
recommended are implemented and substantial improvements are made
to MDA’s cost estimates. According to MDA senior-level officials, the
agency has taken several actions in response to our recommendations.
Finally, we will be issuing a separate report assessing MDA’s progress in
adopting best practices for developing project schedules as well as
broader progress in enhancing and enabling Department of Defense
(DOD) and congressional oversight for MDA.

To assess MDA’s progress, we examined the accomplishments of eight
BMDS elements and supporting efforts that MDA is currently developing
and fielding: the Aegis Ballistic Missile Defense (Aegis BMD) with
Standard Missile-3 Block IA and Block IB; Aegis Ashore; Aegis BMD
Standard Missile-3 Block IIA, Aegis BMD Standard Missile-3 Block IIB;
Ground-based Midcourse Defense (GMD); Precision Tracking and Space
System (PTSS); Targets and Countermeasures; and Terminal High




2
  GAO, Missile Defense: Actions Needed to Improve Transparency and Accountability,
GAO-11-372 (Washington, D.C.: Mar. 24, 2011); Defense Acquisitions: Missile Defense
Transition Provides Opportunity to Strengthen Acquisition Approach, GAO-10-311
(Washington, D.C.: Feb. 25, 2010); Defense Acquisitions: Production and Fielding of
Missile Defense Components Continue with Less Testing and Validation Than Planned,
GAO-09-338 (Washington, D.C.: Mar. 13, 2009); Defense Acquisitions: Progress Made in
Fielding Missile Defense, but Program Is Short of Meeting Goals, GAO-08-448
(Washington, D.C.: Mar. 14, 2008); Defense Acquisitions: Missile Defense Acquisition
Strategy Generates Results but Delivers Less at a Higher Cost, GAO-07-387
(Washington, D.C.: Mar. 15, 2007); Defense Acquisitions: Missile Defense Agency Fields
Initial Capability but Falls Short of Original Goals, GAO-06-327 (Washington, D.C.:
Mar. 15, 2006); Defense Acquisitions: Status of Ballistic Missile Defense Program in 2004,
GAO-05-243 (Washington, D.C.: Mar. 31, 2005); and Missile Defense: Actions Are
Needed to Enhance Testing and Accountability, GAO-04-409 (Washington, D.C.: Apr. 23,
2004).




Page 2                                                        GAO-12-486 Missile Defense
             Altitude Area Defense (THAAD). 3 We reviewed individual element
             responses to GAO data collection instruments, which detailed key
             accomplishments for fiscal year 2011. The results of these reviews are
             presented in detail in appendixes to this report and are also integrated as
             appropriate in our findings related to progress in delivering assets and
             capabilities. For more details on our scope and methodology, see
             appendix I.

             We conducted this performance audit from April 2011 to April 2012 in
             accordance with generally accepted government auditing standards.
             Those standards require that we plan and perform the audit to obtain
             sufficient, appropriate evidence to provide a reasonable basis for our
             findings and conclusions based on our audit objectives. We believe that
             the evidence obtained provides a reasonable basis for our findings and
             conclusions based on our audit objectives.


             MDA’s BMDS is being designed to counter ballistic missiles of all
Background   ranges—short, medium, intermediate, and intercontinental. 4 Since
             ballistic missiles have different ranges, speeds, sizes, and performance
             characteristics, MDA is developing multiple systems that when integrated,
             provide multiple opportunities to destroy ballistic missiles before they can
             reach their targets. The system includes space-based sensors as well as
             ground- and sea-based radars, ground- and sea-based interceptor
             missiles, and a command and control, battle management, and
             communications system providing the warfighter with the necessary
             communication links to the sensors and interceptor missiles. A typical
             engagement scenario to defend against an intercontinental ballistic
             missile would occur as follows:




             3
              The BMDS also includes other elements and supporting efforts such as the Command,
             Control, Battle Management, and Communications and BMDS Sensors which are not
             covered in this report. For this report, we selected and focused our efforts on eight of the
             BMDS elements based on Congressional interest, known acquisition challenges and
             successes, and the current status of these efforts.
             4
               Ballistic missiles are classified by range: short-range ballistic missiles have a range of
             less than 1,000 kilometers (621 miles); medium-range ballistic missiles have a range from
             1,000 to 3,000 kilometers (621 to1,864 miles); intermediate-range ballistic missiles have a
             range from 3,000 to 5,500 kilometers (1,864 to 3,418 miles); and intercontinental ballistic
             missiles have a range greater than 5,500 kilometers (3,418 miles).




             Page 3                                                           GAO-12-486 Missile Defense
                                              •    Infrared sensors aboard early-warning satellites detect the hot plume
                                                   of a missile launch and alert the command authority of a possible
                                                   attack.

                                              •    Upon receiving the cue, land- or sea-based radars are directed to
                                                   track the various objects released from the missile and, if so
                                                   designed, to identify the warhead from among spent rocket motors,
                                                   decoys, and debris.

                                              •    When the trajectory of the missile’s warhead has been adequately
                                                   established, an interceptor—consisting of a kill vehicle mounted atop
                                                   a booster—is launched to engage the threat. The interceptor boosts
                                                   itself toward a predicted intercept point and releases the kill vehicle.

                                              •    The kill vehicle uses its onboard sensors and divert thrusters to
                                                   detect, identify, and steer itself into the warhead. With a combined
                                                   closing speed of up to 10 kilometers per second (22,000 miles per
                                                   hour), the warhead is destroyed above the atmosphere through a “hit
                                                   to kill” collision with the kill vehicle.

                                              •    Inside the atmosphere, interceptors kill the ballistic missile using a
                                                   range of mechanisms such as direct collision between the interceptor
                                                   missile and the inbound ballistic missile or killing it with the combined
                                                   effects of a blast fragmentation warhead (heat, pressure, and
                                                   grains/shrapnel) in cases where a direct hit does not occur.

                                              Table 1 provides a brief description of eight BMDS elements and
                                              supporting efforts currently under development by MDA.

Table 1: Description of MDA’s BMDS Elements
                                 a
BMDS element/supporting effort                    Description
Aegis Ballistic Missile Defense (Aegis BMD)       Aegis BMD is a sea-based missile defense system being developed in incremental,
with Standard Missile-3 (SM-3) Block IA and       capability-based blocks to defend against ballistic missiles of all ranges. Key
Block IB                                          components include the shipboard SPY-1 radar, SM-3 missiles, and command and
                                                  control systems. It also is used as a forward-deployed sensor for surveillance and
                                                  tracking of ballistic missiles. The SM-3 missile has multiple versions in development or
                                                  production. The first two variants are referred to as the SM-3 Block IA and SM-3 Block
                                                      b
                                                  IB.
Aegis Ashore                                      Aegis Ashore is a future land-based variant of the ship-based Aegis BMD. It is
                                                  expected to track and intercept ballistic missiles in their midcourse phase of flight
                                                  using SM-3 interceptor variants as they become available. Key components include a
                                                  vertical launch system and a reconstitutable enclosure that houses the SPY-1 radar
                                                  and command and control system. DOD plans to deploy the first Aegis Ashore with
                                                  SM-3 Block IB in the 2015 time frame as part of the missile defense of Europe called
                                                  the European Phased Adaptive Approach (PAA).




                                              Page 4                                                          GAO-12-486 Missile Defense
                                 a
BMDS element/supporting effort             Description
Aegis BMD SM-3 Block IIA                   The SM-3 Block IIA is the third SM-3 variant to be developed for use with the sea-
                                           based and future land-based Aegis Ballistic BMD. This program began in 2006 as a
                                           joint development with Japan, and it was added to the European PAA when that
                                           approach was announced in 2009. As part of European PAA Phase III, the SM-3
                                           Block IIA is planned to be fielded with Aegis Weapons System version 5.1 by the 2018
                                           time frame.
Aegis BMD SM-3 Block IIB                   The SM-3 IIB is the fourth SM-3 variant planned. It is intended to defend against
                                           medium- and intermediate-range ballistic missiles and provide early intercept
                                           capabilities against some intercontinental ballistic missiles. The SM-3 Block IIB
                                           program began in June 2010 and is planned to be fielded by the 2020 time frame as
                                           part of the European PAA Phase IV. Given its early stage of development, program
                                           management officials stated that the SM-3 Block IIB is not managed within the Aegis
                                           BMD Program Office and has not been baselined.
Ground-based Midcourse Defense (GMD)       GMD is a ground-based missile defense system designed to destroy intermediate and
                                           intercontinental ballistic missiles during the midcourse phase of their flight. Its mission
                                           is to protect the U.S. homeland against ballistic missile attacks from North Korea and
                                           the Middle East. GMD has two ground-based interceptor variants—the Capability
                                           Enhancement I and the Capability Enhancement II. MDA has emplaced its total
                                           planned inventory of 30 interceptors at two missile field sites—Fort Greely, Alaska and
                                           Vandenberg, California.
Precision Tracking and Space System        PTSS is being developed as an operational component of the BMDS designed to
(PTSS)                                     support intercept of regional medium and intermediate range ballistic missile threats to
                                           U.S. forces and allies and long-range threats to the United States. PTSS will track
                                           large missile raid sizes after booster burn-out, which could enable earlier intercepts.
Targets and Countermeasures                MDA develops and manufactures highly complex targets for short, medium,
                                           intermediate and eventually intercontinental ranges used in BMDS flight tests to
                                           present realistic threat scenarios. The targets are designed to encompass the full
                                           spectrum of threat missile ranges and capabilities.
Terminal High Altitude Area Defense        THAAD is a ground-based missile defense system designed to destroy short- and
(THAAD)                                    medium-range ballistic missiles during the late-midcourse and terminal phases of
                                           flight. Its mission is to defend deployed U.S. forces and friendly foreign population
                                           centers.
                                       Source: MDA data.
                                       a
                                        The BMDS also includes other elements and supporting efforts such as the Command, Control,
                                       Battle Management, and Communications and BMDS Sensors efforts, which are not covered in this
                                       report.
                                       b
                                        MDA is currently developing or producing four versions of the SM-3 interceptor—IA, IB, IIA, and IIB.
                                       The SM-3 Block IA and SM-3 Block IB are the earlier variants of the missile. The SM-3 Block IIA and
                                       SM-3 Block IIB are planned to provide successively greater range and velocity to intercept medium to
                                       long-range ballistic missiles. The latter two versions are reported on separately, in appendices V and
                                       VI, respectively.




European Missile Defense               In 2009, DOD altered its approach to European defense, which originally
Acquisition Approach                   focused on ground-based interceptors from the GMD element and a large
                                       fixed radar as well as transportable X-Band radars, in order to provide
                                       defenses against long-range threats to the United States and short-,
                                       medium-, and intermediate-range Iranian threats to Europe. This new
                                       approach, referred to as the European Phased Adaptive Approach (PAA),


                                       Page 5                                                                 GAO-12-486 Missile Defense
                                          consists primarily of Aegis BMD sea-based and land-based systems and
                                          interceptors, as well as various sensors to be deployed over time as the
                                          various capabilities are matured.

                                          The European PAA policy announced by the President articulates a
                                          schedule for delivering four phases of capability to defend Europe and
                                          augment current protection of the U.S. homeland in the following time
                                          frames: Phase 1 in 2011, Phase 2 in 2015, Phase 3 in 2018, and Phase 4
                                          in 2020. DOD’s schedule for the European PAA comprises multiple
                                          elements and interceptors to provide an increasingly integrated ballistic
                                          missile defense capability. It is projected that each successive phase will
                                          deliver additional capability with respect to both threat missile range and
                                          raid size.

                                          Table 2 outlines the plans and estimated delivery time frames associated
                                          with each European PAA phase.

Table 2: European Phased Adaptive Approach Plans and Delivery Time Frames as of 2009


Phase          Plans                                                                                                       Delivery time frame
Phase I        Deploy current and proven missile defense systems, including the sea-based Aegis Weapon                     2011
               System, the SM-3 interceptor (Block IA) and sensors such as the Army/Navy Transportable
               Radar Surveillance system to address regional ballistic missile threats to Europe and
               deployed U.S. personnel and their families.
Phase II       After appropriate testing, deploy a more capable version of the SM-3 interceptor (Block IB)                 2015
               both at sea on Aegis ships and on land in the Aegis Ashore, as well as more advanced
               sensors, to expand the defended area against short- and medium-range missile threats.
Phase III      After development and testing are complete, deploy the more advanced SM-3 Block IIA                         2018
               variant currently under development at sea and on land to counter short-, medium-, and
               intermediate-range threats.
Phase IV       After development and testing are complete, deploy the SM-3 Block IIB at sea and on land to 2020
               help better cope with medium- and intermediate-range missiles and the potential future
               intercontinental range ballistic missile threat to the United States.
                                          Sources: President’s September 17, 2009, policy announcement and MDA data.




                                          MDA experienced mixed results in executing its fiscal year 2011
Mixed Progress in                         development goals and BMDS tests. For the first time in 5 years, we are
Development and                           able to report that all of the targets used in fiscal year 2011 test events
                                          were delivered as planned and performed as expected. Moreover, the
Delivery Efforts                          Aegis BMD program demonstrated the capability to intercept an
                                          intermediate-range target for the first time. Also, the THAAD program
                                          successfully conducted its first operational flight test in October 2011.



                                          Page 6                                                                       GAO-12-486 Missile Defense
                                           However, none of the programs we assessed were able to fully
                                           accomplish their asset delivery and capability goals for the year. At the
                                           same time, several critical test failures as well as a test anomaly and
                                           delays disrupted MDA’s flight test plan and the acquisition strategies of
                                           several components. Overall, flight test failures and an anomaly forced
                                           MDA to suspend or slow production of three out of four interceptors
                                           currently being manufactured. The GMD program, in particular, has been
                                           disrupted by two recent failures, which forced MDA to halt flight testing
                                           and restructure its multi-year flight test program, halt production of the
                                           interceptors, and redirect resources to return-to-flight activities.
                                           Production issues forced MDA to slow production of the THAAD
                                           interceptors, the fourth missile being manufactured.

                                           Table 3 presents a summary of selected MDA goals for fiscal year 2011
                                           that details how well these goals were accomplished. Appendixes IV
                                           through XI further detail MDA’s progress in each of the major programs.

Table 3: BMDS Fiscal Year 2011 Selected Accomplishments

Element            Fully accomplished goals                             Partially or not accomplished goals
Aegis BMD SM-3     Flight test FTM-15 demonstrated capability           Delivered 6 out of 19 planned missiles by the end of fiscal year
Block IA           required for European Phased Adaptive                2011; delivery of 12 missiles is on hold pending the results of
                   Approach (PAA) Phase I. Deployed first ship          the failure investigation of the anomaly in FTM-15. Depending
                   in support of European PAA Phase I.                  on the results, delivered missiles may have to be retrofitted.
Aegis BMD SM-3     Delivered first SM-3 Block IB developmental          The SM-3 Block IB failed to intercept the target during its first
Block IB           interceptor and fired it in the first flight test,   flight test, resulting in a failure review board investigating the
                   FTM-16 E2.                                           cause of the failure. The flight test is scheduled to be re-
                                                                        conducted in 2012, delaying the certification of the Aegis BMD
                                                                        4.0.1 weapon system.
Aegis BMD SM-3     None                                                 Subsystem preliminary design review problems led to a
Block IIA                                                               program replan that adjusted the preliminary design review
                                                                        date to fiscal year 2012 and included new subsystem reviews
                                                                        for several components. The new subsystem reviews were
                                                                        completed in fiscal year 2011 and early fiscal year 2012.
Aegis BMD SM-3     Awarded three concept definition and program Demonstration of low-cost divert and attitude control system
Block IIB          planning contracts in April 2011 and approved components was delayed until the first quarter of fiscal year
                   to begin technology development in July 2011. 2012.
Aegis Ashore       Completed preliminary design review in               A new deckhouse fabrication plan delayed the award of the
                   August 2011.                                         deckhouse fabrication contract, procurement of deckhouse
                                                                        fabrication materials, and the start of construction.
GMD                Completed three of the five limited interceptor      Flight test, FTG-06a, failure in the first quarter of fiscal year
                   upgrades, partially to resolve component             2011 resulted in interceptor production suspension pending the
                   issues identified in developmental testing and       completion of an investigation and a successful nonintercept
                   manufacturing.                                       flight test.




                                           Page 7                                                             GAO-12-486 Missile Defense
Element   Fully accomplished goals                                  Partially or not accomplished goals
PTSS      Completed system requirements and system                  Approval to begin technology development was delayed to the
          design reviews in the second quarter of fiscal            fourth quarter of fiscal year 2012.
          year 2011.
Targets   Launched all 11 targets as planned.                       Delivered 11 out of 14 targets it had planned.
THAAD     Successfully conducted first operational flight           Materiel release to Army delayed to the second quarter of fiscal
          test, FTT-12, in October 2011. Delivered 11               year 2012. THAAD delayed plans to deliver first battery to
          missiles.                                                 fiscal year 2012 because of production issues with the
                                                                    interceptor.
                                Source: GAO analysis of MDA data.

                                Note: BMDS fiscal year 2011 asset and capability deliveries for Airborne Infrared; Command, Control,
                                Battle Management, and Communications; joint U.S.-Israel BMDS; Sea-based X-band radar; and
                                Space Tracking and Surveillance System elements were not reviewed.


                                Highlights of progress and challenges this year include the following:

                                •     Targets: In prior years, we reported that problems with availability and
                                      reliability of targets had caused delays in MDA’s test program;
                                      however, in fiscal year 2011, MDA delivered 11 short- or intermediate-
                                      range targets, and all performed successfully. The targets launched
                                      during the year supported tests of several different BMDS elements,
                                      including Aegis BMD, GMD, and Patriot systems without causing
                                      major delays or failures in flight tests. 5 Among these successful flights
                                      was FTX-17, the return-to-flight of MDA’s short-range air-launched
                                      target in July 2011. This was the target’s first launch since an
                                      essential mechanism that releases it from the aircraft failed in a
                                      December 2009 THAAD flight test. After the failure, the agency
                                      identified shortcomings in the contractor’s internal processes that had
                                      to be fixed before air-launched targets could be used again in BMDS
                                      flight tests. Nineteen months later, these deficiencies appeared to be
                                      overcome when the target missile was successfully air-launched in
                                      FTX-17. To reduce risk, the flight was not planned as an intercept
                                      mission but as a target of opportunity for several emerging missile
                                      defense technologies including the Space Tracking Surveillance
                                      System.

                                •     Aegis BMD: In April 2011, the Aegis BMD program demonstrated
                                      capability for the first time to intercept an intermediate-range target,



                                5
                                 This report does not contain an assessment of the Patriot Advanced Capability-3
                                because its initial development is complete and it has been transferred to the Army for
                                production, operation, and sustainment.




                                Page 8                                                                   GAO-12-486 Missile Defense
    used remote tracking data provided by an Army/Navy Transportable
    Radar Surveillance – Model- 2 radar, and demonstrated support for
    European PAA Phase I. While the Aegis BMD program successfully
    conducted this test, there was an anomaly in a critical component of
    the SM-3 Block IA interceptor. Despite the anomaly, the interceptor
    was able to successfully intercept the target. In September 2011, the
    Aegis BMD program failed in its first attempted intercept of its SM-3
    Block IB missile. During this test—named FTM-16 Event 2—a
    problem occurred in the interceptor and it failed to intercept the target.
    The Aegis program has had to add an additional flight test and delay
    multiple additional flight tests. Program management officials stated
    the SM-3 Block IA deliveries were suspended and the SM-3 Block IB
    production was slowed while the failure reviews are conducted.

•   THAAD: The THAAD program also had some noteworthy testing
    accomplishments in 2011, successfully conducting its first operational
    flight test in October 2011. 6 This test was a significant event for the
    program as it was designed to be representative of the fielded system
    with soldiers conducting the engagement. During the test, the THAAD
    system engaged and nearly simultaneously intercepted two short-
    range ballistic missile targets. However, THAAD also experienced a
    delay in its planned flight test schedule for fiscal year 2011. A flight
    test originally scheduled for the second quarter of fiscal year 2011
    was delayed until fiscal year 2012 due to the availability of air-
    launched targets and then subsequently was canceled altogether.
    This cancellation has delayed verification of THAAD’s capability
    against a medium-range target.

•   GMD: As has been the case since 2005, testing failures continue to
    affect the GMD program in fiscal year 2011. Specifically, as a result of
    the failed flight test in January 2010, 7 MDA added a retest designated
    as FTG-06a. However, this retest also failed in December 2010 due to
    a failure in a key component of the kill vehicle. The GMD program has
    added two additional flight tests in order to demonstrate the Capability
    Enhancement II (CE-II) interceptor. However, since fiscal year 2009
    MDA has already manufactured and delivered 12 interceptors, 2 of



6
  An operational flight test is designed to test the components by having the warfighter
utilize them in an operational environment.
7
 This test—FTG-06—was planned as the first test of GMD’s enhanced version of the kill
vehicle called the Capability Enhancement II.




Page 9                                                          GAO-12-486 Missile Defense
    which have been used in flight tests, prior to halting further deliveries.
    The manufacture of components related to the failure and delivery of
    interceptors has been halted while the failure review and resolution
    actions are ongoing. MDA conducted a failure review investigation
    throughout fiscal year 2011 and concluded that the CE-II interceptor
    design does not work as intended and therefore required redesign
    and additional development. MDA is currently undergoing an
    extensive effort to overcome the design problem and return to
    intercept flight tests.

    According to a GMD program official, the program has already
    conducted over 50 component and subcomponent tests to develop a
    fix and verify the design. MDA also realigned resources from planned
    2011 testing activities to fund the investigation and fund return-to-
    intercept activities including redesign efforts. For example, the
    program delayed funding the rotation of older fielded interceptors into
    flight test assets, delayed funding interceptor manufacturing, and
    delayed purchasing ground-based interceptor (GBI) upgrade kits.
    However, the agency did continue its efforts to increase reliability of
    the interceptors through upgrades and its repair of five interceptors to
    help mitigate the effects on the production line. MDA is planning on
    upgrading 15 interceptors between fiscal years 2013 to 2017.
    Additionally, MDA plans to refurbish five older interceptors between
    2014 and 2017 to support flight tests.

•   SM-3 Block IIA: MDA recognized that the program’s schedule
    included elevated acquisition risks and, as such, took actions in fiscal
    year 2011 to reduce those risks as well as potential future cost
    growth. The program planned to hold its system preliminary design
    review (PDR)—at which it would demonstrate that the technologies
    and resources available for the SM-3 Block IIA would result in a
    product that matched its requirements—but subsystem review
    problems for key components meant the system review had to be
    adjusted by 1 year. The program appropriately added time and money
    to its program by revising its schedule to relieve schedule
    compression between its subsystem and system-level design reviews
    and incorporated lessons learned from other SM-3 variants into its
    development to further mitigate production unit costs. The program
    still expects to meet the 2018 time frame for European PAA Phase 3.




Page 10                                               GAO-12-486 Missile Defense
                      Models and simulations are critical to understanding BMDS capabilities.
Limited Progress in   The complex nature of the BMDS, with its wide range of connected
Developing Models     elements, requires integrated system-level models and simulations to
                      assess its performance in a range of system configurations and
and Simulations;      engagement conditions. Assessing BMDS performance through flight
Much More Remains     tests alone is prohibitively expensive and faces safety and test range
to Be Done            limitations that can best be dealt with through sound, realistic models and
                      simulations.

                      Ensuring that the models and simulations are sound and realistic requires
                      a rigorous process to accomplish two main tasks: (1) developing
                      individual system models and realistically linking those models and
                      simulations and (2) gathering data from MDA’s ground and flight tests to
                      feed into the models. MDA attempts to confirm that the models re-create
                      the actual performance found in BMDS test events. 8 The Operational Test
                      Agency (OTA) independently assesses how realistic the models are in a
                      formal process called accreditation. When a model is accredited it means
                      that it can be trusted to produce high-confidence results for its intended
                      use, and the limitations of the model are known. The development of
                      reliable MDA models depends upon the collection of test data upon which
                      to anchor the models. Because MDA had made very limited progress in
                      identifying and collecting needed data, MDA’s test program was
                      reoriented beginning in 2010 to enable the collection of data to support
                      the development of BMDS models.

                      MDA has made some limited progress in developing the individual system
                      models and linking those models. Originally, MDA’s models had been
                      developed for use only by each element, not for integrated assessments.
                      MDA is still developing these individual element models, while at the
                      same time linking the models to show BMDS-level performance. Since
                      fiscal year 2010, MDA has made progress in creating a common
                      framework, whereby the various BMDS element-level hardware-in-the-
                      loop 9 models are subjected to a common and consistent scene and
                      environment during test events. MDA is now using this framework, known
                      as the Single Stimulation Framework, in assessing BMDS performance.



                      8
                          This process is called anchoring.
                      9
                        With hardware-in-the-loop models, simulations are conducted with actual mission
                      components/hardware in a laboratory environment, and the physical
                      environment/conditions are simulated, under the control of computer equipment.




                      Page 11                                                      GAO-12-486 Missile Defense
MDA officials highlight that the framework is being used to evaluate
BMDS performance in increasingly complex and realistic scenarios,
employing greater numbers of BMDS assets.

The process of developing and linking these models is extremely complex
and difficult and will take many years to accomplish. In August 2009, the
U.S. Strategic Command and OTA jointly informed MDA of 39 system-
level limitations in MDA’s models and simulations program that adversely
affect their ability to assess BMDS performance. Resolution of these 39
limitations, OTA maintains, would permit MDA’s models and simulations
to provide more realistic representations of BMDS performance using the
full complement of fielded BMDS assets. OTA officials have noted that
since August 2009, MDA has partially or fully resolved 7 of these issues
and identified technical solutions for 15 more. According to OTA officials,
most of the resolved limitations are issues that are more easily
addressed, such as the installation of improved communications systems
and the provision of separate workstations for simulation controllers. No
technical solutions have yet been identified for the remaining 17 of the 39
issues and OTA officials maintain that they are still awaiting an MDA
timeline for the complete resolution of these remaining limitations.

We reported, in 2009, problems with MDA’s model development and the
lack of flight test data. In 2009, MDA undertook a new approach to test
planning to focus the test program on gathering critical test data needed
for modeling and simulation. Since 2009, MDA has bolstered efforts to
collect test data for the BMDS model and simulation program; however,
considerable effort and time are required to address all known shortfalls.
Through its ongoing test data collection activities, MDA has collected 309
critical variables since 2009; however, those represent only 15 percent of
the total needed. Flight test failures, anomalies and delays have reduced
the amount of real-world data MDA expected. Additionally, some required
data are difficult to collect, posing challenges even when a flight test is
properly executed. When tests are carried out, considerable post-test
data analysis is required for model development. Under the current plan,
MDA does not foresee complete collection of data on these critical
variables until sometime between 2017 and 2022.

MDA has also made some limited progress in achieving partial
accreditation for some BMDS models—ensuring that they are realistic
and can be trusted and that their limitations are known. MDA models are
accredited for specific functions for which they are to be employed. Over
the past few years, OTA officials have performed assessments of MDA’s
models and simulations and have noted that, amongst the element-level


Page 12                                             GAO-12-486 Missile Defense
                        models, those for THAAD and Aegis BMD are farthest along. While MDA
                        has made some progress toward accreditation of element models for
                        specific functional areas, MDA has not yet achieved OTA accreditation in
                        other key areas, such as any of the 18 environmental models. See
                        appendix III for further details on MDA’s modeling and simulation efforts.


                        To meet the 2002 presidential direction to initially rapidly field and update
MDA’s Highly            missile defense capabilities as well as the 2009 presidential
Concurrent              announcement to deploy missile defenses in Europe, MDA has
                        undertaken and continues to undertake highly concurrent acquisitions.
Acquisition Strategy    For example, large-scale acquisition efforts were initiated before critical
Magnifies the Effects   technologies were fully understood and programs were allowed to move
of Tests and Other      forward into production without having tests completed to verify
                        performance. Such practices enabled MDA to quickly ramp up efforts in
Problems                order to meet tight presidential deadlines, but they were high risk and
                        resulted in problems that required extensive retrofits, redesigns, delays,
                        and cost increases. A program with high levels of concurrency
                        (1) proceeds into product development before technologies are mature or
                        appropriate system engineering has been completed or (2) proceeds into
                        production before a significant amount of independent testing is
                        conducted to confirm that the product works as intended. High levels of
                        concurrency were present in MDA’s initial efforts and are present in
                        current efforts.

                        Recently, the agency has begun emphasizing the need to follow
                        knowledge-based development practices, which encourage accumulating
                        more technical knowledge before program commitments are made and
                        conducting more testing before production is initiated. Developmental
                        challenges and delays are to be expected in complex acquisitions, such
                        as those for missile defense. However, when concurrency is built into
                        acquisition plans, any developmental challenges or delays that do occur
                        exacerbate the cost, schedule, and performance effects of those
                        problems, particularly when production lines are disrupted or assets have
                        already been manufactured and must be retrofitted. In 2009, we
                        recommended that MDA synchronize the development, manufacturing,
                        and fielding schedules of BMDS assets with the testing and validation
                        schedules to ensure that items are not manufactured for fielding before
                        their performance has been validated through testing. In response, DOD
                        partially concurred with our recommendation, maintaining that MDA was
                        pursuing synchronization of development, manufacturing, and fielding of
                        BMDS assets with its established testing and validation requirements.
                        However, because MDA continues to employ concurrent strategies, it is



                        Page 13                                              GAO-12-486 Missile Defense
                         likely that it will continue to experience these types of acquisition
                         problems.


Highly Concurrent        Concurrency is broadly defined as the overlap between technology
Acquisition Strategies   development and product development or between product development
Often Lead to Cost,      and production of a system. The stated rationale for concurrency is to
                         introduce systems in a timelier manner, to fulfill an urgent need, to avoid
Schedule, and            technology obsolescence and to maintain an efficient industrial
Performance              development and production workforce. While some concurrency is
Consequences             understandable, committing to product development before requirements
                         are understood and technologies mature as well as committing to
                         production and fielding before development is complete is a high-risk
                         strategy that often results in performance shortfalls, unexpected cost
                         increases, schedule delays, and test problems. 10 At the very least, a
                         highly concurrent strategy forces decision makers to make key decisions
                         without adequate information about the weapon’s demonstrated
                         operational effectiveness, reliability, logistic supportability, and readiness
                         for production. Also, starting production before critical tests have been
                         successfully completed has resulted in the purchase of systems that do
                         not perform as intended. These premature commitments mean that a
                         substantial commitment to production has been made before the results
                         of testing are available to decision makers. Accordingly, they create
                         pressure to keep producing to avoid work stoppages even when problems
                         are discovered in testing. These premature purchases have affected the
                         operational readiness of our forces and quite often have led to expensive
                         modifications.

                         In contrast, successful programs that deliver promised capabilities for the
                         estimated cost and schedule follow a systematic and disciplined
                         knowledge-based approach. This approach recognizes that development
                         programs require an appropriate balance between schedule and risk and,
                         in practice, programs can be executed successfully with some level of
                         concurrency. For example, it is appropriate to order long-lead production
                         material in advance of the production decision, with the pre-requisite that
                         developmental testing is substantially accomplished and the design


                         10
                           GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early Improves
                         Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002), and Defense
                         Acquisitions: Production and Fielding of Missile Defense Components Continue with Less
                         Testing and Validation Than Planned, GAO-09-338 (Washington, D.C.: Mar. 13, 2009).




                         Page 14                                                     GAO-12-486 Missile Defense
confirmed to work as intended. We have found that, in this approach, high
levels of product knowledge are demonstrated at critical points in
development. 11 This approach is not unduly concurrent because
programs take steps to gather knowledge that demonstrates that their
technologies are mature, their designs are stable, and their production
processes are in control before transitioning between acquisition phases.
This knowledge helps programs identify risks early and address them
before they become problems. It is a process in which technology
development and product development are treated differently and
managed separately. The process of technology development culminates
in discovery—the gathering of knowledge—and must, by its very nature,
allow room for unexpected results and delays. The process of developing
a product culminates in delivery and therefore gives great weight to
design and production. If a program is falling short in technology maturity,
it is harder to achieve design stability and almost impossible to achieve
production maturity. It is therefore key to separate technology from
product development and product development from production—in other
words, it is key to avoid concurrency when these transitions are made.
The result of a knowledge-based approach is a product delivered on time,
within budget, and with the promised capabilities.

See figure 1 for depictions of a concurrent schedule and a schedule that
uses a knowledge-based approach.




11
  GAO, Defense Acquisitions: Assessments of Selected Major Weapon Programs,
GAO-06-391 (Washington, D.C.: Mar. 31, 2006).




Page 15                                                 GAO-12-486 Missile Defense
                          Figure 1: Concurrency Compared to the Knowledge-Based Approach




Programs That Undertook   In fiscal year 2011, due to flight test failures and a flight test anomaly,
Highly Concurrent         MDA suspended production of two interceptors—one in the GMD
Strategies and            program and one in the Aegis BMD program—and slowed production of a
                          third—in the Aegis BMD program. In addition, development problems with
Consequences              a key THAAD component disrupted that program’s interceptor production.

Ground-based Midcourse    MDA undertook a highly concurrent acquisition strategy to meet the
Defense                   President’s 2002 directive to deploy an initial set of missile defense
                          capabilities by 2004. To do so, the GMD element concurrently matured
                          technology, designed the system, tested the design, and produced and
                          fielded a system. While this approach allowed GMD to rapidly field a
                          limited defense that consisted of five CE-I interceptors and a fire control
                          system, the concurrency resulted in unexpected cost increases, schedule
                          delays, test problems, and performance shortfalls. Since then, MDA has
                          produced and emplaced all of its planned CE-I interceptors. To address
                          issues with the CE-I interceptors, MDA has undertaken an extensive
                          retrofit and refurbishment program.

                          Prior to MDA fully completing development and demonstrating the
                          capability of the initial interceptor, MDA committed in 2004 to another




                          Page 16                                             GAO-12-486 Missile Defense
highly concurrent development, production, and fielding strategy for an
enhanced version of the interceptor—CE-II—as shown in figure 2. 12

Figure 2: GMD Concurrent Schedule




Note: CE-I development began in 1996—the first interceptor was delivered in 2004, the first intercept
was completed in 2006, delivery was completed in first quarter of 2009.Testing continues.


MDA proceeded to concurrently develop, manufacture, and deliver 12 of
these interceptors before halting manufacture of components and delivery
of interceptors in 2011 due to the failure in FTG-06a. 13 Although MDA
had not successfully tested this interceptor, failing in both its attempts, it
manufactured and delivered 12 of these interceptors.

The discovery of the design problem while production is under way has
increased MDA costs, led to a production break, may require retrofit of
fielded equipment, delayed delivery of capability to the war-fighter, and
altered the flight test plan. For example, the flight testing cost to confirm
the CE-II capability has increased from $236 million to about $1 billion. 14



12
  An interceptor is composed of two main components: the booster and an
exoatmospheric kill vehicle. The CE-II upgrade was intended to update certain
components, some of which were becoming obsolete. However, updating those
components changed the performance of the interceptor and remains a substantial
developmental challenge.
13
   MDA officials stated that the agency is allowing the contractor to continue work on
those components of the EKV that would not be factors in the FTG-06a flight test failure in
order to keep the production line moving.
14
     These costs include the target, mission planning, range support, and post-test analysis.




Page 17                                                               GAO-12-486 Missile Defense
In addition, the program will have to undertake another retrofit program,
for the 10 CE-II interceptors that have already been manufactured. 15
According to a GMD program official, although the full cost is currently
unknown, he expects the cost to retrofit the CE-II interceptors to be
around $18 million each or about $180 million for all 10. Intended to be
ready for operational use in fiscal year 2009, it will now be at least fiscal
year 2013 before the warfighter will have the information needed to
determine whether to declare the variant operational.

The GMD flight test program has been disrupted by the two back to back
failures. For example, MDA has restructured the planned multiyear flight
test program in order to test the new design prior to an intercept attempt.
MDA currently plans to test the new design in a nonintercept test in fiscal
year 2012.

Because MDA prematurely committed to production before the results of
testing were available, it has had to take steps to mitigate the resulting
production break, such as accelerating retrofits to 5 of the CE-I
interceptors. Program officials have stated that if the test confirms that the
cause of the failure has been resolved, the program will restart the
manufacturing and integration of the CE-II interceptors. According to
MDA, because of the steps taken to develop and confirm the design
change, a restart of the CE-II production line at that time will be low risk.
However, while MDA has established a rigorous test plan to confirm that
the design problem has been overcome, the confirmation that the design
works as intended through all phases of flight, including the actual
intercept, will not occur until an intercept test—FTG-06b—currently
scheduled for the end of fiscal year 2012 or the beginning of fiscal year
2013.

High levels of concurrency will continue for the GMD program even if the
next two flight tests are successful. GMD will continue its developmental
flight testing until at least 2022, well after production of the interceptors
are scheduled to be completed. MDA is accepting the risk that these
developmental flight tests may discover issues that require costly design
changes and retrofit programs to resolve. As we previously reported, to
date all GMD flight tests have revealed issues that led to either a



15
  Since fiscal year 2009 MDA has manufactured and delivered 12 interceptors, 2 of which
have been used in flight tests.




Page 18                                                     GAO-12-486 Missile Defense
                          hardware or software change to the ground-based interceptors. 16 See
                          appendix VIII for more details on the GMD program.

Aegis BMD SM-3 Block IB   The SM-3 Block IB program, the second version of the SM-3 interceptor,
                          is facing both developmental and production challenges that are
                          exacerbated by its concurrent schedule, as shown in figure 3. This
                          interceptor shares many components with the SM-3 Block IA, but the
                          kinetic warhead is new technology that is being developed. The need to
                          meet the presidential directive to field the Aegis BMD 4.0.1/SM-3 Block IB
                          by the 2015 time frame for European missile defense is a key driver for
                          the high levels of concurrency.

                          Figure 3: SM-3 Block IB Concurrent Schedule




                          In response to previous developmental problems and to prevent a
                          production break, MDA has twice had to purchase additional SM-3 Block
                          IA interceptors and faces a similar decision in fiscal year 2012. According
                          to MDA, the additional SM-3 Block IA missiles were purchased to avoid a
                          production gap as well as to keep suppliers active, and to meet
                          combatant command SM-3 missile quantity requirements. The program,
                          according to program management officials, was scheduled to purchase
                          the last SM-3 Block IA in fiscal year 2010 and transition to procurement
                          production of the SM-3 Block IB missiles in fiscal year 2011.

                          MDA began purchasing the SM-3 Block IB in 2009 beyond the numbers
                          needed for flight testing while a critical maneuvering technology was



                          16
                               GAO-11-372.




                          Page 19                                             GAO-12-486 Missile Defense
immature and prior to a successful flight test. According to the Director,
MDA these missiles support development and operational testing; prove
out manufacturing processes; provide information on reliability,
maintainability and supportability; verify and refine cost estimates; and
ensure that the missile will meet its performance requirements on a
repeatable basis. MDA has determined that 18 of the 25 SM-3 Block IB
missiles ordered are to be used for developmental testing; the remaining
7 interceptors are currently unassigned for tests and may be available for
operational use. 17 According to program management officials, these
unassigned rounds represent a small portion of the total planned
purchases.

MDA is also planning to purchase 46 additional SM-3 Block IB missiles in
fiscal year 2012. Meanwhile, testing has yet to validate the missile’s
performance, the cause of the test failures is not yet determined, and
remaining tests may not be completed until 2013. Consequently,
purchasing additional interceptors beyond those needed for development
remains premature. The first SM-3 Block IB developmental flight test
failed in September 2011, and an anomaly occurred in an April 2011 flight
test of the SM-3 Block IA. The flight test failure and the test anomaly
occurred in components that are shared between the SM-3 Block IA and
IB. Program officials are still investigating the reason for these failures.
The program was unable to validate initial SM-3 Block IB capability during
the failed September test, and program officials hope to conduct a series
of three intercept tests in fiscal year 2012 needed to validate SM-3 Block
IB capability. Depending on the timing and content of the failure review
board results, this schedule could change further.

Any SM-3 Block IB missiles ordered in fiscal year 2012 before mitigations
for the anomaly and the failure, if needed, are determined and before the
three flight tests confirm the design works as intended would be at higher
risk of cost growth and schedule delays. In addition, SM-3 Block IB
missiles already manufactured but not delivered also are at higher risk of
requiring a redesign depending on the results of the failure review.
Program management officials stated MDA has slowed SM-3 Block IB
manufacturing until the outcome of the failure review board is known. It


17
   Six of the 18 missiles will not be needed for developmental tests until fiscal year 2015
or later. SM-3 Block IB missiles are used for developmental tests of the SM-3 Block IB and
Aegis Weapons System 4.0.1 program and are planned for use in developmental tests for
Aegis Ashore and Aegis Weapons System 5.0.




Page 20                                                        GAO-12-486 Missile Defense
                              remains unclear whether the additional 46 missiles will be ordered before
                              the failure reviews are complete and the interceptor is able to
                              demonstrate that it works as intended. Recognizing the critical importance
                              of the completing the planned fiscal year 2012 intercept tests, the
                              operational need for SM-3 missiles, the relative success of the SM-3
                              Block IA, as well as the potential for a production break, the Senate
                              Committee on Appropriations directed MDA to use the fiscal year 2012
                              SM-3 Block IB funds for additional Block IA missiles should the test and
                              acquisition schedule require any adjustments during fiscal year 2012.
                              However, a decision to purchase additional SM-3 Block IA missiles in
                              fiscal year 2012 to help avoid a production break may be affected by the
                              SM-3 Block IA failure investigation that has not yet been completed.
                              Program management officials stated most deliveries of the SM-3 Block
                              IA have been suspended pending the results of the failure review.

                              See appendix IV for more details on the Aegis BMD SM-3 Block IB
                              program.

Terminal High Altitude Area   MDA awarded a contract to produce THAAD’s first two operational
Defense                       batteries in December 2006 before its design was stable and
                              developmental testing of all critical components was complete. As a
                              result, the THAAD program has experienced unexpected cost increases,
                              schedule delays, test problems, and performance shortfalls. At that time,
                              MDA’s first THAAD battery, consisting of 24 interceptors, 3 launchers,
                              and other associated assets, was to be delivered to the Army as early as
                              2009. In response to pressure to accelerate fielding the capability,
                              THAAD adopted a highly concurrent development, testing, and production
                              effort that has increased program costs and delayed fielding of the first
                              THAAD battery until early fiscal year 2012. (See fig. 4.)

                              Figure 4: THAAD Concurrent Schedule




                              Page 21                                            GAO-12-486 Missile Defense
                          Problems encountered while THAAD was concurrently designing and
                          producing assets increased costs by $40 million and caused slower
                          delivery rates of both the first and second THAAD batteries. These
                          batteries are not projected to be complete before July 2012—16 months
                          after the original estimate of March 2011. While all assets except the
                          interceptors were complete in 2010, the first operational interceptor for
                          the first THAAD battery was not produced until the second quarter of
                          fiscal year 2011. At the same time, MDA committed to purchasing more
                          assets by signing a production contract for two additional THAAD
                          batteries, despite incomplete testing and qualification of a safety device
                          on the interceptor. During fiscal year 2011, after several production start-
                          up issues, 11 of the expected 50 operational interceptors were
                          delivered. 18 Consequently, the first battery of 24 interceptors was not
                          complete and available for fielding until the first quarter of fiscal year
                          2012—more than 2 years later than originally planned. The same issues
                          have delayed the second battery as well. Although the launchers and
                          other components for the second battery were completed in 2010, the full
                          50 interceptors necessary for both batteries are not expected to be
                          delivered until July 2012.


Newer Programs Continue   MDA has taken steps to incorporate some acquisition best practices in its
High Levels of            newer programs, such as increasing competition and partnering with
Concurrency               laboratories to build prototypes. However, the SM-3 Block IIB, Aegis
                          Ashore, and the PTSS program acquisition strategies still include high or
                          elevated levels of concurrency that set the programs up for increased
                          acquisition risk, including cost growth, schedule delays, and performance
                          shortfalls.

                          •    SM-3 Block IIB: The program has high levels of concurrency because
                               it plans to commit to product development prior to holding a PDR, as
                               depicted in figure 5.




                          18
                            Twelve total interceptors were delivered by the end of fiscal year 2011, but the first,
                          produced in fiscal year 2010, was used in a flight test.




                          Page 22                                                          GAO-12-486 Missile Defense
Figure 5: SM-3 Block IIB Concurrent Schedule




Note: Given the early stage of the program, which does not yet have a baselined schedule, we are
not able to depict the production plans or the end of the product development phase for the SM-3
Block IIB.


     The need to meet the 2020 time frame announced by the President to
     field the SM-3 Block IIB for European PAA Phase IV is a key driver for
     the high levels of concurrency. The program is following some sound
     acquisition practices by awarding competitive contracts to multiple
     contractors to develop options for missile configurations and mature
     key technologies as well as planning to compete the product
     development contract. However, while the program is holding a series
     of reviews that will provide engineering insight into the SM-3 Block IIB
     design, we have previously reported that before starting development,
     programs should hold key system engineering events, culminating in
     the PDR, to ensure that requirements are defined and feasible and
     that the proposed design can meet those requirements within cost,
     schedule, and other system constraints. 19 In addition, based on the
     initial schedule developed by the program and prior history of SM-3
     interceptor development, the SM-3 Block IIB program will need to
     commit to building the first flight test vehicle prior to holding the PDR
     in order to remain on the planned test schedule. According to MDA,
     this approach is a low risk development if the program is funded at
     requested levels. The agency stated that the achievement of an initial
     operating capability will be based on technical progress and execution
     of a “fly before buy” approach.



19
  GAO, Defense Acquisitions: Assessments of Selected Weapons Programs,
GAO-11-233SP (Washington, D.C.: Mar. 29, 2011).




Page 23                                                              GAO-12-486 Missile Defense
•   Aegis Ashore: The program initiated product development and
    established a cost, schedule, and performance baseline early;
    included high levels of concurrency in its construction and
    procurement plan; and has not aligned its flight testing schedule with
    construction and component procurement decisions. The need to
    meet the 2015 time frame announced by the President to field the
    Aegis Ashore for European PAA Phase II is a key driver for the high
    levels of concurrency. The high levels of concurrency are depicted in
    figure 6.


Figure 6: Aegis Ashore Concurrent Schedule




    Aegis Ashore began product development and set the acquisition
    baseline before completing the PDR. This sequencing increased
    technical risks and the possibility of cost growth by committing to
    product development with less technical knowledge than
    recommended by acquisition best practices and without ensuring that
    requirements were defined, feasible, and achievable within cost and
    schedule constraints.

    The program has initiated procurement of components for the
    installation and plans to start fabricating two enclosures called
    deckhouses—one for operational use at the Romanian Aegis Ashore
    installation and one for testing at the Pacific Missile Range Facility—in
    fiscal year 2012, but does not plan to conduct the first intercept test of
    an integrated Aegis Ashore installation until fiscal year 2014. Further,
    the program plans to build the operational deckhouse first, meaning
    any design modification identified through system testing in the test
    deckhouse or the intercept test will need to be made on an existing



Page 24                                               GAO-12-486 Missile Defense
      deckhouse and equipment. As we have previously reported, such
      modifications on an existing fabrication may be costly.

      According to the Director of MDA, Aegis Ashore is a land adaptation
      of the Aegis weapons system sharing identical components. However,
      we previously have reported on the modifications to existing Aegis
      BMD technology that must be made to operate in a new land
      environment. 20 In addition, some of the planned components for Aegis
      Ashore are being developed for future Aegis weapon system
      upgrades and are still undergoing development. Aegis BMD program
      management officials stated that the risks of concurrency in the
      program schedule are low due to the program’s reliance on existing
      technology and the ground testing that will be completed prior to the
      first intercept test. Nevertheless, the program has a limited ability to
      accommodate delays in construction or testing.

•     PTSS: MDA approved a new acquisition strategy for PTSS in January
      2012 that acknowledges some concurrency, but program officials
      stated that they have taken steps to mitigate the acquisitions risks and
      have worked to incorporate several aspects of acquisition best
      practices into the strategy. MDA plans to develop and acquire the
      satellites in three phases. First, a laboratory-led contractor team will
      build two lab development satellites. Second, an industry team,
      selected through open competition while the laboratory team is still in
      a development phase, will develop and produce two engineering and
      manufacturing development satellites. The two laboratory-built and
      the two industry-built development satellites are planned to be
      operational. Third, there will be a follow-on decision for the industry
      team to produce additional satellites in a production phase.

      While the strategy incorporates several important aspects of sound
      acquisition practices, such as competition and short development time
      frames, there remains elevated acquisition risks tied to the
      concurrency between the lab- and industry-built developmental
      satellites, as shown in figure 7.




20
     GAO-11-372.




Page 25                                               GAO-12-486 Missile Defense
              Figure 7: PTSS Concurrent Schedule




              Because the industry-built developmental satellites will be under contract
              and under construction before on-orbit testing of the lab-built satellites,
              the strategy may not enable decision makers to fully benefit from the
              knowledge about the design to be gained from that on-orbit testing before
              making major commitments.

              See appendixes for more details on each program.


              MDA has a long history of pursuing highly concurrent acquisitions in order
Conclusions   to meet challenging deadlines set by the administration. Concurrency can
              enable rapid acquisition of critical capabilities but at a high risk,
              particularly if technologies are not well understood at the outset of a
              program, requirements are not firm, and decisions are made to keep
              moving a program forward without sufficient knowledge about issues,
              such as design, performance, and producibility. In MDA's case, many of
              its highly concurrent acquisition programs began with many critical
              unknowns. While the developmental problems that have been discovered
              in these acquisitions are inherent in complex and highly technical efforts,
              the effects were considerably magnified due to the high levels of
              concurrency, including questions about the performance of fielded assets,
              significant disruptions to production, and expensive retrofits. While MDA
              has embraced the value of reducing unknowns before making key
              decisions in some of its newer programs, such as the SM-3 Block IIA, and
              adopted good practices, such as awarding competitive contracts to
              multiple contractors in the SM-3 Block IIB program, it has continued to



              Page 26                                             GAO-12-486 Missile Defense
                      plan and implement highly concurrent approaches in others. In fact,
                      today, MDA is still operating at a fast pace, as production and fielding of
                      assets remains, in many cases, ahead of the ability to test and validate
                      them.

                      As we recommended in 2009, these disruptions can only be avoided
                      when the development, manufacture, and fielding schedules of BMDS
                      assets are synchronized with the testing and validation schedules to
                      ensure that items are not approved to be manufactured for fielding before
                      their performance has been validated through testing. Moreover, as we
                      have concluded for several years, while concurrency was likely the only
                      option to meet the tight deadlines MDA has been directed to work under,
                      having an initial capability in place should now allow the agency to
                      construct acquisition approaches that are less risky from a cost, schedule
                      and performance perspective. Near-term steps MDA can take to reduce
                      cost, schedule, and performance risks include actions such as
                      demonstrating the second GMD interceptor can work as intended before
                      resuming production and verifying that the SM-3 Block IB completes
                      developmental flight tests before committing to additional production.
                      Longer-term solutions require the Office of the Secretary of Defense to
                      assess the level of concurrency that currently exists within MDA programs
                      and where that concurrency can be reduced. Moreover, while missile
                      defense capabilities play a vital role in the United States' national security
                      and international relationships, decisions about deadlines for delivering
                      capabilities need to be weighed against the costs and risks of highly
                      concurrent approaches.


                      We recommend that the Secretary of Defense take the following seven
Recommendations for   actions to reduce concurrency and strengthen MDA’s near- and long-term
Executive Action      acquisition prospects. To strengthen MDA’s near-term acquisition
                      prospects, we recommend that the Secretary of Defense:

                      For the GMD program, direct MDA to

                          1) demonstrate that the new CE-II interceptor design works as
                             intended through a successful intercept flight test in the
                             operational environment—FTG-06b—prior to making the
                             commitment to restart integration and production efforts and

                          2) take appropriate steps to mitigate the effect of delaying the CE-II
                             production restart until a successful intercept occurs. Specific
                             consideration should be given by MDA to accelerating additional
                             needed CE-I refurbishments.


                      Page 27                                               GAO-12-486 Missile Defense
For the Aegis BMD program, direct MDA to

    3) verify the SM-3 Block IB engagement capability through the
       planned three developmental flight tests before committing to
       additional production beyond those needed for developmental
       testing and

    4) report to the Office of the Secretary of Defense and to Congress
       the root cause of the SM-3 Block IB developmental flight test
       failure, path forward for future development, and the plans to
       bridge production from the SM-3 Block IA to the SM-3 Block IB
       before committing to additional purchases of the SM-3 Block IB.

For the SM-3 Block IIB program, direct MDA to

    5) ensure that the SM-3 Block IIB requirements are defined and
       feasible and that the proposed design can meet those
       requirements within cost, schedule, and other system constraints
       by delaying the commitment to product development until the
       program completes a successful preliminary design review.

To strengthen MDA’s longer-term acquisition prospects, we recommend
that the Secretary of Defense:

    6) Direct the Office of the Under Secretary of Defense for
       Acquisition, Technology and Logistics to review all MDA
       acquisitions for concurrency, and determine whether the proper
       balance has been struck between the planned deployment dates
       and the concurrency risks taken to achieve those dates.

    7) Direct the Office of the Under Secretary of Defense for
       Acquisition, Technology and Logistics to review and report to the
       Secretary of Defense the extent to which the capability delivery
       dates announced by the President in 2009 are contributing to
       concurrency in missile defense acquisitions and recommend
       schedule adjustments where significant benefits can be obtained
       by reducing concurrency.




Page 28                                            GAO-12-486 Missile Defense
                     DOD provided written comments on a draft of this report. These
Agency Comments      comments are reprinted in appendix II. DOD also provided technical
and Our Evaluation   comments, which were incorporated as appropriate.

                     In responding to a draft of this report, DOD concurred with six of our
                     seven recommendations and commented on actions in process or
                     planned in response. In some cases, these actions are responsive to
                     immediate problems, but do not appear to consistently address the
                     implications for concurrency in the future.

                     DOD concurred with our recommendation for the GMD program to
                     demonstrate that the new CE-II interceptor design works as intended
                     through a successful intercept flight test in the operational environment—
                     FTG-06b—prior to making the commitment to restart integration and
                     production efforts. In response to this recommendation, DOD stated that
                     the program plans to restart the CE-II manufacturing upon successful
                     completion of the FTG-06b flight test. This decision will reduce the risk of
                     prematurely restarting CE-II production.

                     DOD also concurred with our recommendation for the Aegis BMD
                     program to verify the SM-3 Block IB engagement capability through the
                     planned three developmental flight tests before committing to additional
                     production, stating that the final decision to purchase SM-3 Block IB
                     missiles with DOD-wide procurement funding will be made after the next
                     three planned flight tests. We remain concerned that MDA is planning to
                     purchase 46 additional SM-3 Block IB missiles prematurely using
                     research, development, test, and evaluation funds in fiscal year 2012
                     before validating the performance of the missile and before determining
                     the root cause of test failures—risking disrupting the supply chain if
                     testing reveals the need to make design changes. We continue to believe
                     that the program should not purchase any additional missiles, regardless
                     of the type of funding used to purchase them, until the SM-3 Block IB’s
                     engagement capability has been verified through the three developmental
                     flight tests currently planned for the program. We have modified the
                     recommendation to focus on verifying the capability before committing to
                     additional production beyond the missiles needed for developmental
                     testing.


                     DOD concurred with our recommendation to direct the Office of the Under
                     Secretary of Defense for Acquisition, Technology and Logistics to review
                     all MDA acquisitions for concurrency, and determine whether the proper
                     balance has been struck between the planned deployment dates and the


                     Page 29                                              GAO-12-486 Missile Defense
concurrency risks taken to achieve those dates. In its response, DOD
stated that it will wait until fielding dates are established to undertake
concurrency assessments, and in the interim it will ensure that knowledge
is gained to support capability deliveries. However, we remain concerned
that DOD continues to focus on gaining key acquisition knowledge much
later than needed. DOD’s approach is to understand the extent to which
the design works as intended after committing to production—a high-risk
strategy—rather than before committing to production. The assessment of
concurrency should precede and should inform the setting of fielding
dates. If the department waits until fielding dates are set to assess
concurrency in the BMDS, it will miss the opportunity and accept the
performance, cost, and schedule consequences. Our position is not
unique in this regard. In recent testimony, the Acting Under Secretary of
Defense for Acquisition, Technology and Logistics confirmed that
excessive concurrency can drive cost growth and result in major schedule
disruptions that produce further inefficiency. Noting that the acceptable
degree of concurrency between development and production depends on
a range of factors, including the risk associated with the development
phase, the urgency of the need, and the likely impact on cost and
schedule of realizing that risk, he stated that the Office of the Secretary of
Defense intends to assess the levels of concurrency within programs, as
our report recommends should be done for missile defense elements.

DOD also concurred with our recommendation to direct the Office of the
Under Secretary of Defense for Acquisition, Technology and Logistics to
review and report to Secretary of Defense the extent to which the
presidentially announced capability delivery dates are contributing to
concurrency in missile defense acquisitions and recommend schedule
adjustments where significant benefits can be obtained by reducing
concurrency. DOD stated that the current missile defense program is
structured to develop and field capabilities at the earliest opportunity while
taking into account prudent risk management practices and executing a
thorough test and evaluation program. The department further noted that
when fielding dates are established, the Office of the Under Secretary of
Defense for Acquisition, Technology and Logistics will review and report
to the Secretary of Defense the extent to which presidentially announced
capability dates may be contributing to concurrency in missile defense
acquisitions and recommend schedule adjustments if significant benefits
can be obtained by reducing concurrency. Given the amount of
concurrency we have found in our reviews of the BMDS, we believe that
significant benefits can be reaped if concurrency is assessed sooner
rather than later.



Page 30                                               GAO-12-486 Missile Defense
DOD partially concurred with our recommendation to report to the Office
of the Secretary of Defense and to Congress the root cause of the SM-3
Block IB developmental flight test failure, path forward for future
development, and the plans to bridge production from the SM-3 Block IA
to the SM-3 Block IB before committing to additional purchases of the
SM-3 Block IB. DOD commented that MDA will report the root cause of
the SM-3 Block IB test failure and the path forward for future development
to the Office of the Secretary of Defense and to Congress upon
completion of the failure review in the third quarter of fiscal year 2012.
However, DOD makes no reference to delaying additional purchases until
the recommended actions are completed, instead stating that MDA is
balancing the need to demonstrate technical achievement and also
ensure that the system is thoroughly tested before fielding with the need
to keep the industrial base and supply chain healthy to ensure that
production transitions as quickly as possible. We believe that an
appropriate balance between schedule and risk is necessary for
development programs. However, our analysis has shown that MDA
undertakes acquisition strategies of accelerated development and
production that have led to disruptions in the supply chain and have
increased costs to develop some BMDS assets. We maintain our position
that MDA should take the recommended actions before committing to
additional purchases of the SM-3 Block IB.


We are sending copies of this report to the Secretary of Defense and to
the Director of MDA. In addition, the report is available at no charge on
the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact
me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made key contributions to this
report are listed in appendix XII.




Cristina Chaplain
Director
Acquisition and Sourcing Management




Page 31                                             GAO-12-486 Missile Defense
List of Committees

The Honorable Carl Levin
Chairman
The Honorable John McCain
Ranking Member
Committee on Armed Services
United States Senate

The Honorable Daniel K. Inouye
Chairman
The Honorable Thad Cochran
Ranking Member
Subcommittee on Defense
Committee on Appropriations
United States Senate

The Honorable Howard P. McKeon
Chairman
The Honorable Adam Smith
Ranking Member
Committee on Armed Services
House of Representatives

The Honorable C.W. Bill Young
Chairman
The Honorable Norman D. Dicks
Ranking Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives




Page 32                          GAO-12-486 Missile Defense
Appendix I: Scope and Methodology
             Appendix I: Scope and Methodology




             To assess the Missile Defense Agency’s (MDA) cost, schedule, testing
             and performance progress, we reviewed the accomplishments of eight
             Ballistic Missile Defense System (BMDS) elements that MDA is currently
             developing and fielding: the Aegis Ballistic Missile Defense (Aegis BMD)
             with Standard Missile-3 Block IA and Block IB; Aegis Ashore; Aegis BMD
             Standard Missile-3 Block IIA; Aegis BMD Standard Missile-3 Block IIB;
             Ground-based Midcourse Defense (GMD); Precision Tracking and Space
             System (PTSS); Targets and Countermeasures; and Terminal High
             Altitude Area Defense (THAAD). 1 We developed data collection
             instruments (DCI) that were completed by the elements’ program offices
             and reviewed the individual element responses. These instruments
             collected detailed information on schedule, cost and budget, contracts,
             testing and performance, and noteworthy progress during the fiscal year.
             We also examined the cost and resource, schedule, and test baselines as
             presented in the BMDS Accountability Report (BAR), 2 Baseline and
             Program Execution Reviews, test schedules and reports, and production
             plans. The results of these reviews are presented in detail in the element
             appendixes of this report and are also integrated as appropriate in our
             findings. We also interviewed officials within program offices and within
             MDA functional directorates, such as the Directorates for Engineering and
             Testing. We discussed the elements’ test programs and test results with
             the BMDS Operational Test Agency and the Department of Defense’s
             Office of the Director, Operational Test and Evaluation.

             To assess whether MDA elements delivered assets and achieved self-
             identified capability goals as planned in fiscal year 2011, we examined
             the 2011 BAR, and compared it to the 2010 and 2009 versions, looking
             for similarities and differences between the three. We also reviewed MDA
             briefings to congressional staffers from March 2011 and responses to our
             DCIs, which detailed key accomplishments and asset deliveries for fiscal
             year 2011. To assess progress on MDA’s development of models and
             simulations, we held discussions with officials at the Missile Defense



             1
              The BMDS also includes other elements and supporting efforts such as the Command,
             Control, Battle Management, and Communications and BMDS Sensors which are not
             covered in this report. For this report, we selected and focused our efforts on eight of the
             BMDS elements based on Congressional interest, known acquisition challenges and
             successes, and the current status of these efforts.
             2
                MDA issued an updated test baseline in March 2012, but we did not include it as part of
             this review because we received it at the end of our audit and did not have time to assess
             it. Our next assessment will include this updated test baseline.




             Page 33                                                          GAO-12-486 Missile Defense
Appendix I: Scope and Methodology




Integration and Operations Center, and the Operational Test Agency, and
reviewed budget documents and MDA’s directive on modeling and
simulation verification, validation, and accreditation.

Our work was performed at MDA headquarters in Fort Belvoir, Virginia,
and in Dahlgren, Virginia; Alexandria, Virginia; Falls Church, Virginia;
Annapolis, Maryland; Colorado Springs, Colorado; Arlington, Virginia; and
at various program offices and contractor facilities located in Huntsville,
Alabama, and Tucson, Arizona. In Fort Belvoir, we met with officials from
the GMD program office and the Advanced Technology Directorate who
manage the Aegis BMD Standard-Missile 3 Block IIB program. In
Dahlgren, we met with officials from the Aegis BMD program office, the
Aegis Ashore program office, and the Aegis Standard-Missile 3 Block IIA
program office. In Alexandria, we met with the Director, Operational Test
and Evaluation, and officials from the Institute for Defense Analysis. In
Falls Church, we met with officials from the PTSS program office. In
Arlington, we met with the Director, Developmental Test and Evaluation,
the Missile Defense Executive Board, officials in the Pentagon Office of
Strategic Warfare, and the Cost Analysis and Program Evaluation group.
In Annapolis, we met with officials from the Defense Spectrum
Organization/Joint Spectrum Center.

In Huntsville, we interviewed officials from the Airborne Infrared program
office; the Terminal High Altitude Area Defense project office; the Targets
and Countermeasures program office; and MDA’s Acquisitions
Directorate, Programs and Integration Directorate, Engineering
Directorate, Test Directorate, Cost Directorate, and Advanced
Technologies Directorate. We also met with Boeing officials in Huntsville
to discuss the failure review investigation for the FTG-06a failure, and
their plan to resolve the resulting manufacturing stop. In addition, we met
with officials from the Operation Test Agency in Huntsville to discuss
MDA’s performance assessment, as well as models and simulations.

In Colorado Springs, we met with officials from U.S. Northern Command,
the Joint Functional Component Command for Integrated Missile
Defense, and the Missile Defense Integration and Operations Center. We
met with Raytheon and Defense Contract Management Agency officials in
Tucson to discuss the manufacturing of the exoatmospheric kill vehicle
and schedule issues for GMD, respectively.

We conducted this performance audit from April 2011 to April 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain


Page 34                                             GAO-12-486 Missile Defense
Appendix I: Scope and Methodology




sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




Page 35                                           GAO-12-486 Missile Defense
Appendix II: Comments from the Department
             Appendix II: Comments from the Department
             of Defense



of Defense




             Page 36                                     GAO-12-486 Missile Defense
Appendix II: Comments from the Department
of Defense




Page 37                                     GAO-12-486 Missile Defense
Appendix II: Comments from the Department
of Defense




Page 38                                     GAO-12-486 Missile Defense
Appendix II: Comments from the Department
of Defense




Page 39                                     GAO-12-486 Missile Defense
Appendix III: BMDS Models and Simulations
                                         Appendix III: BMDS Models and Simulations
                                         Progress



Progress


Fiscal year 2011 events                                    Overview

•   The Missile Defense Agency (MDA) carried out a highly      MDA has made progress in creating a strategic framework for
    complex integrated digital simulated assessment             developing its models and simulations.
    involving six Ballistic Missile Defense System (BMDS)      Model and simulation development is challenging, and much
    elements.                                                   remains to be done.
•   MDA awarded a major contract to build a unified models     Test delays and an anomaly have limited progress in gathering
    and simulations architecture for the BMDS.                  needed data.
                                                               Limited progress made in accrediting element models.




Background and Overview                  Models and simulations are critical to understanding how capable the
                                         Ballistic Missile Defense System (BMDS) is and how well it can function.
                                         The complex nature of the BMDS, with its wide range of connected
                                         elements, requires integrated system-level models and simulations to
                                         assess its performance. Assessing BMDS performance through flight
                                         tests alone cannot be done, for it is prohibitively expensive and faces
                                         safety and test range limitations that can best be dealt with through
                                         sound, realistic models and simulations.

                                         Ensuring models and simulations that are sound and realistic requires a
                                         rigorous process to accomplish two main tasks—(1) developing individual
                                         element models and realistically linking those models and simulations and
                                         (2) gathering data from the Missile Defense Agency’s (MDA) ground and
                                         flight tests to feed into the models. The BMDS Operational Test Agency
                                         (OTA), an independent multi-service organization, then assesses how
                                         realistic the BMDS models are in order to accredit the models for use in
                                         simulating various levels of system performance. 1 When a model is
                                         accredited it means that it can be reliably trusted to produce high-
                                         confidence results for its intended use and the limitations of the model are
                                         known. Since developing reliable MDA models depends upon the



                                         1
                                           As per MDA’s Integrated Master Test Plan, or IMTP, OTA’s accreditation
                                         recommendation, when signed by the Accreditation Authority (the Commanding General
                                         of the Army Test and Evaluation Center, the lead service test agency for the BMDS), is
                                         the official certification that models and simulations’ products are acceptable for their
                                         intended use. While OTA is responsible for carrying out accreditation for this certification
                                         process, MDA also carries out internal model accreditation assessments.




                                         Page 40                                                          GAO-12-486 Missile Defense
                            Appendix III: BMDS Models and Simulations
                            Progress




                            collection of test data upon which to anchor them, MDA’s test program 2
                            plays a crucial role in model development and BMDS performance
                            assessments.


MDA Has Made Progress       MDA’s models and simulations development effort is making progress in
in Developing a Strategic   developing top-level planning documents, but two are not yet final. Two
Framework for Improving     MDA planning documents, the Integrated Master Assessment Plan and
                            the Integrated Models and Simulations Master Plan, are being developed
the Models and              to better focus and link the testing and assessment efforts. According to
Simulations Program         OTA officials, the Integrated Master Assessment Plan is based on sound
                            methodology, which should improve MDA’s models and simulations
                            program, in part by elevating BMDS evaluation and assessment
                            requirements as the key driver of test design. OTA officials noted that the
                            Integrated Models and Simulations Master Plan should also lead to a
                            greater emphasis on model development needs in driving the design of
                            MDA’s test events.


Model and Simulation        The task of developing and linking the element-level models and
Development Is              simulations together into an integrated BMDS model is extremely
Challenging, and Much       complex and difficult and will take years to accomplish. Last year, we
                            reported that the overall performance of the BMDS could not be assessed
Remains to Be Done          because MDA models and simulations had not matured sufficiently and
                            may not be fully mature until 2017. Since that time, there has been limited
                            progress in resolving model issues that would provide more realistic
                            representations of BMDS performance.

                            In August 2009, U.S. Strategic Command and OTA jointly informed MDA
                            of 39 system-level limitations in MDA’s models and simulations program
                            that adversely affect their ability to assess BMDS performance. Resolving
                            these limitations, OTA maintains, would permit MDA’s models and
                            simulations to provide more realistic representations of BMDS
                            performance using the full complement of fielded BMDS assets. MDA
                            officials have noted that since August 2009, MDA has fully resolved or is
                            in the process of resolving 7 of these issues and has identified technical
                            solutions for 15 more. According to OTA officials, most of the limitations
                            resolved are issues that are more easily addressed, such as installing



                            2
                                As laid out in MDA’s IMTP.




                            Page 41                                             GAO-12-486 Missile Defense
Appendix III: BMDS Models and Simulations
Progress




improved communications systems and providing separate workstations
for simulation controllers. No technical solutions have yet been identified
for the remaining 17 issues, and OTA officials maintain that they are still
awaiting an MDA timeline for the complete resolution of these remaining
limitations. Among the remainder are some critical model deficiency
issues, which result in modeled performance that does not reflect realistic
operation and conditions. For instance, models for certain radars have
artificial limitations constraining data processing, so that a simulation
involving high debris levels would effectively shut down the model.
Another model limitation is the need for accurate interceptor modeling for
all BMDS weapon systems in system-level assessments, the absence of
which prevents a determination of engagement success in such
simulations. 3

MDA has made some progress in developing a single, integrated model
and simulation approach for the BMDS. Originally, MDA’s models were
developed for use by each element and not for integrated assessments.
Since fiscal year 2010, MDA has made progress in creating a common
framework, whereby the various BMDS element-level hardware-in-the-
loop (HWIL) 4 models are subjected to a common and consistent scene
and environment during test events. MDA is now using this framework,
known as the Single Stimulation Framework, in assessing BMDS
performance, and MDA officials maintain that progress achieved in
developing it has facilitated MDA’s efforts to resolve some of the 39
limitations. MDA officials further highlight that the framework is being
used to evaluate BMDS performance in increasingly complex and realistic
scenarios, employing greater numbers of BMDS assets.

MDA officials have also claimed some success in creating and integrating
a purely digital model and simulation framework for the BMDS. In fiscal
year 2011, MDA officials stated that they have achieved some measure of
success in using the digital model in a key assessment of six key BMDS
elements. According to MDA officials, in this assessment, the simulation
tool was more successful than the previous major digital simulation event,


3
 MDA does, however, employ modeling tools external to the system-level simulations, in
order to model interceptor performance and thereby assess end-game performance and
probability of success.
4
  With HWIL models, closed loop simulations are conducted with actual mission
components/hardware in a laboratory environment, and the physical
environment/conditions are simulated, under the control of computer equipment.




Page 42                                                      GAO-12-486 Missile Defense
                        Appendix III: BMDS Models and Simulations
                        Progress




                        which was carried out in 2009. MDA officials have noted a downward
                        trend in simulation trouble or incident reports for both the Single
                        Stimulation Framework and the digital model.

                        MDA plans to integrate these two efforts into a single Objective
                        Simulation Framework (OSF). OSF is planned as an end-to-end
                        representation of the BMDS in support of testing, training, exercises, and
                        system development. OSF is scheduled to go online in the second
                        quarter of fiscal year 2014, with the current digital simulation architecture
                        phased out by fiscal year 2016. According to OTA officials, the common
                        BMDS-level test framework that OSF is intended to provide has multiple
                        advantages, such as the provision of a single tool with which to conduct
                        data verification cross-checks. Additionally, this tool could serve to fill
                        gaps that currently exist in the hardware-based models.


Test Delays and an      MDA’s difficulty in executing the test plan has limited the progress of
Anomaly Have Limited    modeling and simulations. The agency has refocused the design of its
Progress in Gathering   test program on collection of test data to strengthen the development of
                        the models. As we reported in 2010, MDA revised its testing approach in
Needed Data             response to GAO and Department of Defense concerns and began to
                        base test scenarios on identified modeling and simulation data needs. In
                        order to collect data required to fill certain model data gaps, MDA had
                        increased planned testing in certain areas, such as ground testing.
                        However, according to OTA officials, MDA has had difficulty conducting
                        its test plan, since actual test events are not always carried out in
                        accordance with the schedule. We have also reported consistent
                        problems in conducting tests over the past few years. Test schedule
                        disruptions delay not only the MDA test schedule, but also the models
                        and simulations’ efforts that depend on the test data.

                        Despite MDA’s increased efforts to collect test data for the BMDS model
                        and simulation program, it will take considerable effort and time to fill all
                        knowledge gaps. MDA has succeeded in collecting some 309 critical
                        variables since 2009; but, by the end of fiscal year 2011, those
                        represented only 15 percent of the required total identified by MDA.
                        Under the current plan, MDA does not foresee complete collection of
                        these data until sometime between 2017 and 2022.

                        Limited test data is a significant challenge MDA faces in developing
                        accredited models. Flight test failures, an anomaly, and delays in fiscal
                        year 2011 have reduced the amount of data MDA expected to have
                        available to support the anchoring of its models and simulations. MDA


                        Page 43                                               GAO-12-486 Missile Defense
                           Appendix III: BMDS Models and Simulations
                           Progress




                           officials also maintain that some required data are difficult to collect and
                           are challenging to obtain even when a flight test is properly executed.
                           When tests are carried out, considerable post-test data analysis is
                           required for model development, MDA officials maintain. However, MDA
                           officials indicated that MDA must often limit the scope of its analysis to
                           discrete model development objectives. Because of the challenges in
                           carrying out the full range of testing required to collect the anchoring data
                           to develop models, MDA is concurrently exploring alternative methods for
                           model development, such as greater use of subject matter experts.
                           According to OTA officials, the subject matter experts focus MDA’s efforts
                           toward scenario factors that are most important for actual and likely
                           BMDS operation, thereby reducing the amount of testing data required.


Limited Progress Made in   MDA has also made some limited progress in achieving partial
Accrediting Element        accreditation for some BMDS models. MDA models may be partially
Models                     accredited for some, but not all, intended functions due to limitations in
                           the models or gaps in the data. Over the past few years, BMDS OTA
                           officials have assessed MDA’s models and simulations in an effort to fully
                           understand the performance of the current BMDS configuration, and have
                           noted that among the element-level BMDS models, those for Terminal
                           High Altitude Area Defense (THAAD) and Aegis Ballistic Missile Defense
                           (Aegis BMD) are farthest along developmentally. In an April 2011
                           accreditation report, independent assessors from the Johns Hopkins
                           University Applied Physics Laboratory found improvements in five of six
                           functional areas for a key THAAD modeling tool, noting that available
                           data permitted accreditation for three areas. The report also noted
                           progress with two key Aegis BMD models, each of which was assessed
                           for limited accreditation in two of four BMDS target negation areas. MDA
                           officials have also noted significant progress in the development of a key
                           model for the Command, Control, Battle Management, and
                           Communications element of the BMDS. MDA has made some progress
                           toward accreditation of BMDS element models for specific functional
                           areas, but MDA officials acknowledged that the agency has not yet
                           achieved OTA accreditation in other key areas, such as any of the 18
                           environmental models.

                           While MDA has progressed in its use of simulated BMDS assessments,
                           there are risks inherent in collecting information from unaccredited
                           sources. Currently, both of the BMDS modeling and simulation
                           frameworks rely on currently unaccredited models, despite the
                           improvements that MDA has noted in the results of such assessments.
                           OTA officials expressed lowered confidence in the data collected from


                           Page 44                                              GAO-12-486 Missile Defense
Appendix III: BMDS Models and Simulations
Progress




such simulated assessments. The reliance on unaccredited models could
result in poorly crafted tactics, techniques, and procedures and in the
production and fielding of a system that is not able to actually counter
real-world threats. As the BMDS matures and the number of fielded
assets increases, modeling and simulation capabilities and laboratory
representations of BMDS assets must keep pace to maintain operational
realism.




Page 45                                           GAO-12-486 Missile Defense
Appendix IV: Aegis Ballistic Missile Defense
                                            Appendix IV: Aegis Ballistic Missile Defense
                                            (Aegis BMD) with Standard Missile-3 (SM-3)
                                            Block IA and Block IB


(Aegis BMD) with Standard Missile-3 (SM-3)
Block IA and Block IB

Fiscal year 2011 events                                            Overview

•   In September 2011, the SM-3 Block IB failed during its first      Aegis BMD achieved some significant accomplishments in
    developmental flight test.                                         fiscal year 2011.
•   The planned 2011 SM-3 Block IB production decision was            SM-3 Block IB’s concurrent schedule overlaps development
    delayed to fiscal year 2013.                                       and production.
•   In April 2011, the SM-3 Block IA successfully intercepted an      Failure in SM-3 Block IB’s first flight test led to cost growth and
    intermediate-range missile. During the test, the missile           schedule delays.
    experienced an anomaly. The anomaly occurred in a                 The production transition to the SM-3 Block IB from the SM-3
    component also used in the SM-3 IB.                                Block IA has been repeatedly disrupted.
•   Deliveries of both the SM-3 Block IA and the SM-3 Block IB        SM-3 Block IA production and deliveries are on hold while the
    are on hold until separate failure review boards are               April 2011 flight test anomaly is being investigated.
    completed.
•   The program supported the deployment of the first ship for
    the European Phased Adaptive Approach Phase I.




Background and Overview                     Aegis BMD with the SM-3 Blocks IA and IB is a ship-based missile
                                            defense system designed to intercept short- to intermediate-range
                                            ballistic missiles during the midcourse phase of their flight. Key
                                            components include the Aegis Weapons System, shipboard SPY-1 radar,
                                            battle management and command and control systems, and SM-3
                                            missiles. Missile Defense Agency (MDA) continues to develop Aegis BMD
                                            in spirals for the weapon system and successive capability-based variants
                                            of the SM-3 interceptor to improve defense against increased threat
                                            missile range, type, and raid size. The SM-3 missile has multiple versions
                                            in development or production: the SM-3 Blocks IA, IB, IIA, and IIB. The
                                            currently deployed system is Aegis BMD 3.6.1 with SM-3 Block IA, which
                                            is designed to hit short- to medium-range threat missiles. This system is
                                            included in Phase I of the European Phased Adaptive Approach (PAA).
                                            The next generation version is Aegis BMD 4.0.1 with SM-3 Block IB,
                                            which has greater capabilities. The SM-3 Block IB uses many of the same
                                            components as the SM-3 Block IA, but features an improved two-color
                                            target seeker capability for increased onboard discrimination, an
                                            advanced signal processor for engagement coordination, an improved
                                            throttleable divert and attitude control system (TDACS) for adjusting its
                                            course, and increased range and raid capabilities. The SM-3 Block IB
                                            with Aegis BMD 4.0.1 is planned to be deployed as part of European PAA
                                            Phase II in the 2015 time frame. The SM-3 Blocks IIA and IIB interceptors
                                            are discussed in appendixes V and VI.




                                            Page 46                                                          GAO-12-486 Missile Defense
                            Appendix IV: Aegis Ballistic Missile Defense
                            (Aegis BMD) with Standard Missile-3 (SM-3)
                            Block IA and Block IB




Aegis BMD Achieved Some     Aegis BMD made several significant accomplishments in fiscal year 2011.
Significant                 The Aegis BMD 4.0.1/SM-3 Block IB program successfully conducted
Accomplishments in Fiscal   simulated flight test FTM-16 E1 in March 2011, delivered the SM-3 Block
                            IB pathfinder round to hold FTM-16 E2, and gained sufficient data in
Year 2011                   FTM-16 E2 in September 2011 to support certification of the Aegis BMD
                            4.0.1 weapon system, planned in the second quarter of fiscal year 2012.

                            As for the Block IA interceptor, DOD fielded the Aegis BMD 3.6.1/SM-3
                            Block IA-equipped ship, U.S.S. Monterey, for Phase I of the European
                            PAA in April 2011, meeting the 2011 time frame for deployment. During
                            the fiscal year, MDA also installed one Aegis BMD 3.6.1 weapon system
                            on a ship. In addition, the Aegis BMD program conducted a successful
                            flight test of the Aegis BMD SM-3 Block IA, referred to as FTM-15,
                            despite experiencing an anomaly during the test. The Aegis BMD SM-3
                            Block IA was also used in a Japanese flight test—JFTM-4—in which two
                            U.S. Aegis BMD ships cooperated to detect, track, and conduct a
                            simulated intercept engagement against the same target.

                            Overall, the Aegis BMD 3.6.1/SM-3 Block IA program has had eight out of
                            nine successful flight tests. In addition, Japanese Aegis BMD has
                            conducted three out of four successful intercepts using SM-3 Block IA
                            interceptors. 1 The Aegis BMD 3.6.1 weapon system was the first MDA
                            element to be assessed as operationally effective and suitable for combat
                            by independent test officials, with limitations.


SM-3 Block IB’s             Problems with concurrency are affecting the production of SM-3 Block IB
Concurrent Schedule         interceptors and delaying the phaseout of the SM-3 Block IA production.
Overlaps Development and    The acquisition plan for the SM-3 Block IB interceptor includes high levels
                            of concurrency—buying weapon systems before they demonstrate,
Production                  through testing, that they perform as required—between development
                            and production. Specifically, the program purchased interceptors before
                            confirming that the design works as intended by completing
                            developmental tests and prior to ensuring that a key subcomponent has
                            overcome prior developmental problems. The need to field the Aegis
                            BMD 4.0.1/SM-3 Block IB by the 2015 time frame for European PAA
                            Phase II announced by the President is a key driver for the high levels of


                            1
                              Japanese flight tests help MDA officials understand performance, but because they are
                            not U.S. assets, they are not considered developmental flight tests from a programmatic
                            perspective.




                            Page 47                                                      GAO-12-486 Missile Defense
Appendix IV: Aegis Ballistic Missile Defense
(Aegis BMD) with Standard Missile-3 (SM-3)
Block IA and Block IB




concurrency. According to MDA, the program is purchasing interceptors
for a variety of reasons, including in support of developmental and
operational testing, proving the manufacturing process, and ensuring the
missile will meet its performance requirements on a repeatable basis. See
figure 8 for a depiction of the SM-3 Block IB’s concurrent schedule.

Figure 8: SM-3 Block IB Schedule




The SM-3 Block IB’s acquisition plan includes high levels of concurrency.
We reported in February 2010 that planned interceptor production would
precede knowledge of interceptor performance, and recommended that
MDA delay a decision to produce interceptors to follow successful
completion of developmental testing, a flight test, and manufacturing
readiness review. 2 In March 2010, we reported that the Aegis BMD
program is putting the SM-3 Block IB at risk for cost growth and schedule
delays by planning to begin manufacturing in 2010 before its critical
technologies have been demonstrated in a realistic environment. 3 We
also reported in December 2010 that the SM-3 Block IB test schedule
was not synchronized with planned production and financial
commitments. 4 Finally, in March 2011, we reported that the schedule had
become even more compressed due to the redesign and requalification of


2
    GAO-10-311.
3
  GAO, Defense Acquisitions: Assessments of Selected Weapon Programs,
GAO-10-388SP (Washington, D.C.: Mar. 30, 2010).
4
 GAO, Missile Defense: European Phased Adaptive Approach Acquisitions Face
Synchronization, Transparency, and Accountability Challenges, GAO-11-179R
(Washington, D.C.: Dec. 21, 2010).




Page 48                                                  GAO-12-486 Missile Defense
Appendix IV: Aegis Ballistic Missile Defense
(Aegis BMD) with Standard Missile-3 (SM-3)
Block IA and Block IB




a missile component, and in response, MDA deferred key program
milestones so that it would have better informed production decisions. 5

The program began production of SM-3 IB interceptors before resolving
development issues with the TDACS, a key interceptor component that
maneuvers the kill vehicle during the later stages of flight. The TDACS
failed qualification testing in early 2010 and required a redesigned
propellant moisture protection system. In order to hold the first SM-3
Block IB developmental flight test, FTM-16 Event 2, in September 2011
as scheduled, MDA only partially completed TDACS qualification testing
and the version used in the failed flight test was not identical with the
approved production design. The TDACS is expected to complete
qualification testing in 2012; however, any additional issues discovered
during qualification testing or developmental flight testing may require
additional redesigns.

The commitment to produce SM-3 Block IB interceptors beyond those
needed for developmental testing was made before the program had a
sufficient level of knowledge about the missile’s technology maturity and
performance. MDA has determined that 18 of the 25 SM-3 Block IB
missiles ordered are to be used for developmental testing. The remaining
7 interceptors are currently unassigned for tests and may be available for
operational use. 6

According to MDA, these interceptors will be used to support
developmental and operational testing; to prove out the manufacturing
processes; to provide information about reliability, maintainability, and
supportability; to verify and refine cost estimates; and to ensure that the
missile meets performance requirements. MDA officials acknowledged
that missiles not consumed by testing could be used operationally.
Program management officials stated that the unassigned missiles
represent a very small portion of the total number of interceptors they



5
    GAO-11-372.
6
  Six of the 18 missiles will not be needed for developmental tests until fiscal year 2015 or
later. SM-3 Block IB missiles are used for developmental tests of the SM-3 Block IB and
Aegis Weapons System 4.0.1 program and are planned for use in developmental tests for
Aegis Ashore and Aegis Weapons System 5.0. MDA originally planned to purchase 34
SM-3 Block IB interceptors; however, to address developmental issues with the TDACS in
the SM-3 Block IB, in 2011, MDA reduced the planned quantity on order to 25 SM-3 Block
IB interceptors. One interceptor was used in FTM-16 E2, leaving 24 interceptors.




Page 49                                                         GAO-12-486 Missile Defense
                             Appendix IV: Aegis Ballistic Missile Defense
                             (Aegis BMD) with Standard Missile-3 (SM-3)
                             Block IA and Block IB




                             plan to purchase, representing less than 5 percent of the total 472
                             interceptors that the program plans to purchase through fiscal year 2020.
                             MDA decided that the risk was low given that many of the SM-3 Block IB
                             critical technologies were based on critical technologies that were tested
                             and used successfully by the SM-3 Block IA.

                             MDA is also planning to purchase 46 additional SM-3 Block IB missiles in
                             fiscal year 2012. However, there are two failure investigations ongoing
                             that affect SM-3 Block IB production that could delay three planned
                             developmental flight tests that need to occur to validate SM-3 Block IB
                             capability. It therefore remains unclear whether the additional 46 missiles
                             will be ordered before the failure reviews are complete and the interceptor
                             is able to demonstrate that it works as intended through these flight tests.

                             The program’s highly concurrent schedule is shaped primarily by the
                             need to achieve initial capability for the fielding of Phase II of the
                             European PAA by the 2015 time frame announced by the President. In
                             addition, the program must be ready to participate in the second BMDS
                             operational test in 2015. Program officials report that they are on track to
                             achieve these time frames. However, until development is complete, any
                             additional issues could lead to additional cost growth or schedule delays.


Failure in SM-3 Block IB’s   The SM-3 Block IB failed its first developmental flight test, FTM-16 E2,
First Flight Test Led to     leading to cost growth and schedule delays compounded by the
Cost Growth and Schedule     disruption to ongoing production, the full extent of which has yet to be
                             determined. During the flight test, the SM-3 Block IB experienced an
Delays                       unexpected energetic event in the third-stage rocket motor and failed to
                             intercept a short-range ballistic missile target. Following the flight test, the
                             program convened a failure review board to determine the root cause of
                             the failure, modified the missile production contract, restructured the flight
                             test program, and delayed key production decisions. While the failure
                             review board is still investigating the flight test, MDA slowed production of
                             SM-3 Block IB interceptors. The program had planned to deliver an
                             additional three SM-3 Block IB missiles for flight testing in fiscal year
                             2011. However, the delivery of the remaining three has been delayed until
                             spring 2012.

                             Program officials estimate that the flight test failure—including the failure
                             investigation, design modifications, testing, and requalification for return
                             to flight—may cost approximately $187 million in fiscal year 2012. In
                             addition, because officials are still investigating the cause of the flight test
                             failure and how many already-produced missiles may have to be


                             Page 50                                                 GAO-12-486 Missile Defense
                             Appendix IV: Aegis Ballistic Missile Defense
                             (Aegis BMD) with Standard Missile-3 (SM-3)
                             Block IA and Block IB




                             retrofitted, they do not yet know how much the retrofits, if required, will
                             cost. At this point, the program does not have an approved plan to avoid
                             an SM-3 production gap.

                             The flight test failure also had several other consequences. The SM-3
                             Block IB manufacturing readiness review has been delayed from the
                             second quarter of fiscal year 2011 to the third quarter of fiscal year 2012,
                             and the procurement production decision for additional SM-3 Block IB
                             missiles was moved from fourth quarter of fiscal year 2011 to the fourth
                             quarter of fiscal year 2013. The failed flight test will be re-conducted in
                             mid-2012, which may delay additional developmental flight testing.


The Transition to the SM-3   Aegis BMD’s transition to the SM-3 Block IB has been repeatedly
Block IB from the SM-3       disrupted because the transition was risky given the technology maturity
Block IA Has Been            of components developed for the SM-3 Block IB and the program’s
                             concurrent schedule. Originally, MDA planned that production of SM-3
Repeatedly Disrupted         Block IA interceptors would end in fiscal year 2009 as production of SM-3
                             Block IB interceptors began. However, due to developmental issues with
                             the SM-3 Block IB, MDA twice had to extend SM-3 Block IA production—
                             in 2010 and 2011—to cover emerging production gaps with the SM-3
                             Block IB. To date, MDA has contracted for 41 more SM-3 Block IA
                             missiles than originally planned in order to bridge the production gaps.
                             Now, following the September 2011 flight test failure, MDA is facing
                             another production gap. It is extending production once again—it
                             purchased 23 SM-3 Block IA missiles in fiscal year 2011 and is
                             considering whether to purchase additional SM-3 Block IA missiles in
                             fiscal year 2012.

                             In addition, the program has twice had to adjust the procurement of SM-3
                             Block IB missiles. Instead of purchasing 24 SM-3 Block IB missiles as
                             planned in 2010, it purchased 18 SM-3 Block IA missiles and it did not
                             procure 8 SM-3 Block IB missiles in 2011 as planned. To free up funding
                             needed to improve TDACS operational suitability, MDA reduced the
                             planned SM-3 Block IB missiles from 34 to 25 in fiscal year 2011. Thus
                             far, the program has purchased 41 fewer missiles than previously
                             planned. Due to the FTM-16 E2 developmental flight test failure, delivery
                             of these SM-3 Block IB missiles is now being slowed until the failure
                             review board completes its investigation and any possible retrofits are
                             made. Despite the test failure and delivery hold, MDA is considering
                             purchasing 46 SM-3 Block IB interceptors in fiscal year 2012 and 29 SM-
                             3 Block IB interceptors in fiscal year 2013. Recognizing the critical
                             importance of the completing the planned fiscal year 2012 intercept tests,


                             Page 51                                              GAO-12-486 Missile Defense
                           Appendix IV: Aegis Ballistic Missile Defense
                           (Aegis BMD) with Standard Missile-3 (SM-3)
                           Block IA and Block IB




                           the operational need for SM-3 missiles, the relative success of the SM-3
                           Block IA, as well as the potential for a production break, the Senate
                           Committee on Appropriations directed MDA to use the fiscal year 2012
                           SM-3 Block IB funds for additional Block IA missiles should the test and
                           acquisition schedule require any adjustments during fiscal year 2012.


SM-3 Block IA Deliveries   As a result of an anomaly in the latest SM-3 Block IA flight test—FTM-15
Are on Hold While the      in April 2011—MDA halted acceptance of SM-3 Block IA deliveries.
April 2011 Flight Test     During the April 2011 flight test, MDA demonstrated the Aegis BMD 3.6.1
                           weapon system’s ability to launch the SM-3 Block IA interceptor using
Anomaly Is Being           data from a remote sensor against a separating intermediate-range
Investigated               ballistic missile target and the capability of the interceptor to engage
                           threat missiles in the range expected for Phase I of the European PAA.
                           However, although the SM-3 Block IA interceptor intercepted the target, it
                           experienced an anomaly. The anomaly occurred in a component also
                           used in the SM-3 Block IB.

                           At the time of our review, the program had not completed its investigation
                           into the cause of the anomaly or decided how it will address the issue.
                           The program convened a failure review board, which has not yet
                           completed its investigation of the root cause of the anomaly. Twelve
                           assembled SM-3 Block IA missiles are not being accepted for delivery
                           and are being held at the production factory until the investigation of the
                           anomaly is complete and any possible refurbishments are made. This
                           represents about 10 percent of the population of SM-3 Block IA missiles.
                           Program management officials report that thus far seven missiles will
                           need to be refurbished. Because the failure review board has not yet
                           completed its investigation, an unknown quantity of additional SM-3 Block
                           IA missiles may need to be refurbished due to the anomaly. At the time of
                           our review, the program did not have an approved plan for how it will
                           refurbish the affected missiles. Despite these issues, MDA purchased 23
                           SM-3 Block IA missiles in September 2011 and is considering whether to
                           purchase additional missiles in 2012 to avoid production gaps and to
                           keep SM-3 suppliers active.




                           Page 52                                             GAO-12-486 Missile Defense
Appendix V: Aegis Ballistic Missile Defense
                                         Appendix V: Aegis Ballistic Missile Defense
                                         (Aegis BMD) Standard Missile-3 (SM-3) Block
                                         IIA


(Aegis BMD) Standard Missile-3 (SM-3) Block
IIA

Fiscal year 2011 events                                         Overview

•   The program discovered problems with four key components        The SM-3 Block IIA program began in 2006 as a cooperative
    during subsystem preliminary design reviews (PDR) and            development with Japan. It is required to be fielded by 2018 as
    held reviews to resolve the issues with two components in        part of the European Phased Adaptive Approach Phase III.
    fiscal year 2011.                                               Design review problems in fiscal year 2011 led to a program
•   The program was restructured in response to subsystem            restructure and likely increased current costs.
    PDR problems, adjusting planned flight tests.                   Program actions in fiscal year 2011 reduced acquisition risk
•   Preliminary testing of some U.S. and Japanese components         and potential future cost growth.
    began.                                                          Despite positive changes in program schedule, technology
                                                                     development concerns remain.




Background and Overview                  The SM-3 Block IIA is the third SM-3 version to be developed for use with
                                         the sea-based and future land-based Aegis BMD. This interceptor is
                                         planned to have increased velocity and range compared to earlier SM-3s
                                         due to a larger 21-inch diameter, more sensitive seeker technology, and
                                         an advanced kinetic warhead. 1 Most of the SM-3 Block IIA components
                                         will differ from the versions used in the SM-3 Block IB, so technology has
                                         to be developed for the majority of the SM-3 IIA components. The SM-3
                                         Block IIA is expected to defend against short-, medium-, and
                                         intermediate-range ballistic missiles.

                                         Initiated in 2006 as a cooperative development program with Japan, the
                                         SM-3 Block IIA program was added to the European Phased Adaptive
                                         Approach (PAA) in 2009. As part of European PAA Phase III, the SM-3
                                         Block IIA is planned to be fielded with Aegis Weapons System 5.1 by the
                                         2018 time frame and is expected to provide engage on remote capability,
                                         in which data from off-board sensors is used to engage a target, and
                                         expand the range available to intercept a ballistic missile. The program is
                                         managing both the development of the SM-3 Block IIA and its integration
                                         with Aegis Weapons System 5.1, which also is still under development. In
                                         this appendix, we evaluate only the SM-3 Block IIA.




                                         1
                                           A kinetic warhead is a “hit-to-kill” warhead that collides with a ballistic missile’s warhead
                                         to destroy it.




                                         Page 53                                                           GAO-12-486 Missile Defense
                               Appendix V: Aegis Ballistic Missile Defense
                               (Aegis BMD) Standard Missile-3 (SM-3) Block
                               IIA




Design Review Problems         The program planned to hold its system preliminary design review
in Fiscal Year 2011 Led to a   (PDR)—at which it would demonstrate that the technologies and
Program Restructuring and      resources available for the SM-3 Block IIA would result in a product that
                               matched its requirements—but problems with the reviews of key
Likely Increased Current       components meant the system review had to be adjusted by 1 year. To
Costs                          prepare for the system review, the program held 60 subsystem reviews,
                               for its components to ensure that they were feasible given the technology
                               and resources available. Two components—divert and attitude control
                               system (DACS) and DACS propellant—failed their subsystem reviews
                               and two components—nosecone and third stage rocket motor (TSRM)—
                               had their reviews suspended, indicating that the technological capability
                               of these critical components and SM-3 Block IIA requirements were
                               mismatched. The program took steps to resolve each of the four
                               subsystem review problems, including restructuring the program to
                               reduce future acquisition risk.

                               The DACS, used to adjust the course of the kinetic warhead, failed its
                               subsystem review because it was not meeting weight and divert
                               acceleration requirements, which the program resolved by reviewing and
                               rebalancing subsystem requirements. The system-level DACS
                               requirements did not change. The DACS propellant that failed the
                               subsystem review was susceptible to a moisture problem, and the
                               program selected a different propellant. 2 The nosecone, which encloses
                               the kinetic warhead, was overweight and could become more so, and the
                               mitigation plan for the weight issue was insufficient. To resolve these
                               issues, the program evaluated weight reduction opportunities and risks.
                               The TSRM, used to lift the missile out of the atmosphere and direct the
                               kinetic warhead to the target, was also not meeting weight requirements,
                               and one of its components, the attitude control system, was not meeting
                               thrust accuracy and alignment requirements. To resolve this issue, the
                               program rebalanced subsystem requirements, but did not change system-
                               level TSRM requirements. 3




                               2
                                 The SM-3 Block IB propellant was affected by the moisture issue, which was caused by
                               a component shared with the initial SM-3 Block IIA propellant.
                               3
                                 The reviews to close out the nosecone and TSRM subsystem PDRs failures occurred in
                               fiscal year 2011. The closeout review for the DACS and DACS propellant occurred early in
                               fiscal year 2012.




                               Page 54                                                      GAO-12-486 Missile Defense
                            Appendix V: Aegis Ballistic Missile Defense
                            (Aegis BMD) Standard Missile-3 (SM-3) Block
                            IIA




                            The subsystem review issues also required program schedule changes,
                            which included the following:

                            •   Adjusting the system PDR from January 2011 to March 2012.
                            •   Splitting in two the critical design review (CDR), at which the program
                                will determine that the product’s design matches the SM-3 Block IIA
                                requirements and cost, schedule, and reliability goals. This led to
                                schedule adjustments of 13 and 19 months, respectively, for each of
                                the CDRs.
                            •   Adjusting the interceptor flight test schedule. The program previously
                                planned to hold its first intercept tests in fiscal years 2014 and 2015
                                as part of the co-development with Japan, but with the schedule
                                adjustment, it will now have these tests in calendar year 2016.The
                                United States and Japan finalized the development program
                                restructuring on September 30, 2011. Despite this adjustment, the
                                interceptor remains aligned with European PAA Phase III in the 2018
                                time frame.

                            Aegis BMD program management officials stated that the subsystem
                            PDR problems and subsequent program restructure may increase current
                            program costs, but they are not certain how much because the
                            completion contract, which will run through fiscal year 2017, was still
                            being negotiated as of December 2011.


Program Actions in Fiscal   The SM-3 Block IIA program took actions in 2011 that could reduce
Year 2011 Reduced           acquisition risk and mitigate future cost growth. Its previous schedule was
Acquisition Risk and        compressed, which raised acquisition risk. For example, there was limited
                            recovery time to investigate and resolve potential problems between
Potential Future Cost       program reviews as well as flight tests. The new schedule, made final in
Growth                      September 2011, relieves some compression concerns and adjusts to the
                            subsystem review issues by adding time between the subsystem reviews
                            and the system review to ensure that the technology issues are resolved.
                            We have previously reported that reconciling gaps between requirements
                            and resources before product development begins makes it more likely
                            that a program will meet cost, scheduling, and performance targets, and
                            programs that commit to product development with less technical
                            knowledge and without ensuring that requirements are defined, feasible,
                            and achievable within cost, schedule, and other system constraints face




                            Page 55                                             GAO-12-486 Missile Defense
                           Appendix V: Aegis Ballistic Missile Defense
                           (Aegis BMD) Standard Missile-3 (SM-3) Block
                           IIA




                           increased technical risks and possibility of cost growth. 4 The new SM-3
                           Block IIA schedule allows the program to have more knowledge before
                           committing to product development in the second quarter of fiscal year
                           2014, a strategy that may reduce future cost growth and development
                           risks. The new schedule also adds flexibility in the test schedule by
                           adding an option for a third controlled test vehicle flight if needed. If the
                           first two test vehicles prove to be successful and a third is not needed,
                           this test can be converted into the first intercept test of the SM-3 Block
                           IIA.

                           In addition to the schedule change, in fiscal year 2011 the program
                           identified some steps to avoid the difficulties that affected SM-3 Block IB
                           component production. For example, it found that using proven materials,
                           standardizing inspections with vendors, and ensuring that designs
                           included reasonable tolerances were practices to follow based on lessons
                           learned from the SM-3 Block IB experience.

                           Finally, the SM-3 Block IIA program identified alternatives to one
                           advanced seeker component that it had identified, based on the
                           experience of the SM-3 Block IB, as potentially increasing production unit
                           costs by 5 percent. Program management officials stated that they
                           identified a viable alternative for this component and worked with the SM-
                           3 Block IIB program to further develop manufacturing improvements for
                           this technology.


Despite Positive Changes   The program still faces significant technology development challenges.
in Overall Program         While the SM-3 Block IIA is a variant of the SM-3 missile, the majority of
Schedule, Technology       its components will change from their SM-3 Block IB configuration. The
                           program must develop these components, some of which have
Development Concerns       consistently been technologically challenging for SM-3 development. In
Remain                     addition, two technology maturity challenges have emerged. Two critical
                           technologies, the second and third stage rocket motors, experienced
                           problems during testing that may require redesign and a potential CDR
                           rescheduling. The program was investigating the problems and potential
                           effects at the end of fiscal year 2011. In addition, following the subsystem


                           4
                            GAO, Best Practices: Using a Knowledge-Based Approach to Improve Weapon
                           Acquisition, GAO-04-386SP (Washington, D.C.: January 2004), and Defense Acquisitions:
                           Assessments of Selected Weapons Programs, GAO-11-233SP (Washington, D.C.:
                           Mar. 29, 2011).




                           Page 56                                                    GAO-12-486 Missile Defense
Appendix V: Aegis Ballistic Missile Defense
(Aegis BMD) Standard Missile-3 (SM-3) Block
IIA




review failure and selection of an alternate propellant, analysis of the
DACS propellant performance showed that there may be a shortfall in
divert performance for some missions. As of the end of fiscal year 2011,
the program was still determining the extent of this issue.




Page 57                                            GAO-12-486 Missile Defense
Appendix VI: Aegis Ballistic Missile Defense
                                           Appendix VI: Aegis Ballistic Missile Defense
                                           (Aegis BMD) Standard Missile-3 (SM-3) Block
                                           IIB


(Aegis BMD) Standard Missile-3 (SM-3) Block
IIB

Fiscal year 2011 events                                      Overview

•   The program entered the technology development              SM-3 Block IIB program began in June 2010 and is planned to be
    phase and awarded initial contracts for concept              fielded by the 2020 time frame as part of European PAA Phase IV.
    definition and technology risk reduction.                   The program awarded three concept definition and program
•   Inclusion of additional technology development efforts       planning contracts to develop schedule and design options.
    to support European Phased Adaptive Approach (PAA)          The current program plan includes high levels of concurrency and
    Phase IV led to realignment of the resource plan.            acquisition risk.
•   The program continued development of technologies           Full program acquisition costs have not been developed given the
    that may contribute to SM-3 interceptor variants.            early stage of the program.
•   The program prepared for transfer of SM-3 Block IIB         The benefits of early intercept capability are unclear and the lack of
    development to Aegis BMD program office in 2013.             analysis of alternatives may result in warfighter needs not being met
                                                                 within resource constraints.




Background and Overview                    The SM-3 Block IIB is a planned interceptor for the Aegis BMD program
                                           that is intended to contribute to U.S. homeland defense by providing early
                                           intercept capabilities against some intercontinental ballistic missiles and
                                           regional defense against medium- and intermediate-range ballistic
                                           missiles. This interceptor has been described by the Missile Defense
                                           Agency (MDA) as critical to the Ballistic Missile Defense System (BMDS)
                                           and developing solutions to future BMDS capability shortfalls. The SM-3
                                           Block IIB program began in June 2010 and entered the technology
                                           development phase in July 2011. Given its early stage of development,
                                           the SM-3 Block IIB does not have cost, schedule or performance
                                           baselines and is not managed within the Aegis BMD program office.
                                           Instead, this program has a tentative schedule and is being managed
                                           within MDA’s Advanced Technology office until a planned 2013 transition
                                           to the Aegis BMD program office. The SM-3 Block IIB is planned to be
                                           fielded by the 2020 time frame as part of the European Phased Adaptive
                                           Approach Phase IV.

                                           The program received a significant funding reduction in the fiscal year
                                           2012 budget and, as of January 2012, was determining how to adjust its
                                           tentative schedule and future program plans. The program’s fiscal year
                                           2012 budget request was reduced by $110 million to $13 million.




                                           Page 58                                                        GAO-12-486 Missile Defense
                            Appendix VI: Aegis Ballistic Missile Defense
                            (Aegis BMD) Standard Missile-3 (SM-3) Block
                            IIB




Program Awarded Initial     The SM-3 Block IIB program is following a two-pronged development
Contracts to Develop        strategy. First, program officials have awarded competitive contracts to
Schedule, Design Options,   generate options for missile configurations and development plans.
                            Second, in a separate effort, they are using multiple contractors to reduce
and Crosscutting            risks by developing technologies that may be used in the SM-3 Block IIB
Technology                  and other SM-3 variants. The program awarded three concept definition
                            and program planning contracts to define and assess viable missile
                            configurations, conduct trade studies, and define a development plan.
                            The contractors will develop alternative missile concepts, technologies
                            and schedule for interceptor development beyond 2013. According to the
                            program, the purpose of this competition is to minimize cost, schedule,
                            and technical risks. There will be another competition to select one
                            contractor for the product development phase in 2013. We have reported
                            previously that competition among contractors can result in increased
                            technological innovation that leads to better and more reliable products. 1

                            The program is using technology risk reduction contracts to develop
                            technologies that may cut across versions of the SM-3, such as the focal
                            plane array, 2 and to invest in materials or technology that will increase
                            missile velocity and containment of threat missiles. For example, this
                            effort produced a major technical first when a contractor working on focal
                            plane array issues changed the process for creating a component of the
                            focal plane array in a way that may reduce the number of defects in the
                            production of that component.


Current Program Plan        Program management officials have issued a tentative schedule beyond
Includes High Levels of     the technology development phase, but this plan, if implemented,
Concurrency and             includes high levels of concurrency and acquisition risk. We have
                            previously reported the following:
Acquisition Risk
                            •     Concurrency leads to major problems being discovered in production,
                                  when it is either too late or very costly to correct them. 3



                            1
                              GAO, Joint Strike Fighter: Assessment of DOD’s Funding Projection for the F136
                            Alternate Engine, GAO-10-1020R (Washington, D.C.: Sept. 15, 2010).
                            2
                                The focal plane array is a component of the seeker.
                            3
                             GAO, High-Risk Series: Defense Weapons Systems Acquisition, GAO/HR-93-7
                            (Washington, D.C.: December 1992).




                            Page 59                                                     GAO-12-486 Missile Defense
Appendix VI: Aegis Ballistic Missile Defense
(Aegis BMD) Standard Missile-3 (SM-3) Block
IIB




•   Before starting product development, programs should hold key
    engineering reviews, culminating in the preliminary design review
    (PDR), to ensure that the proposed design can meet defined, feasible
    requirements within cost, schedule, and other system constraints. 4

•   Committing to production and fielding before development is complete
    is a high-risk strategy that often results in unexpected cost increases,
    schedule delays, test problems, and performance shortfalls. 5

Successful defense programs ensure that their acquisitions begin with
realistic plans and baselines before the start of their development. 6
According to the tentative SM-3 Block IIB schedule, the product
development decision will occur before the March 2015 PDR. As a result,
MDA is planning to commit to developing a product with less technical
knowledge than our prior work has shown is needed and without fully
ensuring that requirements are defined, feasible, and achievable within
cost, schedule, and other system constraints. This sequencing increases
both technical risks and the possibility of cost growth. 7 In addition, the
program will not have a stable design when it must commit to building
flight test vehicles. According to acquisition best practices, a design is
considered stable when the technologies are mature and the critical
design review (CDR) confirms that at least 90 percent of the drawings are
releasable for manufacturing. Based on the experience of other SM-3
interceptors, the program must commit to produce flight test interceptors
2 years before the March 2016 first flight. However, this timeline means
the commitment to a flight test vehicle would occur a year before the SM-
3 Block IIB PDR has confirmed that the design is feasible and more than
a year and a half before CDR has confirmed that the design is stable. See
figure 9 for a depiction of the tentative SM-3 Block IIB schedule.



4
 GAO, Defense Acquisitions: Assessments of Selected Weapons Programs,
GAO-11-233SP (Washington, D.C.: Mar. 29, 2011).
5
 GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early Improves
Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002).
6
 GAO, Defense Acquisitions: Missile Defense Transition Provides Opportunity to
Strengthen Acquisition Approach, GAO-10-311 (Washington, D.C.: Feb. 25, 2010).
7
  Given the funding reduction in fiscal year 2012, the program plans to delay the product
development decision by several months but was still determining in January 2012 what
other modifications to the tentative schedule would take place. We reviewed the tentative
schedule issued prior to the funding reduction.




Page 60                                                       GAO-12-486 Missile Defense
Appendix VI: Aegis Ballistic Missile Defense
(Aegis BMD) Standard Missile-3 (SM-3) Block
IIB




Figure 9: SM-3 Block IIB Schedule




Note: Given the early stage of the program, we are not able to depict the production plans or the end
of the product development phase for the SM-3 Block IIB.


Program management officials stated that they have taken steps in the
tentative schedule that reduce acquisition risk. According to SM-3 Block
IIB program information, the tentative schedule is based on the
experience of programs with similar magnitude and complexity, and the
concept definition and program planning contractors will develop detailed
product development schedules that will help refine the program
schedule. Further, activities during the technology development phase,
such as evaluating the performance of multiple contractor concepts,
simulations conducted by the contractors, and affordability assessments,
are designed to reduce risk in SM-3 Block IIB development. In addition,
while the program plans to hold its production development decision prior
to the PDR, it will hold a series of reviews with the concept definition
contractors to receive engineering insight into each contractor’s plans.
Program management officials told us they also plan to hold a
government-only system requirements review prior to the initiating the
product development contract competition. This review is planned to
confirm that SM-3 Block IIB has specific technical requirements that the
developer can use to establish a product baseline as well as conduct a
risk and technology readiness assessment.

Another key step for successful programs is ensuring that only mature
technologies are brought into product development. MDA has identified
technologies that are important for SM-3 variants and is investing in these
technologies, particularly the less mature technologies, to facilitate SM-3
Block IIB development. However, as of October 2011, the program had
not named specific critical technologies for the SM-3 Block IIB. Program
officials stated that they do not plan to do so until the product
development decision. The concept definition contractors are required to



Page 61                                                               GAO-12-486 Missile Defense
                           Appendix VI: Aegis Ballistic Missile Defense
                           (Aegis BMD) Standard Missile-3 (SM-3) Block
                           IIB




                           identify technology investments to increase the maturity of the
                           technologies by demonstrating them in a relevant environment by the end
                           of fiscal year 2013, which coincides with the product development
                           decision. MDA, however, does not require that a program mature
                           technologies to this level by this decision. Without knowing the specific
                           critical technologies, it is not possible to identify the risk of including them
                           in the product development phase. As we have previously reported,
                           including immature technologies in product development can lead to
                           delays and contribute to cost increases. 8


Full Program Acquisition   While the program has proposed that $1.673 billion in research and
Costs Not Developed        development funding is needed from fiscal years 2012 to 2016, a full
Given Early Stage of the   program acquisition cost has not yet been developed. Given the early
                           stage of the program, and that key decisions about requirements and the
Program                    missile configuration have not been made, a full acquisition cost estimate
                           is not currently feasible. According to MDA, the program plans to
                           complete a detailed cost estimate prior to entering product development.

                           A cost estimate cannot be developed until key acquisition decisions are
                           made. Program management officials stated that warfighter and system
                           requirements for the SM-3 Block IIB have not been set, and discussions
                           about the delivery schedule beyond the initial capability are ongoing.
                           Further, whether the propellant will be liquid or solid, the SM-3 Block IIB’s
                           diameter, and whether modifications must be made to a vertical launch
                           system are not yet known given the early stage of the program. In
                           addition, as there is not yet a final schedule, the currently proposed
                           funding is not informed by a complete post-product development decision
                           schedule. Program management officials note that these key decisions
                           are being informed by activities occurring during the technology
                           development phase, such as trade studies involving the propulsion and
                           missile diameter, and they are updating current cost estimates as they
                           receive information from contractors as well as working on developing
                           detailed cost estimates.




                           8
                            GAO, Best Practices: Better Management of Technology Development Can Improve
                           Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999).




                           Page 62                                                 GAO-12-486 Missile Defense
                              Appendix VI: Aegis Ballistic Missile Defense
                              (Aegis BMD) Standard Missile-3 (SM-3) Block
                              IIB




Benefits of Early Intercept   MDA determined that a key goal for the SM-3 Block IIB is to provide an
Capability Are Unclear and    early intercept capability. However, a recent Defense Science Board
the Lack of Analysis of       study suggested that other capabilities are more important than early
                              intercept. The study concluded that early intercept capability is not useful
Alternatives May Result in    for regional missile defense. Further, while early intercept with shoot-look-
Warfighter Needs Not          shoot capability could be part of a cost-effective defense of the U.S.
Being Met within Resource     homeland if a sufficiently fast missile was available, the size of the
Constraints                   battlespace and not early intercept capability is the key driver of cost-
                              effectiveness. 9 In addition, it is unclear if early intercept is possible for
                              defense of the U.S. homeland due to the velocity required for an early
                              intercept of an intercontinental ballistic missile aimed at the United States
                              and the state of current missile technology. Finally, the value of a shoot-
                              look-shoot capability relies on a robust ability to determine if the first
                              missile was successful, often called kill assessment, but this ability has
                              not been established. 10 In response, MDA stated that the Defense
                              Science Board study had used a limited definition of early intercept and
                              ignored significant benefits of the strategy that stem from decreasing the
                              time available to the adversary to deploy countermeasures. Such benefits
                              include providing a longer viewing time of deployment maneuvers for
                              forward-based sensors, reducing the flight time of the interceptor, and
                              increasing the complexity to the attacker of deploying countermeasures.

                              The program office did not conduct a formal analysis of alternatives to
                              compare the operational effectiveness, cost, and risks of a number of
                              alternative potential solutions to address valid needs and shortfalls in
                              operational capability prior to embarking on the technology development
                              phase. The program did assess some missile concepts for early intercept
                              capability in a review that was not a formal analysis of alternatives. The
                              program currently plans to conduct engineering and trade studies—
                              including cost trades—that will be completed in the fourth quarter of fiscal
                              year 2012 and review additional alternative concepts as part of the
                              concept definition process. While MDA programs are not required to
                              conduct an analysis of alternatives, we have previously reported that it is
                              key to planning and establishing a sound business case. Specifically, an
                              analysis of alternatives provides a foundation for developing and refining



                              9
                               Shoot-look-shoot, also known as shoot-assess-shoot, means firing one interceptor,
                              observing the results of the initial shot, and then launching the subsequent missile(s).
                              10
                                 The Aegis BMD weapons system includes an integrated kill assessment system. We
                              did not evaluate the capabilities of this system.




                              Page 63                                                         GAO-12-486 Missile Defense
Appendix VI: Aegis Ballistic Missile Defense
(Aegis BMD) Standard Missile-3 (SM-3) Block
IIB




the operational requirements for a weapons system program and
provides insight into the technical feasibility and costs of alternatives.
Further, without a full exploration of alternatives, the program may not
achieve an optimal concept that satisfies the warfighter’s needs within
available resource constraints. 11 Without this sound basis for program
initiation, the SM-3 Block IIB is at risk for cost and schedule growth as
well as not meeting the warfighter’s needs with the resources available.




11
   GAO, Defense Acquisitions: Many Analyses of Alternatives Have Not Provided a
Robust Assessment of Weapon System Options, GAO-09-665 (Washington, D.C.:
Sept. 24, 2009).




Page 64                                                    GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore
                                            Appendix VII: Aegis Ashore




Fiscal year 2011 events                                         Overview

•   Initiated manufacturing of Aegis Ashore test site at            Concurrent development and production schedule increase
    Pacific Missile Range Facility in preparation for testing in     potential for cost growth and schedule delays.
    fiscal year 2014.                                               Various Aegis Ashore components require modification for a land-
•   Successfully completed system design review and                  based configuration and development uncertainties remain.
    preliminary design review.                                      Unstable Aegis Ashore program content, affecting both the
•   Signed agreement with Romania to host Aegis Ashore               resource baseline and cost estimates, reduces transparency and
    site as part of European Phased Adaptive Approach                impedes oversight and accountability.
    Phase II.
•   Restructured acquisition strategy for deckhouse twice.




Background and Overview                     Aegis Ashore is the Missile Defense Agency’s (MDA) planned land-based
                                            version of the ship-based Aegis Ballistic Missile Defense (Aegis BMD),
                                            which will track and intercept ballistic missiles in their midcourse phase of
                                            flight using Standard Missile-3 (SM-3) interceptors. Key components
                                            include a vertical launching system (VLS) with SM-3 missiles and a
                                            reconstitutable enclosure, referred to as a deckhouse, that contains the
                                            SPY-1 radar and command and control system. Aegis Ashore will share
                                            many components with the sea-based Aegis BMD and will use next
                                            generation versions of the Aegis weapons systems—Aegis 4.0.1 and
                                            Aegis 5.0—that are still under development. In accordance with the
                                            September 2009 European Phased Adaptive Approach (PAA)
                                            announcement, the Department of Defense (DOD) plans to deploy the
                                            first Aegis Ashore installation with the SM-3 Block IB in the 2015 time
                                            frame and the second installation in the 2018 time frame.


Concurrent Development                      Given the commitment to field Aegis Ashore by the 2015 time frame, the
and Production Schedule                     program’s schedule contains a high level of concurrency—buying weapon
Increase Potential for Cost                 systems before they demonstrate, through testing, that they perform as
                                            required—between development and production. The program began
Growth and Schedule                         product development early, included high levels of concurrency in its
Delays                                      construction and procurement plan, and has not aligned its testing
                                            schedule with component procurement and construction. As we have
                                            reported previously, an acquisition strategy for accelerated fielding, such
                                            as that of Aegis Ashore, will likely accept higher risk primarily through




                                            Page 65                                                        GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore




concurrent development and production. 1 Under such a strategy, major
problems are more likely to be discovered in production, when it is either
too late or very costly to correct them.

The program began product development and established the Aegis
Ashore cost, schedule, and performance baseline in June 2010, which
was 14 months before completing its preliminary design review. This
concurrent sequencing can increase technical risks and the possibility of
cost growth by committing to product development with less technical
knowledge than needed by acquisition best practices and without
ensuring that requirements are defined, feasible, and achievable within
cost and schedule constraints. In addition, the program has a concurrent
schedule for constructing deckhouses and procuring Aegis Ashore
components. Since committing to product development and establishing
the product development baseline, the acquisition strategy for deckhouse
construction has been revised twice. The current plan, called the dual
deckhouse plan, is to construct two deckhouses—first, an operational
deckhouse planned for installation in Romania and a second for
developmental testing in Hawaii. The test deckhouse will begin
construction a quarter later than the operational deckhouse and will be
installed for testing at the Pacific Missile Range Facility in Hawaii. Aegis
BMD program management officials stated that a third deckhouse, for the
Aegis Ashore installation in Poland, will be constructed at a later date to
be set based on funding availability. The program also has initiated
procurement of equipment, such as the VLS and SPY-1 radar that are
needed for the Aegis Ashore installations.

This plan means that knowledge gained from testing the Hawaiian
installation cannot be used to guide the construction of the Romanian
deckhouse or procurement of components for operational use. Any
design changes that arise from testing in Hawaii will have to occur on a
complete deckhouse and on already procured components intended for
operational use. As we have previously reported, rework on an existing
fabrication is costly. Aegis Ashore is currently scheduled to participate in
four flight tests, three of which are intercepts, with the first intercept flight
test scheduled for the second half of fiscal year 2014, at which point two
of the three deckhouses will be completed and Aegis Ashore site



1
 GAO, High-Risk Series: Defense Weapons System Acquisition, GAO/HR-93-7
(Washington, D.C.: December 1992).




Page 66                                                  GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore




construction and interceptor production will be well under way. The final
flight test is planned for the fourth quarter of fiscal year 2015. See figure
10 for a depiction of Aegis Ashore’s concurrent schedule.

Figure 10: Aegis Ashore Schedule




However, Aegis BMD program management officials state that Aegis
Ashore has taken steps to lower the acquisition risks. First, the officials
note that the program is using components already in use aboard Aegis
BMD ships, reducing the technical risk of the program. The Director of
MDA has stated that the sea-based system and Aegis Ashore will share
identical components. According to program documentation, the dual
deckhouse plan reduces risk and creates fabrication and construction
efficiencies. Aegis BMD program management officials noted that the
dual deckhouse plan has significant advantages over prior plans, all of
which had the operational deckhouse built before the test deckhouse. For
example, they noted that prior Aegis Ashore deckhouse construction
plans required testing a different deckhouse design in Hawaii than the
one that would be used at the operational sites. Constructing two
deckhouses concurrently provides for greater efficiency in purchasing
material and equipment and allows for one contractor to build both
deckhouses. The Director of MDA stated that the deckhouse construction
methodology is the most cost effective and efficient under the program’s
time constraints. In addition, the program expects to be able to modify the
operational deckhouse prior to its installation in Romania if flight tests
reveal that a modification is needed. The program management officials
also stated that the dual deckhouse plan provides more time for testing
the equipment that goes in the deckhouse. Aegis BMD program
management officials stated that this plan allows them to test the
electrical system in the Romanian deckhouse and to complete these tests



Page 67                                                GAO-12-486 Missile Defense
                           Appendix VII: Aegis Ashore




                           more than 1 year earlier than previously scheduled. Finally, they noted
                           that constructing two deckhouses also facilitates testing, including
                           conducting Aegis Light Off events that consist of preflight test verification
                           of the integration of Aegis Ashore components.

                           Aegis BMD program management officials told us that the schedule does
                           contain more risk before the first controlled test vehicle flight test, which is
                           the first time all of the Aegis Ashore components will be integrated, and
                           less risk between that test and the fielding in Romania. They stated that
                           they decided to increase the risk at the start of the schedule in order to
                           meet the presidentially announced date of 2015 for the first Aegis Ashore
                           installation.

                           While Aegis BMD program management officials are confident that the
                           risks of a concurrent schedule are low given the nature of the Aegis
                           Ashore program, the short time frame for integrating and fielding Aegis
                           Ashore could magnify the effects of any problems that may arise.
                           Program documentation states that there is limited to no margin in the
                           schedule to deal with possible delays in fabrication or system testing, and
                           as this effort is the first time a land-based deckhouse has been
                           constructed, there is no prior experience on which to draw to alleviate any
                           schedule delays.


Various Aegis Ashore       While Aegis Ashore will use components already developed and used
Components Require         operationally in the sea-based Aegis BMD, key components—the VLS
Modification for a Land-   and radar—will be modified for use on land. In addition, the multimission
                           signal processor, a key component for both the sea-based and land-
Based Configuration, and   based system that processes radar inputs from ballistic and cruise missile
Development                targets, is still under development and behind schedule. The first time all
Uncertainties Remain       of the Aegis Ashore components are expected to be integrated and flight
                           tested will be in fiscal year 2014. Given the concurrent schedule for the
                           program, any difficulties with the modified components or partly
                           developed components may affect the overall schedule, potentially
                           leading to cost growth or an installation not meeting expectations
                           because a needed modification was discovered too late.

                           The Aegis Ashore installations will include a VLS currently used on Aegis
                           BMD ships, but it is planned to be located at a greater distance from the
                           deckhouse. The communications system between the deckhouse and the
                           VLS will require modification because of this increased distance. In
                           addition, the VLS is planned to be surrounded by an environmental
                           enclosure at Aegis Ashore installations. Aegis BMD program


                           Page 68                                                GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore




management officials stated that this enclosure will include the heating
and cooling system and provide power to the launcher. Testing of this
modification is planned for fiscal year 2014.

Aegis Ashore’s SPY-1 radar likely will face challenges related to the
radio-frequency spectrum, which is used to provide an array of wireless
communications services, such as mobile voice and data services, radio
and television broadcasting, radar, and satellite-based services. The
radar might need to be modified if the performance of wireless devices in
Romania is degraded by the SPY-1. Furthermore, Romania’s future use
of the radio-frequency spectrum is unknown but could allow more
domestic wireless communications services to operate in or near the
radar’s operating frequency. Consequently, the Aegis Ashore site may
need modifications to resolve this potential issue, or alternatively,
Romanian wireless broadband devices may need to be modified. An
initial analysis of radio-frequency spectrum use in Romania by the
Defense Spectrum Organization, DOD’s organization that provides
information and assistance on radio frequency analysis, planning, and
support, recommended to MDA that additional study of Romanian radio-
frequency spectrum use occur. Aegis BMD management officials told us
that they recognize the risks associated with operating the SPY-1 radar
on land and that MDA plans additional study in fiscal year 2012 to better
understand Romanian spectrum use and the potential effect of the SPY-1
radar on land, including study of existing land-based SPY-1 radars. There
may be modifications to the SPY-1 radar to mitigate this potential issue,
but the officials told us they do not currently know what modifications
could be required to mitigate any frequency issues because of this need
for further study. Depending on spectrum policy and usage in the host
nation, this issue may be a long-term challenge over the life of the Aegis
Ashore installations regardless of where they are fielded.

In addition, urban clutter—which could affect the ability to acquire,
maintain track, and perform imaging on long-range targets—could affect
the SPY-1 radar. Program documentation states that both the Romanian
and Polish Aegis Ashore sites have clutter from urban structures and
wind farms. Urban clutter may require modifications of the radar, such as
software modifications, or may require additional testing or affect
operations of the Aegis Ashore installation.




Page 69                                             GAO-12-486 Missile Defense
                           Appendix VII: Aegis Ashore




                           In addition to the aforementioned VLS and radar issues, developmental
                           uncertainties also exist for the multimission signal processor. We have
                           previously reported that it is behind schedule, with a significant
                           percentage of its software increments still needing to be integrated. 2 This
                           component of Aegis Ashore was unable to demonstrate planned
                           functionality for a radar test event in December 2010, and the Defense
                           Contract Management Agency has identified the multimission signal
                           processor schedule as high risk.

                           As we have reported previously, Aegis Ashore is dependent upon next
                           generation versions of Aegis systems—Aegis 4.0.1 and Aegis 5.0—as
                           well as the SM-3 Block IB interceptor, all of which are still under
                           development. 3


Unstable Aegis Ashore      Aegis Ashore’s requirements, acquisition strategy and overall program
Program Content,           content were not stable when the resource baseline—the expected
Affecting Both the         investment in the development and delivery of a product—was
                           established, and subsequent program changes obscure the assessment
Resource Baseline and      of program progress. MDA’s acquisition directive states that baselines are
Cost Estimates, Reduces    used to assess programs and program maturity. We have previously
Transparency and Impedes   reported that baselines provide the best basis for transparency over
Oversight and              actual program performance, giving decision makers key information
Accountability             about program progress and cost. 4 Baseline variances give management
                           information about where corrective action may be needed to bring the
                           program back on track. Variation from the baseline can provide valuable
                           insight into program risk and its causes and can empower management
                           to make decisions about how to best handle risks. However, this
                           transparency is limited if the initial baseline is not sound or if the reporting
                           of progress against the baseline obscures actual program cost or
                           performance.




                           2
                             GAO, Arleigh Burke Destroyers: Additional Analysis and Oversight Required to Support
                           the Navy’s Future Surface Combatant Plans, GAO-12-113 (Washington, D.C.: Jan. 24,
                           2012).
                           3
                               GAO 11-372.
                           4
                            GAO, Defense Acquisitions: Missile Defense Acquisition Strategy Generates Results but
                           Delivers Less at a Higher Cost, GAO-07-387 (Washington D.C.: Mar. 15, 2007).




                           Page 70                                                     GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore




Aegis Ashore’s resource baseline, established at the developmental
baseline review on June 22, 2010, was initially $813 million. The initial
resource baseline established the resources needed to develop and build
two Aegis Ashore systems—one test and one operational— and deploy
them in the 2015 time frame. In the June 25, 2010 BMDS Accountability
Report (BAR) submitted to Congress 3 days after the review, MDA
reported a revised resource baseline of $966 million, an increase of
$153 million or 19 percent. According to information provided by the
program, the reason for the increase was a refinement of the program
requirements and a review of resource estimates provided earlier in fiscal
year 2010.

Beyond this resource baseline adjustment, the anticipated cost of the
program has grown as program plans have developed. By February
2012, program management officials provided information that the
program was reporting a cost growth of $622 million over the 2010
baseline for a total cost estimate of $1.6 billion. 5 Aegis BMD management
officials provided information attributing the cost growth to changes in the
deckhouse fabrication plans, an increase in the cost of the Aegis
Weapons system, and a refinement of equipment needs. In addition, the
program has adjusted the calculations for the average procurement unit
cost (APUC), or the ratio of procurement costs to the number of
operational units, across the life of the program. At the developmental
baseline review in June 2010, the APUC was based on the test
installation in Hawaii. By June 2011, the program included two
installations—for Romania and Poland—in the APUC. However, at the
end of fiscal year 2011, the program changed the quantity to one Aegis
Ashore installation. Information provided by the program office states that
the increase to two installations occurred due to the addition of all
European PAA phases to the program during the year and that the fiscal
year 2012 BAR will include only one installation to be consistent with the
2011 BAR. The current estimate for the APUC also has changed. The
baseline for the average procurement cost is $272 million for each Aegis
Ashore system. Program management officials reported that by February




5
  According to information provided by the program, the $1.6 billion includes costs for
development, military construction, operations and support and disposal. It excludes Navy
military construction, manning and fleet operations costs as well as costs for MDA’s
command, control, battle management, and communications system. It also does not
include any procurement funding.




Page 71                                                       GAO-12-486 Missile Defense
Appendix VII: Aegis Ashore




2012, the estimate for the APUC was $380 million, a 40 percent increase
over the baseline unit cost.




Page 72                                          GAO-12-486 Missile Defense
Appendix VIII: Ground-based Midcourse
                                            Appendix VIII: Ground-based Midcourse
                                            Defense (GMD)



Defense (GMD)


Fiscal year 2011 events                                              Overview

•   In December 2010, the program failed to successfully                GMD has not been able to verify the capability of the CE-II
    intercept a target during a retest of the unsuccessful January       interceptor.
    2010 intercept attempt. The tests were designed to verify the       MDA’s cost to demonstrate the CE-II through flight testing has
    capability of the enhanced version of the kill vehicle called        grown significantly.
    the Capability Enhancement II (CE-II EKV).
                                                                        MDA has pursued a highly concurrent acquisition strategy for
•   Due to the failed intercept test, the Director, MDA halted           the GMD program that allowed for rapid fielding, but with
    final integration of the remaining CE-II EKVs.                       increased risks.
•   At the request of the U.S. Northern Command, delivered a            Consequences of MDA’s highly concurrent acquisition strategy
    second fire control system to Fort Greely Alaska so that             include schedule delays, cost growth and reduced
    testing can occur while the system is also operational.              understanding of system performance.
•   GMD participated in a BMDS ground test during which
    operational personnel executed tactics, techniques and
    procedures for the defense of the United States.




Background and Overview                     The GMD element enables combatant commanders from the U.S. Space
                                            and Missile Defense Command 1 to defend the United States against a
                                            limited attack from intermediate- and intercontinental-range ballistic
                                            missiles from nations such as North Korea and the Middle East during the
                                            midcourse phase of flight. GMD consists of a ground-based interceptor
                                            (GBI)—a booster with an exoatmospheric kill vehicle (EKV) on top—and a
                                            fire control system that receives target information from Ballistic Missile
                                            Defense System sensors in order to formulate a battle plan. The GMD
                                            program has emplaced two EKV versions. The first, fielded since 2004, is
                                            known as the Capability Enhancement I (CE-I) 2 and the second, the
                                            current version in production, is called the Capability Enhancement II
                                            (CE-II). GMD has fielded its entire planned inventory of 30 GBIs. 3
                                            According to the Director, Missile Defense Agency (MDA), GMD is
                                            expected to remain in service until at least 2032.




                                            1
                                                This command is the Army service component to U.S. Strategic Command.
                                            2
                                              The original EKV’s delivered were called Test Bed kill vehicles, however, they have
                                            since been renamed Capability Enhancement I.
                                            3
                                              In 2009, the Secretary of Defense reduced the number of planned emplaced GBIs from
                                            44 to 30, reducing the number of GBIs needed. The reduced inventory includes 30
                                            operational interceptors and an additional 22 for testing and spares.




                                            Page 73                                                         GAO-12-486 Missile Defense
                               Appendix VIII: Ground-based Midcourse
                               Defense (GMD)




                               In fiscal year 2011, MDA continued to provide U.S. Northern Command a
                               capability to defend the nation against a limited ballistic missile attack and
                               delivered a second fire control system to Fort Greely, Alaska, to provide
                               flexibility to operate while also testing the system.


GMD Has Not Been Able to       MDA has not successfully demonstrated the ability of the CE-II to
Verify the Capability of the   intercept a target. The first two attempts failed—the first in January 2010
CE-II EKV Interceptor          due to a quality control issue and the second in December 2010 due to a
                               design issue. 4 During this second attempted test, MDA launched an
                               intermediate-range target with a simulated reentry vehicle and associated
                               objects. A forward-based radar provided acquisition and track data to the
                               GMD system. In addition, the Sea-based X-band radar provided
                               discrimination data to the GMD system. The GMD interceptor was
                               launched from a silo at Vandenberg Air Force Base, flew as expected to
                               its designated point, and deployed the CE-II EKV, which reached the
                               target and identified the most lethal object but failed to intercept it.

                               After this failure, the Director, MDA, testified that the agency’s top priority
                               was to confirm the root cause, fix it, and successfully repeat the previous
                               flight test. 5 Accordingly, MDA undertook an extensive and rigorous effort
                               to determine the root cause of the failure and develop design solutions to
                               resolve the failure. The investigation concluded the following: (1) ground
                               testing cannot replicate the environment in which the kill vehicle operates
                               and (2) the CE-II EKV, specifically the inertial measurement unit, requires
                               redesign and additional development, which MDA has undertaken. For
                               example, according to a GMD program official, the program has
                               conducted over 50 component and subcomponent failure investigation
                               and resolution tests. Additionally, the program has developed new testing
                               techniques and special instrumentation to provide additional data in future
                               flight tests.




                               4
                                The failure review investigation concluded that FTG-06 failed due to a quality control
                               escape where a lockwire was not inserted during the EKV manufacturing process. The
                               contractor has altered its processes now to ensure that all steps are properly followed.
                               5
                                 According to GMD and contractor officials, they understood that they could not replicate
                               the failure during ground testing and have since developed new technology to measure
                               the frequencies that they might experience in space. Although the frequency ranges are
                               not fully compatible, they are making progress.




                               Page 74                                                        GAO-12-486 Missile Defense
                             Appendix VIII: Ground-based Midcourse
                             Defense (GMD)




                             MDA realigned resources from planned 2011 activities to fund the
                             investigation and fund return-to-intercept activities including redesign
                             efforts. For example, the program delayed funding the rotation of older
                             fielded interceptors into flight test assets, delayed funding interceptor
                             manufacturing, and delayed purchasing GBI upgrade kits. However, the
                             agency did continue its efforts to increase reliability of the interceptors
                             through upgrades and repair of five interceptors although the
                             refurbishments conducted to date do not fix all known issues or provide a
                             guarantee of reliability.


MDA’s Cost to                The cost to confirm the CE-II capability through flight testing has
Demonstrate the CE-II        increased from $236 million to about $1 billion dollars due to the flight test
Initial Capability through   failures as noted in table 4. In addition to the costs of the actual flight
                             tests, the total cost for determining the root cause and developing the
Flight Testing Has Grown     design changes has not been fully developed.
Significantly
                                                                                                                                              a
                             Table 4: Flight Test and Failure Review Cost to Assess CE-II Capability

                                 Dollars in millions
                                 Activity                                                                                                             Cost
                                 FTG-06                                                                                                               $236
                                 FTG-06a                                                                                                               240
                                 CTV-01 Costs as of February 2012                                                                                      141
                                 FTG-06b Costs as of February 2012                                                                                     269
                                 Failure review costs as of February 2012                                                                                91
                                 Total                                                                                                                $976
                             Source: MDA.
                             a
                                 Flight test costs include the target, mission planning, range support, and post-test analysis.



                             While the cost incurred by MDA to verify the CE-II variant through flight
                             testing, as noted above, is about $1 billion, it does not reflect the costs
                             already expended during development of the interceptor and target. For
                             example, the cost of the flight test excludes nonrecurring development
                             costs, such as the development costs for the interceptor or target and its
                             support as well as those for systems engineering and test and evaluation,
                             among others. Often these are costs that were incurred many years
                             before the flight test was conducted. MDA has not separately reported the
                             nonrecurring development costs for the CE-II interceptor, but instead
                             reports the program acquisition unit costs (which are the development,
                             production, deployment, and military construction costs divided by the
                             total number of operationally configured units) for the combined CE-I and



                             Page 75                                                                                              GAO-12-486 Missile Defense
                           Appendix VIII: Ground-based Midcourse
                           Defense (GMD)




                           CE-II interceptor effort. For these interceptors, the program acquisition
                           unit costs are reported to be $421 million as of February 2011 and are
                           likely increasing to address the flight test failure. MDA reports the
                           nonrecurring costs for the targets used in these flight test as $141 million
                           each. Consequently, including nonrecurring development costs for both
                           the CE-II and the targets would substantially increase the costs for each
                           flight test and the overall costs outlined in table 4.


MDA Has Pursued a Highly   To meet a 2002 presidential directive to deploy an initial missile defense
Concurrent Acquisition     capability by 2004, MDA concurrently matured technology, designed the
Strategy for GMD That      element, tested the design and produced and fielded an initial capability.
                           A 2008 MDA briefing acknowledged that fielding while still in the
Allowed for Rapid          development and test phase led to very risky decisions regarding
Fielding, but with         schedule, product quality, and program cost. For example, the EKV team
Increased Risks            focused on technical aspects of design instead of also ensuring that the
                           design could be produced, which led to a lack of production control and
                           near continuous engineering changes. While this approach did lead to the
                           rapid fielding of a limited defense, it also resulted in schedule delays,
                           unexpected cost increases, a refurbishment program, and a reduced
                           knowledge of system reliability necessary for program sustainment, as
                           well as variations between delivered CE-I EKVs. (See fig. 11.)




                           Page 76                                              GAO-12-486 Missile Defense
Appendix VIII: Ground-based Midcourse
Defense (GMD)




Figure 11: GMD Concurrent Schedule




MDA emplaced its first GBI in 2004, although it had little of the data, such
as interceptor reliability, that it would normally have had before fielding a
system. Accordingly, the Director, MDA, testified on March 2011 that
GMD put interceptors “that are more akin to prototypes than production
representative missiles in the field.” Additionally, interceptors were
emplaced in silos before successfully conducting a flight test of this
configuration.

In 2004, MDA committed to another highly concurrent development,
production, and fielding strategy for the new CE-II interceptor, approving
the production before completing development of the prior version or flight
testing the new components. 6 MDA proceeded to concurrently develop,
manufacture, and deliver 12 of these interceptors even though MDA has
not yet successfully tested this new version.




6
 The CE-II EKV was not originally a reliability upgrade or a performance upgrade
program. Its initial priority was replacing obsolete components. However, updating certain
components is expected to result in increased performance.




Page 77                                                        GAO-12-486 Missile Defense
                           Appendix VIII: Ground-based Midcourse
                           Defense (GMD)




Consequences of MDA’s      MDA’s concurrent approach to developing and fielding assets has
Highly Concurrent          disrupted its acquisition efforts, resulted in cost growth and expensive
Acquisition Strategy       retrofits, and reduced the planned knowledge of the system’s capabilities
                           and limitations. In response to the failure of FTG-06a in December 2010,
Include Schedule Delays,   MDA restructured its fiscal year 2011 manufacturing plan by halting
Cost Growth, and Reduced   deliveries of remaining CE-II EKVs until the completion of the failure
Understanding of System    review and a nonintercept attempt in fiscal year 2012. 7
Performance
                           To help mitigate the affect of the production halt, the GMD program
                           planned to perform five limited upgrades to previously manufactured CE-I
                           interceptors. According to contractor officials, in order to keep the
                           production line viable, they were directed to complete five limited
                           interceptor upgrades; however, the program was only able to complete
                           three and expects to complete the other two in fiscal year 2012. As we
                           previously reported, in 2007 MDA began a refurbishment and retrofit
                           program of the CE-I interceptors to replace questionable parts identified
                           in developmental testing and manufacturing. 8 This program was to
                           develop an overall plan to address known hardware upgrades and service
                           life limitations, issues discovered since the interceptors were emplaced.
                           However, MDA has yet to complete all planned refurbishments of CE-I
                           EKVs, and program officials discovered additional problems during early
                           refurbishments causing MDA to expand this effort. Consequently,
                           refurbishments are planned to continue for many more years and the cost
                           to refurbish each CE-I interceptor could range from $14 million to
                           $24 million.

                           Additionally, MDA will have to undertake a major retrofit program for the
                           CE-II EKVs that have already been manufactured and delivered in
                           addition to the retrofit program for the CE-I GBIs that is already underway.
                           According to GMD program management officials, the final cost for this
                           effort has not been determined, but they expect the effort to cost about
                           $18 million per EKV, resulting in an additional cost of about $180 million
                           for 10 interceptors. 9



                           7
                             A GMD program official stated that the agency is allowing the contractor to continue
                           work on those components of the EKV that would not be factors in the FTG-06a flight test
                           failure in order to keep the production line moving.
                           8
                               GAO-09-338 and GAO-10-311.
                           9
                              Although the program has delivered 12 EKVs, 2 have already been expended during
                           flight testing.




                           Page 78                                                      GAO-12-486 Missile Defense
Appendix VIII: Ground-based Midcourse
Defense (GMD)




The agency has also had to restructure its flight test program, adding two
tests that were not previously planned before the failure. To verify the
new design of the kill vehicle, MDA inserted a nonintercept test scheduled
for the third quarter of fiscal year 2012. This test is designed to exercise
as many CE-II EKV functions as possible that have not been
demonstrated in either FTG-06 or FTG-06a. Performing the nonintercept
mission, using an upgraded inertial measurement unit, provides the
benefit of scripting the test in order to best stress the EKV design and to
fully demonstrate the resolution of the failure in FTG-06a. MDA officials
have stated that if the test confirms that the cause of the failure has been
resolved, the program will restart the manufacturing and integration of the
CE-II EKVs.

However, successfully completing an intercept that demonstrates the full
functionality of the kill vehicle is necessary to validate that the new design
works as intended. MDA added a new intercept flight test (FTG-06b) in
the fourth quarter of fiscal year 2012, however due to further
developmental challenges with the EKV, it has been delayed until at least
the second quarter fiscal year 2013 to demonstrate CE-II intercept
capability and achieve the unmet objectives of the two previous tests
(FTG-06 and FTG-06a). 10 As a result, confirmation that the design works
as intended will take place more than 9 years after the decision to begin
production and more than 4 years after the first planned test. 11

Lastly, MDA’s continued inability to conduct the GMD developmental flight
testing has resulted in less knowledge of the fielded systems capabilities
and limitations than planned. For example, GMD has been only able to
successfully conduct two intercept tests since 2006—the last successful
intercept being conducted December 2008. 12 Additionally, GMD has yet
to conduct a salvo test. As we reported in our last assessment, GMD
cancelled its planned 2011 salvo test due to the failure in the January
2010 flight test and scheduled a salvo test for fiscal year 2015.



10
  This schedule to return to flight may also be at risk because a key component
redesigned due to the earlier failure has experienced more problems in production.
11
  As we reported in 2009, MDA had originally planned to assess CE-II capability in fiscal
year 2008. However, early ground test failures in the inertial measurement unit caused
delivery delays and resulted in a redesign of the component. Consequently, the program
had to delay the test. See GAO-09-338.
12
     GMD conducted FTG-03a in September 2007 and FTG-05 in December 2008.




Page 79                                                       GAO-12-486 Missile Defense
Appendix VIII: Ground-based Midcourse
Defense (GMD)




Consequently, neither the CE-I nor CE-II variant capability is fully
understood and according to the Director, Operational Test and
Evaluation’s fiscal year 2010 assessment, the continuing evolution of the
interceptor design has resulted in multiple interceptor configurations
among the fielded interceptors and test assets. These configuration
differences complicate assessment of operational capability.

GMD’s acquisition strategy will continue its high levels of concurrency.
Developmental flight testing will continue through 2022, well after the
currently planned completion of production. In following this concurrent
acquisition strategy, the Department of Defense is accepting the risk that
these later flight tests may discover issues that require costly design
changes and retrofit programs to resolve.




Page 80                                             GAO-12-486 Missile Defense
Appendix IX: Precision Tracking Space
                                            Appendix IX: Precision Tracking Space System
                                            (PTSS)



System (PTSS)


Fiscal year 2011 events                                             Overview

•   PTSS was initiated in the second quarter of fiscal year 2011.      PTSS is just beginning the early acquisition phases.
•   Johns Hopkins University’s Applied Physics Laboratory,             PTSS revised its acquisition strategy in January 2012, includes
    under contract with the Missile Defense Agency, awarded             some acquisition best practices, but also elevated levels of
    subcontracts to industry partners, to bring them into the           concurrency.
    development process.                                               Projected size and cost of the entire PTSS constellation is
•   The PTSS program completed its System Requirements                  unknown.
    Review in March 2011.                                              While many technologies are well developed, technology
                                                                        maturation of key components still needed.
                                                                       PTSS concept development benefits from STSS testing.




Background and Overview                     The Missile Defense Agency’s (MDA) PTSS is being developed as a
                                            space-based infrared sensor system to provide persistent overhead
                                            tracking of ballistic missiles after boost and through the midcourse phase
                                            of flight. Being a space-based sensor system, PTSS is not constrained by
                                            geographical considerations that affect the placement of ground-, air-,
                                            and sea-based radar systems. While the number of PTSS satellites to
                                            make up the constellation has not yet been determined, the system is
                                            expected to expand the Ballistic Missile Defense System’s (BMDS) ability
                                            to track ballistic missiles in the post-boost phase and plans to fill coverage
                                            gaps existing within the current BMDS radar configuration. According to
                                            PTSS officials, the constellation will provide coverage of some 70 percent
                                            of the earth’s surface with a minimum of six satellites. Furthermore, the
                                            enhanced coverage planned for PTSS would help increase the size of the
                                            missile raids that the BMDS can track and respond to. The PTSS
                                            program plans to launch its first two development satellites in the fourth
                                            quarter of fiscal year 2017 and to increase the constellation to nine
                                            satellites by 2022.

                                            The PTSS program plans to create a satellite constellation that can
                                            accommodate subsequent configuration adjustments. The program
                                            intends to create a flexible on-orbit and ground architecture that could
                                            accommodate such changes as an increase to the constellation size or
                                            changes to the communications infrastructure. This flexibility would permit
                                            the system to evolve in response to changes in the threat environment.


PTSS Is Just Beginning the                  The PTSS program officially began as a new program in the second
Early Acquisition Phases                    quarter of fiscal year 2011. Johns Hopkins University’s Applied Physics



                                            Page 81                                                         GAO-12-486 Missile Defense
                             Appendix IX: Precision Tracking Space System
                             (PTSS)




                             Laboratory (APL) is the lead system developer for PTSS. In this capacity,
                             APL advises the PTSS program office on systems engineering and
                             integration issues, while leading the other laboratories involved in the
                             development effort. In early 2011, APL awarded six integrated system
                             engineering team subcontracts to industry partners to provide
                             manufacturing and producibility recommendations for the development of
                             the PTSS initial article satellites: Raytheon, Northrop Grumman,
                             Lockheed Martin, Ball Aerospace, Orbital Science, and Boeing. MDA’s
                             decision to involve the laboratories in initial development work is an action
                             that we have previously recommended for other space acquisition
                             programs.

                             During the course of 2011, the PTSS program made several schedule
                             changes, in part due to budgetary issues. PTSS was scheduled to begin
                             the Technology Development Phase in the fourth quarter of fiscal year
                             2011, but delayed it until the fourth quarter of fiscal year 2012. One of the
                             key early analytical knowledge points, the establishment of mass raid
                             engagement time windows, was also delayed from the fourth quarter of
                             fiscal year 2011 to the first quarter of fiscal year 2012. Finally, the
                             planned launch date for the first two initial satellites was delayed from the
                             fourth quarter of fiscal year 2015 and is now planned for the fourth quarter
                             of fiscal year 2017. The PTSS program also delayed the projected launch
                             dates of production satellites for the PTSS constellation.


PTSS Revised Its             According to the acquisition strategy report signed in January 2012, MDA
Acquisition Strategy in      plans to develop and acquire the satellites in three phases. First, the APL-
February 2012; It Includes   led laboratory team will produce two lab-built development satellites.
                             Second, an industry team, selected through open competition while the
Some Acquisition Best        APL-led laboratory team is still in a development phase, will develop and
Practices but Also           produce two industry-built engineering and manufacturing development
Elevated Levels of           satellites. Third, there will be a follow-on decision for the industry team to
Concurrency                  produce additional satellites in a production phase. (See fig. 12.)




                             Page 82                                               GAO-12-486 Missile Defense
Appendix IX: Precision Tracking Space System
(PTSS)




Figure 12: PTSS Concurrent Schedule




The strategy acknowledges some concurrency but maintains that there
are benefits to this approach. Under the plan, the industry team will be
approved for production of long-lead items for the two development
satellites, while the laboratory team is still working to complete the first
two development satellites. The program intends that by engaging
industry concurrently at this development stage, industry can influence
the selection of parts and subsystems in a manner that will minimize the
need for system design changes between the two laboratory development
satellites and the two initial industry satellites. The program intends to
conduct on-orbit checkout and testing of the two laboratory-produced
development satellites prior to the decision to complete the assembly of
the two industry-built development satellites.

According to MDA, the approach aligns with several aspects of GAO’s
acquisition best practices. The program will establish firm requirements
before committing to production, it will ensure full and open competition,
the development cycle will be less than 5 years, it has a simple payload
design and can deploy larger numbers in the constellation and it is
deferring advanced capabilities until a second spiral thereby limiting the
technological development challenge for the initial satellites.

According to program management officials, they have taken steps
intended to mitigate cost, schedule, and performance risks. PTSS is
being designed strictly for BMDS use, so the satellite payload is geared
toward the BMDS missile tracking mission, with the objective of keeping
the design as simple and stable as possible. Additionally, the acquisition


Page 83                                             GAO-12-486 Missile Defense
                             Appendix IX: Precision Tracking Space System
                             (PTSS)




                             strategy stipulates that PTSS will not duplicate functions found elsewhere
                             in the BMDS, but instead will remain focused on the specific function for
                             which it is being designed. The program aims to shorten its development
                             schedule through the use of proven technologies with high technology
                             readiness levels. According to PTSS program management officials, the
                             use of currently available technologies helps to keep the PTSS design
                             cost-effective. In addition, according to those officials, the government
                             intends to acquire unlimited data rights, government purpose data rights,
                             or both for the duration of the program, so that the government is not
                             locked in with any particular contractor.

                             Because the PTSS acquisition strategy was only recently developed, we
                             had limited time to assess the strategy for this review. We intend to
                             review this new strategy next year. Building developmental and
                             engineering and manufacturing development satellites is a positive step.
                             However, the strategy may enable decision makers to fully benefit from
                             the knowledge to be gained and the risk reduction opportunity afforded
                             through on-orbit testing of the lab-built satellites before committing to the
                             industry-built developmental satellites. The industry-built development
                             satellites will be under contract and under construction before on-orbit
                             testing of the first two lab-built satellites can confirm that the design works
                             as intended.


Projected Size and Cost of   Currently, the PTSS program office has not determined how many
the Entire PTSS              satellites will make up the PTSS constellation, though the program is
Constellation Are            progressing with a flexible approach toward the number of satellites in the
                             constellation. The size of a full PTSS constellation would depend on
Unknown                      factors that have yet to be determined, most specifically, the size of
                             missile raid that the system would be expected to track. In fiscal year
                             2011, the program conducted physics-based analysis to demonstrate the
                             system’s performance within the BMDS in handling a range of raid
                             scenarios. The satellites for the PTSS constellation are expected to have
                             a 5-year design life, though officials stated that they expect the
                             operational life will exceed the 5 years. Relative to other military space
                             programs, the PTSS satellite is intended to be a low-cost unit, which can
                             be readily replaced as on-orbit units degrade over time. However, the full
                             cost of development has not yet been determined, and it is currently
                             unclear how many satellites will need to be replaced annually, as this will
                             be determined by such factors as design life and the total number on
                             orbit. The cost to launch a satellite into orbit can be very expensive,
                             sometime exceeding $100 million or more. Because the full size of the
                             constellation has also not yet been determined, the PTSS program is


                             Page 84                                                GAO-12-486 Missile Defense
                           Appendix IX: Precision Tracking Space System
                           (PTSS)




                           unable to estimate the anticipated full costs of the acquisition and
                           operation of the system.


While Many Technologies    In leveraging proven technologies with high technology readiness, many
Are Well Developed,        of the system’s technologies are in relatively high states of maturity for a
Technology Maturation of   program in this early stage of development. The program office has
                           identified two PTSS critical technologies: the optical payload and the
Key Components Is Still    communications payload. Many of the underlying components for the
Needed                     optical and communications payloads have been demonstrated in an
                           environment relevant to the conditions under which they will be employed
                           in the PTSS satellites. However, certain key components of these critical
                           technologies require further development to reach maturity, and until
                           these key components mature, they reduce the overall technological
                           maturity of the payloads. Program management officials stated that they
                           plan to have both critical technologies in functional form by the time of the
                           preliminary design review, which is scheduled for the end of fiscal year
                           2013.

                           The high radiation environment in which the PTSS satellites will operate
                           creates technical challenges for the development effort. The PTSS
                           program has instituted risk reduction measures to address radiation risks
                           pertinent to two technologies. For risk issues pertaining to the focal plane
                           array, the PTSS’s risk mitigation efforts are on schedule, with two
                           contracts having been awarded to explore manufacturing processes to
                           address radiation hardness requirements for the satellites’ anticipated on-
                           orbit environment. Radiation mitigation efforts are also required for the
                           satellite’s star tracker, a component of the system’s guidance and control
                           subsystem. The PTSS program plans to award contracts to several
                           vendors in 2012 to evaluate options to address this concern.


PTSS Concept               The PTSS development effort is benefiting from MDA’s two operational
Development Benefits       Space Tracking and Surveillance System (STSS) satellites, which were
from STSS Testing          launched into orbit in 2009. BMDS test events involving STSS have been
                           useful in providing key information to the PTSS program. According to
                           PTSS officials, the success of STSS in the FTM-15 flight test conducted
                           in 2011 served as a “proof of principle” for PTSS, as the event
                           demonstrated multiple aspects of the PTSS concept of operations, such
                           as the ability to provide data from which interceptor missiles could be
                           remotely launched and directed toward a missile threat. The FTM-12 flight
                           test in late 2011 repeated the positive results noted in FTM-15, with
                           tracking sensors locking onto targets and successfully providing direction


                           Page 85                                              GAO-12-486 Missile Defense
Appendix IX: Precision Tracking Space System
(PTSS)




for the fired interceptors. The STSS tests are assisting the PTSS program
office as it develops the system’s concept of operations.




Page 86                                           GAO-12-486 Missile Defense
Appendix X: Targets and Countermeasures
                                            Appendix X: Targets and Countermeasures




Fiscal year 2011 events                                           Overview

•   Eleven targets were delivered and successfully launched.         Targets were not a source of testing problems in 2011.
•   Successful “return-to-flight” of short range air-launched        Availability of targets increasing risk for first Ballistic Missile
    target.                                                           Defense System operational flight test and affecting other
•   Intermediate-range ballistic missile (IRBM) target contract       planned tests.
    signed in March 2011.                                            Missile Defense Agency slowly building inventory of targets and
•   Undefinitized contract action for eight reentry vehicles          implementing backup strategy.
    issued.                                                          Significant decisions consolidate new work with the prime
•   Intercontinental ballistic missile (ICBM) target contract         contractor.
    solicitation canceled February 16, 2011.




Background and Overview                     The Missile Defense Agency’s (MDA) Targets and Countermeasures
                                            program designs, develops, produces and procures missiles serving as
                                            targets for testing missile defense systems. The targets program involves
                                            multiple acquisitions covering the full spectrum of threat missile
                                            capabilities (separating and nonseparating reentry vehicles, varying radar
                                            cross sections, countermeasures, etc.) and ranges. 1 Some target types
                                            have been used by MDA’s test program for years while others have been
                                            recently or are now being developed and can represent more complex
                                            threats.

                                            As MDA’s test program has matured, its Targets and Countermeasures
                                            program has worked toward developing, in parallel, more complex targets
                                            that can more closely represent modern-day threats. Since the program
                                            was initiated in 2001, it has done this using several different acquisition
                                            strategies—the third was issued in 2011. Initially, MDA used many
                                            contractors to design and build the targets, but in 2003, it chose a single
                                            prime contractor, Lockheed Martin to lead the acquisition. Shortly after,
                                            MDA decided to pursue what it called the Flexible Target Family
                                            approach to acquiring targets, which used common components and
                                            shared inventory and promised reduced cycle time, cost savings, and



                                            1
                                              Ballistic missiles are classified by range: short-range ballistic missiles have a range of
                                            less than 1,000 kilometers (621 miles); medium-range ballistic missiles have a range from
                                            1,000 to 3,000 kilometers (621 to1,864 miles); intermediate-range ballistic missiles have a
                                            range from 3,000 to 5,500 kilometers (1,864 to 3,418 miles); and intercontinental ballistic
                                            missiles have a range greater than 5,500 kilometers (3,418 miles).




                                            Page 87                                                           GAO-12-486 Missile Defense
                            Appendix X: Targets and Countermeasures




                            increased capability. At that time, MDA began work on the 72-inch
                            diameter launch vehicle (LV)-2 target and the 52-inch diameter targets.
                            When this approach proved more costly and less timely than expected,
                            MDA suspended the 52-inch effort, focusing on the LV-2. Responding to
                            congressional concern 2 about these problems and our 2008
                            recommendations, MDA revised its acquisition approach in 2009, seeking
                            to increase competition by returning to a multiple contract strategy with
                            four separate target classes and a potential of four prime contractors.
                            MDA completed the intermediate-range target contract award, which
                            reduced target costs. However, as proposals for the new medium-range
                            ballistic missile (MRBM) contract were submitted, the program
                            determined that costs associated with this approach were higher than
                            anticipated. Solicitations for the medium-range and the intercontinental
                            classes of targets were then canceled, and MDA began the process of
                            revising its acquisition strategy for the third time.


Targets Were Not a Source   In the past, we have reported that availability and reliability of targets
of Testing Problems in      caused delays in MDA’s testing of Ballistic Missile Defense System
Fiscal Year 2011            (BMDS) elements. However, in fiscal year 2011, MDA delivered 11
                            targets, all of which were successfully launched and did not negatively
                            affect the test program. The targets launched during the year supported
                            tests of several different BMDS elements, including Ground-based
                            Midcourse Defense (GMD), Aegis Ballistic Missile Defense, and Patriot
                            systems. 3

                            All targets that were delivered or launched within the fiscal year were
                            either short- or intermediate-range targets and performed as expected.
                            Most notably, in July 2011 MDA successfully accomplished the return-to-
                            flight of MDA’s short-range air-launched target. This was the target’s first
                            launch since an essential mechanism that releases it from the aircraft
                            failed in a December 2009 Terminal High Altitude Area Defense (THAAD)
                            flight test. After the failure, the agency identified shortcomings in the
                            contractor’s internal processes that had to be fixed before air-launched
                            targets could be used again in BMDS flight tests. Nineteen months later,



                            2
                             H.R Rep No. 110-477 at 824 (2007) (Conf. Rep.) (accompanying the National Defense
                            Authorization Act of 2008. Pub. L. No. 110-181).
                            3
                             This report does not contain an assessment of the Patriot Advanced Capability-3, which
                            has been transferred to the Army for production, operation, and sustainment.




                            Page 88                                                      GAO-12-486 Missile Defense
                            Appendix X: Targets and Countermeasures




                            these deficiencies were satisfactorily addressed when the target missile
                            was successfully extracted from the rear of the C-17 aircraft in FTX-17.
                            To reduce risk, the flight was not planned as an intercept mission but as a
                            target of opportunity for several emerging missile defense technologies,
                            including Space Tracking Surveillance System.

Availability of Targets     According to MDA and Director, Operational Test & Evaluation test
Increasing Risk for First   officials, the availability of targets has affected planned future flight tests.
BMDS Operational Flight     MDA has scheduled the first two extended medium-range ballistic
                            missiles (eMRBM) to launch in a crucial operational flight test (FTO-01)
Test and Affecting Other    by the end of 2012, which is the first system-level test of the BMDS. On a
Planned Tests               tight schedule to meet this deadline, MDA is accepting higher risk that
                            target issues could affect this test by launching the first two of the new
                            targets in this operational test, rather than conducting a risk reduction
                            flight first. Risk reduction flight tests are conducted the first time a system
                            is tested in order to confirm that it works before adding other test
                            objectives. The lack of such a test was one factor that delayed a previous
                            GMD flight test (FTG-06) in 2010. While the target, the LV-2, was
                            successfully flown in that flight test, aspects of its performance were not
                            properly understood and lack of modeling data prior to the test
                            contributed to significant delays in the test program.

                            In addition, the next air-launched target test was scheduled to use the
                            new medium-range extended air launched target in 2012, but the flight
                            test—FTT-13—was cancelled because of budgetary concerns and test
                            efficiency. As a result, the first flight test using this target is not planned
                            until the third quarter of fiscal year 2014, though it may be available for
                            use as early as the fourth quarter of fiscal year 2012. Since the short-
                            range air-launched target was successfully launched in July 2011, MDA
                            now plans to continue acquisition of the one short-range and the two
                            extended air-launched targets that are currently under contract through
                            fiscal year 2014.


MDA Slowly Building         As development and production processes mature, the targets program is
Inventory of Targets and    slowly developing an inventory of targets for use in BMDS testing. In
Implementing Backup         previous years, failures of target missiles have caused major disruptions
                            to MDA’s flight test program, in part because no spare targets were
Strategy                    available to retest once the cause of any failure was determined. In
                            response, MDA has slowly begun working toward developing an inventory
                            of both backup and spare missiles to support the test plan. The targets
                            program currently has a limited backup strategy in place. In the event of a
                            target failure, backup missiles assigned to a future mission could be taken


                            Page 89                                                 GAO-12-486 Missile Defense
                            Appendix X: Targets and Countermeasures




                            from inventory and used for an earlier test and be replaced by newer
                            missiles.


Significant Decisions       The Targets and Countermeasures program made several key decisions
Cancel or Defer             in fiscal year 2011 that will shape future target acquisition. Two key
Competition and             contracts were definitized in 2011; the eMRBM contract in October 2011,
                            and an intermediate-range ballistic missile (IRBM) target contract in
Consolidate New Work        March 2011. MDA realigned funding planned for the medium-range
with the Prime Contractor   competition, which was canceled in 2010, to manufacture additional IRBM
                            targets. MDA canceled the planned intercontinental ballistic missile
                            (ICBM) competition because the new test plan delays the need for the
                            first ICBM target by several years. Finally, MDA issued an undefinitized
                            contract action to the prime contractor for reentry vehicles. One overall
                            consequence of these decisions has been a consolidation of work with
                            the prime contractor. (See table 5.)

                            Table 5: Fiscal Year 2011 Acquisition Events by Target Class

                                Target class                                Fiscal year 2011 acquisition events
                                Short-range ballistic missile targets       •    Air-launched target return-to-flight in July
                                                                                 2011
                                                                                                                a
                                                                            •    Undefinitized contract action issued for
                                                                                 acquisition of one foreign military target
                                MRBM targets                                •    eMRBM production contract definitized in
                                                                                 October 2011 for five targets
                                                                            •    Undefinitized contract action issued in July
                                                                                 2011 for seven specialized MRBM targets
                                                                            •    Extended-range air-launched target
                                                                                 qualification process resumed
                                IRBM targets                                •    IRBM contract awarded in March 2011 for
                                                                                 eight targets
                                ICBM targets                                •    Specialized ICBM contract solicitation
                                                                                 canceled in February 2011—acquisition
                                                                                 delayed to align with first ICBM test in
                                                                                 2020
                                Reentry vehicle                             •    Undefinitized contract action for eight
                                                                                 common reentry vehicles issued to prime
                                                                                 contractor
                            Source: GAO Analysis of MDA data.
                            a
                             To meet urgent needs, DOD can issue undefinitized contract actions, which authorize contractors to
                            begin work before reaching a final agreement on contract terms. Undefinitized contract action means
                            any contract action for which the contract terms, specifications, or price are not agreed upon before
                            performance is begun under the action. Defense Federal Acquisition Regulation Supplement
                            217.7401(d).




                            Page 90                                                               GAO-12-486 Missile Defense
Appendix X: Targets and Countermeasures




An agreement on price was reached for the production of five eMRBM
targets in September 2011. MDA began developing the eMRBM for
operational use in 2003 as part of the Flexible Target Family when it was
referred to as the 52-inch target. Though development and production
had been on hold since 2008 because of continuing cost and schedule
problems, MDA resumed acquisition of eMRBMs through the existing
prime contractor due to a target failure. The production contract was
definitized in October 2011 after being undefinitized for about 540 days. 4
The Defense Federal Acquisition Regulation states that undefinitized
contract actions shall provide for definitization by the earlier of either, 180
days after issuance of the action or the date on which more than 50
percent of the not-to-exceed price has been obligated. The 180-day
threshold may be extended but may not exceed the date that is 180 days
after the contractor submits a qualifying proposal. MDA program officials
stated that because MDA continued to change the requirements on the
undefinitized contract action, the contractor did not submit a qualifying
proposal until March 2011. MDA definitized the contract approximately
194 days after receiving the proposal. During the 18-month delay, while
the contract was being negotiated and requirements continued to change,
the contractor spent over $82 million, 5 the quantity of targets under
contract increased, and some capability was deferred to later years. The
final negotiated price at completion was $321 million, $175 million less
than the previously expected price ceiling. MDA contracting officials
acknowledged that undefinitized contract actions can lead to undefined
costs, but believe they are a good tool to use to meet urgent
requirements.

MDA initiated three new undefinitized target contract actions in fiscal year
2011.

•   First, an action for seven “T3” medium-range ballistic missile targets
    was initiated in July 2011. A requirement for this target type was



4
 Defense Federal Acquisition Regulations 217.7404-3 If a contractor submits a qualifying
proposal before the 50 percent threshold has been reached, then the limitation on
obligations may be increased to no more than 75 percent.
5
   The extended use of undefinitized contract actions has previously been identified by
GAO and others as risky for the government. Because, under undefinitized contract
actions, contracting officers normally reimburse contractors for all reasonable, allocable,
and allowable costs they incur before definitization, contractors bear less risk and have
little incentive to control costs during this period.




Page 91                                                         GAO-12-486 Missile Defense
Appendix X: Targets and Countermeasures




    accelerated in the test plan to the first quarter of fiscal year 2014. T3s
    are unique targets designed for more specialized maneuvers in their
    respective ranges.

•   Second, an action for a foreign military asset target to meet a fourth
    quarter of fiscal year 2012 requirement.

•   Third, an action for eight common reentry vehicles, which will replace
    earlier ones.

MDA set up a common components project office to manage the
acquisition strategy for the reentry vehicles, which are intended for flight
tests in mid-2014. They have the potential to fly on any target launch
vehicle, but the program is still developing more specific acquisition plans.

In 2011, MDA began implementing its third acquisition strategy for targets
by acquiring common reentry vehicles from a single source, a significant
change in the acquisition strategy for the program office. Reentry vehicles
for targets were previously acquired separately, were more specifically
tailored to the target launch vehicle, and were procured from more than
one contractor. The single-source strategy implemented with the 2011
undefinitized contract action is intended to maximize commonality and
could reduce costs through purchasing larger numbers. Through 2013,
the single source will be the targets prime contractor. MDA plans to
decide in the second quarter of fiscal year 2012 whether to issue a
competitive solicitation for a new provider.




Page 92                                               GAO-12-486 Missile Defense
Appendix XI: Terminal High Altitude Area
                                           Appendix XI: Terminal High Altitude Area
                                           Defense (THAAD)



Defense (THAAD)


Fiscal year 2011 events                                              Overview

•   The first production interceptor was delivered in March 2011.       Army and Department of Defense test organizations
•   Eleven of 50 THAAD operational interceptors were delivered in        successfully conducted the first THAAD operational flight
    fiscal year 2011.                                                    test.
•   The production decision was made and the contract was               THAAD’s highly concurrent acquisition strategy led to
    issued for additional THAAD batteries.                               delayed delivery of THAAD batteries.
•   Qualification problems with the ignition safety system and          Requirements and design were not stable before THAAD
    production start-up issues caused interceptor production rates       interceptor production began.
    to be slower than planned.                                          THAAD interceptor production issues delay deliveries and
•   Two Missile Defense Agency knowledge points, once planned            increase program costs.
    for fiscal year 2011, have not yet been achieved due to target      THAAD achieved conditional materiel release to the Army in
    availability issues.                                                 2012; full materiel release date not yet known
                                                                        Ongoing concurrency increases acquisition risks until
                                                                         developmental testing is complete.




Background and Overview                    THAAD is a rapidly deployable ground-based system designed to defend
                                           against short- and medium-range ballistic missile attacks during their late
                                           midcourse and terminal stages. A THAAD battery consists of interceptor
                                           missiles, six launchers, a radar, a fire control and communications
                                           system, and other support equipment. The program is producing batteries
                                           for initial operational use for conditional materiel release to the Army. For
                                           this to occur, the Army must certify that the batteries are safe, suitable,
                                           and logistically supported. The date for full materiel release has not yet
                                           been determined because the program is still conducting flight tests to
                                           prove out the system, and production rates have been slower than
                                           planned.


Army and Department of                     THAAD successfully conducted its first operational flight test in October
Defense Test Organizations                 2011, a major accomplishment because this was its first operational test
Successfully Conducted                     with the Army and Department of Defense test and evaluation
                                           organizations fully engaged to ensure that the execution and test results
the First THAAD                            were representative of the fielded system. During the test, the THAAD
Operational Flight Test                    system engaged and nearly simultaneously intercepted two short-range,
                                           threat-representative, ballistic missile targets. The test demonstrated the
                                           ability to perform in the full battle sequence, from planning through live
                                           operations, under operationally realistic conditions (within the constraints
                                           of test range safety). The U.S. Army Test and Evaluation Command and
                                           the Ballistic Missile Defense System (BMDS) Operational Test Agency
                                           will review data collected from this event to make an operational


                                           Page 93                                                       GAO-12-486 Missile Defense
                          Appendix XI: Terminal High Altitude Area
                          Defense (THAAD)




                          assessment of the THAAD system. In addition, the Director, Operational
                          Test and Evaluation, will also independently evaluate the operational
                          effectiveness of the system. The assessment of this event will support
                          upcoming production and fielding decisions.


THAAD’s Highly            The Missile Defense Agency (MDA) awarded a contract for THAAD’s first
Concurrent Acquisition    two operational batteries in December 2006, before its design was mature
Strategy Led to Delayed   and developmental testing of all critical components was complete. At
                          that time, MDA’s first THAAD battery, consisting of 24 interceptors, 3
Delivery of THAAD         launchers, and other associated assets, was to be delivered to the Army
Batteries                 as early as 2009. While some assets were delivered by this time, the
                          interceptors were delayed because of issues with components that had
                          not passed all required testing. In response to pressure to accelerate
                          fielding the capability, THAAD adopted a highly concurrent development,
                          testing, and production effort, as shown in figure 13, that has increased
                          program costs and delayed fielding of the first THAAD battery until early
                          fiscal year 2012.




                          Page 94                                           GAO-12-486 Missile Defense
Appendix XI: Terminal High Altitude Area
Defense (THAAD)




Figure 13: THAAD Concurrent Schedule




Problems encountered while THAAD was concurrently designing and
producing assets caused slower delivery rates of both the first and
second THAAD batteries, which are not currently projected to be
complete before July 2012—16 months after the original estimate of
March 2011. While all assets, except the interceptors, were complete in
2010, the first production interceptor for the first THAAD battery was not
produced until the second quarter of fiscal year 2011. In the same
quarter, MDA committed to purchasing additional assets by signing a
production contract for two additional THAAD batteries, despite
incomplete testing of a safety device on the interceptor. During fiscal year
2011, after several production start-up issues, 11 of the expected 50
operational interceptors were delivered. 1 Consequently, the first battery of
24 interceptors was not complete and available for fielding until the first
quarter of fiscal year 2012—more than 2 years later than originally
planned. The same issues have delayed the second battery as well.
Although the launchers and other components for the second battery 2
were completed in 2010, the full 50 interceptors necessary for both
batteries are not expected to be delivered until July 2012.




1
 Twelve total interceptors were delivered by the end of fiscal year 2011, but the first,
produced in fiscal year 2010, was used in a flight test.




Page 95                                                          GAO-12-486 Missile Defense
                          Appendix XI: Terminal High Altitude Area
                          Defense (THAAD)




Requirements and Design   A production contract was signed in 2006 before the requirements or
Were Not Stable before    design for a required safety device called an optical block was complete.
THAAD Interceptor         Housed in the flight sequencing assembly, an optical block is an ignition
                          safety device designed to prevent inadvertent launches of the missile.
Production Began          The program experienced design and qualification issues with this
                          component until testing was complete in the fourth quarter of fiscal year
                          2011. Incorporating an optical block device into the THAAD interceptor
                          has been a primary driver of design, qualification, and production delays
                          for the program since as early as 2003, shortly after the Army issued a
                          standard requirement for this type of safety device on munitions ignition
                          systems. The original THAAD design did not have an optical block device,
                          and MDA did not modify the development contract to include this
                          requirement until 2006. Program management officials explained that the
                          military standard is primarily written for smaller, more typical, munitions’
                          fuses, not systems as technically complex as THAAD.

                          According to program management officials, THAAD has worked with the
                          Army to tailor requirements and associated testing required of the optical
                          block device during the past few years. The part failed initial qualification
                          testing in early fiscal year 2010 and was not fully qualified until that
                          September. Also, in May 2010, the Army added requirements to test the
                          flight sequencing assembly during exposure to electrical stress and other
                          environments, such as extreme temperature, shock, humidity, and
                          vibration. Testing failures led THAAD to make minor design changes and
                          extensive manufacturing process changes, which required requalification
                          of the optical block and delayed production of the interceptors.
                          Environmental testing was complete in March 2011, but the stress test
                          was not completed until September 2011—after the first interceptor was
                          produced.

                          As recently as fiscal year 2011, the program was considering further
                          design changes to the optical block to make it more producible; however,
                          the program estimated that the cost to make the needed design changes
                          would be $150 million, an investment that could not be easily recouped in
                          production savings in the near future. Program mangers decided not to
                          make those changes because of improved flight sequencing assembly
                          and optical block manufacturing performance, and program funding
                          constraints. The current design was also successfully demonstrated in the
                          recent flight test and in the other testing in support of conditional materiel
                          release. Therefore, the program determined that the benefits of
                          continuing the redesign no longer justified the cost.




                          Page 96                                               GAO-12-486 Missile Defense
                            Appendix XI: Terminal High Altitude Area
                            Defense (THAAD)




THAAD Interceptor           Production issues have collectively delayed interceptor delivery by 18
Production Issues Delay     months and are projected to cost the program almost $40 million. While
Deliveries and Increase     issues with the flight sequencing assembly have been the most costly,
                            three production start-up issues emerged in fiscal year 2011 that also
Program Costs               caused delays. First, the program encountered problems with the
                            availability of a solution containing nitrogen needed for production.
                            Program management officials explained that since all of the liquid could
                            not be extracted out of a newly designed bottle, due to unanticipated
                            design changes in the delivery mechanism, more had to be ordered
                            before production could continue, which caused the delay. Another
                            production delay of over a month took place because of debris found in a
                            transistor on the interceptor. Program management officials explained
                            that a root cause analysis determined that the part had not undergone
                            proper testing, which would have detected such debris. The transistors
                            had to be replaced with properly tested parts. A third delay occurred
                            because ragged, raised edges were discovered inside several of the fuel
                            tanks. According to program management officials, in the unlikely event
                            that a small metal edge broke off during pressurization of the fuel tank, it
                            could cause an interceptor failure. They said that after conducting a risk
                            analysis, the program decided to remove the rough edges on future
                            procurements, but not on the first 50 interceptors, since the possibility of
                            such risk was low.

                            The interceptor’s flight sequencing assembly is currently being produced
                            at or above the expected rate of about four per month. Due to start-up
                            issues, which are common to new production lines, interceptor production
                            rates have fluctuated, ranging anywhere from 0 to 5 in recent months.
                            Also, some recent production rates could be artificially high as delays with
                            some components have allowed others more time than usual to stockpile
                            for future production. These stockpiles are projected to help with
                            production through the second battery. The program needs to achieve a
                            steady production rate in order to deliver the second THAAD battery by
                            July 2012. After this date, the contractor is scheduled to return to a rate of
                            3 interceptors per month.


THAAD Achieved              THAAD achieved conditional materiel release to the Army in February
Conditional Materiel        2012, though at one time, it had been expected as early as September
Release in 2012; Full       2010. It was delayed over a year due to ongoing safety issues with the
                            interceptor and, most recently, to incorporate data from the October 2011
Materiel Release Date Not   operational flight test. Conditional materiel release is an interim step to
Yet Known                   the Army’s full materiel release decision. For example, for conditional
                            materiel release, the Army Ignition System Safety Review Board requires


                            Page 97                                               GAO-12-486 Missile Defense
Appendix XI: Terminal High Altitude Area
Defense (THAAD)




that three flight sequencing assembly units complete a series of tests to
evaluate the interceptor in various electrical and other stressing
environments. By the end of fiscal year 2011, all these tests had been
successfully completed. While THAAD has performed all test events
required for conditional materiel release, including its most recent flight
test (FTT-12), analysis of data is ongoing and the Army is still refining its
requirements for full materiel release. Program management officials
expect the gap in knowledge between conditional materiel release and full
materiel release to be defined in second quarter of fiscal year 2012 as
well. At that time, they explained, the Army will have developed a list of
the remaining conditions that the program must address in order to
receive full materiel release.

One of the conditions that must be met to achieve full materiel release of
THAAD to the Army is the incorporation of the required Thermally Initiated
Venting System, a safety feature of the interceptor that prevents the boost
motor from becoming propulsive or throwing debris beyond a set distance
in the event that the canister holding the interceptor heats up to a certain
temperature. Development and testing of this system has been done
concurrently with production of fielded interceptors. Even if the latest
design and near-term testing is successful, the system will be approved
too late to be incorporated in the first 50 interceptors. Although the
system is not required for conditional materiel release, the program
expects it to be required for full materiel release, unless the Army grants a
waiver. Since the last two developmental tests of this safety feature have
failed, THAAD is at risk of not complying with the requirement. The next
test is scheduled for the second quarter of fiscal year 2012. According to
program management officials, if it fails, the program will be forced to
seek a waiver for the current design and accept the risk of not having the
design on the interceptors. Program management officials explained that
the requirement for a Thermally Initiated Venting System is primarily
written for smaller-scale systems, not for a system as large as THAAD.
Although officials said they are working to comply with the requirement,
the technology may not be available to make it work. At best, the program
could not incorporate the safety system into the interceptor until
production of the third battery. The Army has approved fielding the first 48
interceptors configured without the safety system based on available
testing and it has chosen to accept the associated risk.




Page 98                                              GAO-12-486 Missile Defense
                            Appendix XI: Terminal High Altitude Area
                            Defense (THAAD)




Ongoing Concurrency         While MDA is committed to producing four THAAD batteries, more flight
Increases Acquisition       tests are needed to achieve two remaining MDA developmental
Risks until Developmental   knowledge points set for the program. Both are tied to flight tests that
                            were, at one time, planned for fiscal year 2011 but were delayed into later
Testing Is Complete         fiscal years. MDA’s knowledge points identify information required to
                            make key decisions throughout the program and are typically defined
                            early in the acquisition phase to manage program risks. Although success
                            of the first operational test increases confidence in THAAD, we have
                            reported that good acquisition outcomes require high levels of knowledge
                            before significant decisions are made. The building of knowledge consists
                            of information that should be gathered at critical points over the course of
                            a program before committing to production.

                            To achieve the first remaining MDA knowledge point, THAAD must
                            conduct an integrated flight test against a medium-range ballistic missile
                            target. This test was originally scheduled for the second quarter of fiscal
                            year 2011, but after an air-launched target failure in December 2009 and
                            subsequent target availability issues, the agency moved the test to the
                            third quarter of fiscal year 2012. Later in fiscal year 2011, the test was
                            cancelled altogether because of budgetary concerns and test efficiency.
                            The agency now plans to test the objective in the first BMDS operational
                            test (FTO-01) in late fiscal year 2012. This test is not only planned as the
                            first against a medium-range target for THAAD, but it will also be the first
                            flight of the newly developed extended medium-range ballistic missile
                            target. Assuming several new “firsts” during this high-level operational
                            test poses significant additional risk for the agency and for achieving the
                            knowledge point.

                            The second knowledge point is to demonstrate THAAD’s Army Navy
                            /Transportable Radar Surveillance - Model-2 radar advanced
                            discrimination in terminal mode. This knowledge point was delayed from
                            the first quarter of fiscal year 2010 into the fourth quarter of fiscal year
                            2011 because of the same 2009 target issue. However, this knowledge
                            point was not accomplished in 2011 either. Additional changes to the
                            flight test plan in 2011 moved this objective to a flight test scheduled for
                            the third quarter of fiscal year 2013. As THAAD continues to gather data
                            from these developmental flight tests, the program continues to
                            concurrently produce interceptors, launchers, and associated equipment
                            for operational use. As a result, the program is at risk for discovering new
                            information that could lead to costly design changes and a need to retrofit
                            missiles either already in the production process or in inventory.




                            Page 99                                              GAO-12-486 Missile Defense
Appendix XII: GAO Contact and Staff
                  Appendix XII: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Cristina Chaplain, (202) 512-4841 or chaplainc@gao.gov
GAO Contact
                  In addition to the contact named above, David B. Best, Assistant Director;
Staff             Letisha J. Antone; Ivy Hübler; LaTonya Miller; Jonathan A. Mulcare;
Acknowledgments   Kenneth E. Patton; John H. Pendleton; Karen Richey; Ann Rivlin; Luis E.
                  Rodriguez; Steven Stern; Robert Swierczek; Hai V. Tran; and Alyssa Weir
                  made key contributions to this report.




(120996)
                  Page 100                                           GAO-12-486 Missile Defense
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.