oversight

Missile Defense: Alternate Approaches to Space Tracking and Surveillance System Need to Be Considered

Published by the Government Accountability Office on 2003-05-23.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Subcommittee on
             Strategic Forces, Committee on Armed
             Services, U.S. Senate


May 2003
             MISSILE DEFENSE

             Alternate Approaches
             to Space Tracking and
             Surveillance System
             Need to Be
             Considered




GAO-03-597
                                                May 2003


                                                MISSILE DEFENSE

                                                Alternate Approaches to Space Tracking
Highlights of GAO-03-597, a report to           and Surveillance System Need to Be
Subcommittee on Strategic Forces,
Senate Committee on Armed Services              Considered



The Department of Defense’s                     MDA purposely adopted a strategy that would evolve STSS over time rather than
Missile Defense Agency (MDA) is                 trying to make a big leap in its capability, deferring some requirements, and
developing a ballistic missile                  calling for competition in the development of the sensors aboard the satellite.
defense system designed to counter              Recent decisions, however, will limit MDA’s ability to achieve its original goals
a wide spectrum of ballistic missile
                                                as well as the knowledge that could be gained from its satellite demonstrations.
threats. A future element of this
system is the Space Tracking and                Specifically:
Surveillance System (STSS). STSS
will eventually be composed of a                •   MDA recently reduced its efforts to sustain competition by eliminating funds
constellation of satellites that will               set aside to procure an alternative satellite sensor from a competing
work together to detect and track                   contractor. It now plans to fund only efforts to design an alternative sensor.
missiles throughout all phases of                   If it chooses to pursue STSS as part of the missile defense system, STSS may
their flight. GAO was asked to                      end up being more expensive in the future because MDA could be locked
analyze MDA’s approach to                           into a single contractor for the design and production of the large
demonstrate capabilities for STSS.                  constellation of satellites.

                                                •   MDA decided to delay development and launch of new demonstrators in
To optimize MDA’s approach to                       order to focus on completing development of two legacy satellites. MDA
demonstrating space-based missile                   already knows that it would like to pursue different designs and different
tracking capabilities, GAO                          technologies for its target system given that the legacy satellites do not
recommends that MDA focus                           support a producible design. As a result, delaying work on the next
spending to assessing what needs                    generation of satellites will delay work that could offer a better basis from
to be done to complete work on                      which MDA could build an operational capability.
existing satellite components so
that it has a reasonable basis for its          •   MDA’s decision to launch in 2007 lacks important knowledge. MDA has
cost and scheduling estimates.
                                                    established a launch date before it has completed its assessment of the
GAO also recommends that MDA
assess alternatives to its current                  working condition of the equipment it needs to assemble in order to finish
strategy that may offer                             building the two satellites it would like to launch. As a result, it does not
opportunities to reduce risks and                   know the extent of work that must be done or how much it will cost because
gain more knowledge. In                             the number components found to be in working or non-working order have
commenting on a draft of this                       not yet been identified.
report, DOD partially concurred
with two of our recommendations                 MDA has considered pursuing alternate approaches, but all are constrained by
and concurred with two others. In               the need to participate in 2006-2007 missile defense tests. These approaches
its comments, DOD stated that it                include (1) launching the legacy satellites in 2008 instead of 2007 and (2)
would not be prudent to delay                   stopping work on the legacy satellites and focusing instead on developing new
launching satellites given the need
                                                demonstrators. Both of these approaches would enable MDA to inject more
to make overall ballistic missile
defense system sensor                           competition into the STSS program, reduce scheduling risks, and demonstrate
assessments.                                    more capabilities. However, they also have drawbacks; primarily, they would
                                                delay MDA’s ability to make informed trade-offs on missile defense sensors.



www.gao.gov/cgi-bin/getrpt?GAO-03-597.

To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Katherine
Schinasi at (202) 512-4841 or
SchinasiK@gao.gov.
Contents


Letter                                                                                 1
           Results in Brief                                                           2
           Background                                                                 4
           Capabilities Remain to Be Proven through Testing in Space                  8
           MDA’s Approach to Demonstrate STSS Capabilities                           10
           MDA May Not Be Able to Achieve Original Goals With Its Revised
              Strategy                                                               16
           Alternate Approaches May Garner More Knowledge                            23
           Conclusions                                                               26
           Recommendations for Executive Actions                                     26
           Agency Comments and Our Evaluation                                        27
           Scope and Methodology                                                     29

Appendix   Comments from the Department of Defense                                   31



Tables
           Table 1: DOD’s Attempts to Demonstrate or Field Space-Based
                    Missile Tracking Capabilities and Their Outcomes                   7
           Table 2: TRL Assessment for Critical Technologies for Legacy
                    Satellites                                                       15
           Table 3: Work That Must Be Done on Legacy Satellites                      15
           Table 4: Potential Risks for the Current STSS Strategy                    19
           Table 5: Comparison of MDA’s Strategy and Alternatives Not
                    Being Considered                                                 24


Figure
           Figure 1: Notional Configuration of STSS and the Ballistic Missile
                    Defense System                                                     5




           Page i                                             GAO-03-597 Missile Defense
Abbreviations

AEHF              Advanced Extremely High Frequency
DOD               Department of Defense
MDA               Missile Defense Agency
MSX               Midcourse Space Experiment
SBIRS             Space-Based Infrared System
STSS              Space Tracking and Surveillance System
TRL               Technology readiness level


This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                                      GAO-03-597 Missile Defense
United States General Accounting Office
Washington, DC 20548




                                   May 23, 2003

                                   The Honorable Wayne Allard
                                   Chairman
                                   The Honorable Bill Nelson
                                   Ranking Minority Member
                                   Strategic Forces Subcommittee
                                   Committee on Armed Services
                                   United States Senate

                                   The Department of Defense’s (DOD) Missile Defense Agency (MDA) is
                                   developing a ballistic missile defense system designed to counter a wide
                                   spectrum of ballistic missile threats. A future element of this system is the
                                   Space Tracking and Surveillance System (STSS), formerly known as
                                   Space-Based Infrared System-low (SBIRS-low). STSS will eventually be
                                   composed of a constellation of an as yet undefined number of satellites
                                   that will work together to detect and track missiles throughout all phases
                                   of their flight—from launch through midcourse and finally into reentry
                                   phase—and pass that information to other missile defense elements. The
                                   satellites will orbit earth at a low altitude, and they will carry infrared
                                   sensors and supporting subsystems based on sophisticated technologies.
                                   DOD currently expects to spend about $3.1 billion on STSS through 2009.

                                   DOD has initiated several programs and spent several billion dollars over
                                   the past 2 decades trying to develop a system to track missiles from space,
                                   but has not yet demonstrated certain critical capabilities. While some
                                   capabilities have been demonstrated through computer modeling and
                                   simulations, DOD believes all of the capabilities need to be proven in
                                   space before a large number of satellites can be acquired.

                                   Given the challenges associated with the program, you requested that we
                                   determine which capabilities still need to be demonstrated for STSS,
                                   analyze MDA’s approach for doing so, and identify alternative approaches
                                   for demonstrating the capabilities that may offer better outcomes.




                                   Page 1                                              GAO-03-597 Missile Defense
                   To be able to track missiles from space, MDA still needs to demonstrate
Results in Brief   that:

                   •   tracking information can be passed between sensors within a satellite;
                   •   tracking information can be passed between satellites;
                   •   missiles can be tracked in the midcourse phase of their flight;
                   •   data from two satellites at different locations and angles can be
                       successfully integrated, processed, and analyzed;
                   •   data from the satellites can be successfully passed to other space-, air-,
                       land-, and sea-based platforms;
                   •   satellites can operate and make some decisions autonomously; and
                   •   satellites can discriminate warheads from decoys.

                   Achieving these capabilities is technically challenging given the difficulties
                   associated with tracking cool objects against the cold background of
                   space as well as the harsh space environment and the short time frames
                   required to successfully identify, track, and intercept an incoming
                   warhead. Yet MDA believes most of these capabilities are needed to have a
                   system that can play a useful role in the overall missile defense system.
                   Two capabilities—autonomous operation and discrimination—do not
                   need to be demonstrated as quickly, but they would significantly enhance
                   a space-based missile tracking system.

                   MDA purposely adopted a strategy that would evolve STSS over time
                   rather than trying to make a big leap in capability, as had been the strategy
                   in the past. It deferred requirements that were too technically challenging
                   or beyond its immediate missile defense mission. MDA also called for
                   competition in the development and production of the sensors aboard the
                   satellite that would detect a missile launch (acquisition sensor) and track a
                   missile flight (tracking sensor) so that costs could be contained in the
                   future and the best technical solution could be pursued. In addition, MDA
                   opted to launch “demonstration” satellites before developing and
                   producing them in large numbers. This strategy helps to reduce risks
                   because it ensures technology is sufficiently mature and capabilities have
                   been demonstrated before a greater investment is made.

                   Recent decisions, however, will limit MDA’s ability to achieve its original
                   goals as well as the knowledge that could be gained from its satellite
                   demonstrations.

                   •   In order to take part in broader missile defense tests scheduled for
                       2006 and 2007, MDA decided to retrieve satellites and ground
                       components that were partially built in a previous effort and put into



                   Page 2                                               GAO-03-597 Missile Defense
    storage 4 years ago, complete the assembly of this equipment, and
    launch two satellites in 2007. Using these satellites in the 2007 test will
    help MDA to make trade-off decisions among missile defense systems.
    To be able to launch both satellites in 2007, however, MDA eliminated
    its plans to have two contractors compete in the production of satellite
    acquisition sensors. Instead, the program office now plans to fund the
    separate development of an alternative sensor design, but if the funds
    available do not allow for a meaningful design effort, it will be
    canceled. By choosing this approach, overall program costs could be
    higher because MDA could be locked in to using a single contractor for
    the production of a larger constellation of satellites.

•   In order to complete the development of the legacy satellites for launch
    in 2007, MDA also decided to delay the development and launching of
    new demonstrators. While MDA could learn a great deal about missile
    tracking capabilities from the legacy satellites, MDA already knows
    that it would like to pursue different designs and different technologies
    for its target system given that the legacy satellites do not support a
    producible design. As a result, delaying work on the next generation of
    satellites will delay work that could offer a better basis from which
    MDA could build operational capability.

•   MDA’s decision to launch in 2007 was based on limited knowledge.
    MDA established a launch date before it had completed its assessment
    of the working condition of the equipment it needs to assemble in order
    to finish building the two satellites it would like to launch. As a result,
    it does not know the extent of work that must be done or how much it
    will cost. More specifically, while MDA may know the cost to test the
    satellite component hardware, it does not know how many components
    will be found in nonworking order, nor the costs to fix these
    components. Moreover, MDA has identified a number of activities that
    will pose scheduling risks, such as (1) completing development of
    software for the ground segment and the infrared sensor software and
    (2) integrating the payload hardware and software. Though MDA has
    set aside funds to cover the risks, it will not have the knowledge it
    needs to really know if it can meet its target date until early 2004—
    when its assessment of the working condition of the existing
    equipment will be complete.

MDA has considered alternative approaches, but has not pursued any that
would not allow STSS to participate in 2006-2007 testing. Alternative
approaches not considered include (1) launching the legacy satellites in
2008 instead of 2007, which would allow another year to complete
development of the legacy satellites and procure a sensor of different



Page 3                                                GAO-03-597 Missile Defense
             design, and (2) stopping work on the legacy satellites and focusing instead
             on developing new technology, which would enable MDA to demonstrate
             and eventually field an operational capability sooner than its current
             approach. Both of these approaches would enable MDA to inject more
             competition into the STSS program, reduce scheduling risks, and
             demonstrate more capabilities. However, they also have drawbacks;
             primarily, they would delay MDA’s ability to make informed trade-offs
             among missile defense systems.

             We are making recommendations in this report that are intended to guide
             MDA in selecting the best approach for demonstrating missile tracking
             capabilities from space. DOD partially concurred with two of our
             recommendations and concurred with two others. In its comments, DOD
             stated that it would not be prudent to delay launching satellites given the
             need to make overall ballistic missile defense system sensor assessments.


             DOD is developing a ballistic missile defense system designed, over time,
Background   to counter a wide spectrum of ballistic missile threats. It will rely on space
             and ground-based systems to detect and track missiles; ground-, sea-, and
             air-based systems to intercept missiles in all stages of flight (which
             includes boost, midcourse, and reentry); and an overarching command
             and control system to plan and execute actions to counter enemy attacks.

             STSS will serve as the satellite network that will detect and track missiles
             throughout their flight and relay necessary cuing data to other elements in
             the missile defense system. The satellites will orbit the earth at low
             altitudes in order to allow for better missile viewing angles and high
             resolution.1 Each satellite will contain two infrared sensors—one to watch
             for bright missile plumes during the boost phase (acquisition sensor) and
             one to follow the missile through midcourse and reentry (tracking sensor).
             To provide for worldwide coverage, STSS would consist of a large
             constellation of satellites (between 21 and 28) as well as a supporting
             ground infrastructure. MDA has decided that significantly fewer satellites
             could be used to provide a meaningful capability based on the
             contributions and configurations of the other elements in MDA’s ballistic
             missile defense system. However, at this time MDA has not decided on the
             number of satellites that it plans to acquire.



             1
              The satellites will operate about 1,350 kilometers above the earth. By comparison,
             satellites in geo-synchronous orbit operate at about 36,000 kilometers.



             Page 4                                                       GAO-03-597 Missile Defense
Figure 1: Notional Configuration of STSS and the Ballistic Missile Defense System




History of Problems in                   DOD has had considerable difficulty for almost 20 years in developing a
Developing a Missile                     space-based missile tracking capability. Though it has spent several billion
Tracking System                          dollars through a series of development and acquisition programs since
                                         1984, it has not launched a single satellite or demonstrated any space-
                                         based missile tracking capabilities from space using technologies similar
                                         to those to be used by STSS. This is partly due to the technical challenges
                                         associated with building a system like STSS. For example, the satellites’
                                         sensors need to be able to track missiles in the midcourse phase of their
                                         flight, when missiles can no longer be easily detected by their bright
                                         plume. To do this, detection sensors must be cooled to very low
                                         temperatures for very long periods of time to detect and track a cool
                                         warhead against the cold background of space. In addition, systems
                                         aboard the satellite and on the ground must send that data to other missile



                                         Page 5                                             GAO-03-597 Missile Defense
defense systems quickly enough to allow them to target and destroy
incoming missiles and they must work under harsh environmental
conditions of space. This requires fast data processing and communication
links as well as materials that can withstand radiation and cold
temperatures.

Within this environment of significant technical challenges, DOD has not
yet established a program that it could execute. As we have reported2 over
the years, DOD and the Air Force did not relax rigid requirements to more
closely match technical capabilities that were achievable. Program
baselines were set based on artificial time and/or money constraints. Over
time, it became apparent that the lack of knowledge of program challenges
had led to overly optimistic schedules and budgets that were funded at
less than what was needed. Attempts to stay on schedule by approving
critical milestones without meeting program criteria resulted in higher
costs and more slips in technology development efforts. For example, our
1997 and 2001 reviews of DOD’s $1.7 billion SBIRS-low program, STSS’
immediate predecessor, showed that the program would enter the product
development phase with critical technologies that were immature and with
optimistic deployment schedules. In order to reduce costs, schedule,
performance, and technical risks, we recommended that DOD restructure
the program and analyze alternatives to satisfy critical ballistic missile
defense requirements in case SBIRS-low could not be deployed according
to the original acquisition strategy. DOD eventually restructured the
SBIRS-low program because of the cost and scheduling problems, and it
put the equipment it had partially built into storage. Table 1 further
highlights problems affecting space-based missile tracking programs since
1990.




2
 U.S. General Accounting Office, Defense Acquisitions: Space-Based Infrared System-low
at Risk of Missing Initial Deployment Date, GAO-01-6 (Washington, D.C.: Feb. 28, 2001)
and National Missile Defense: Risk and Funding Implications for the Space-Based
Infrared Low Component GAO/NSIAD-97-16 (Washington, D.C.: Feb. 25, 1997).




Page 6                                                    GAO-03-597 Missile Defense
Table 1: DOD’s Attempts to Demonstrate or Field Space-Based Missile Tracking Capabilities and Their Outcomes

 Program
 start         Program title      Purpose/mission                    Problems                         Outcome
 1990          Brilliant Eyes     Development program. Acquire       Program funding was sharply      No demonstration satellites
                                  and track missiles during late     reduced for Brilliant Eyes and   launched. The Congress
                                  boost and midcourse phases         other space-based systems.       transferred the program from
                                  and discriminate warheads from     The lack of funding hindered     the Ballistic Missile Defense
                                  decoys.                            the program from meeting its     Organization (now MDA) to the
                                                                     objectives.                      Air Force in 1993.
 1993          Space and Missile Development program. Acquire        Technical, funding, and          No demonstration satellites
               Tracking System   and track missiles and              management problems              were launched. In 1994, DOD
                                 discriminate warheads from          delayed the scheduled launch     consolidated its infrared space
                                 decoys during post-boost            of two demonstration             requirements and selected the
                                 phases.                             satellites.                      Space-Based Infrared System
                                                                                                      as a “system of systems”
                                                                                                      approach. Program was
                                                                                                      terminated.
 1996          Space-Based        Acquisition program. Support       Negative trends in cost,         Satellite demonstration effort
               Infrared System-   national and theater missile       schedule, and performance        canceled and the program
               low                defense by tracking missiles       estimates for the SBIRS-low      development risk reduction
                                  over their entire flights and      program resulted in DOD          phase restructured. Program
                                  discriminating warheads from       taking it off an acquisition     subsequently transferred from
                                  decoys in supporting the missile   track, and returning it to a     the Air Force to MDA.
                                  defense mission.                   sustained and deliberate
                                                                     technology development
                                                                     track.
 2002          Space Tracking     Development program. Acquire,      Not applicable. Program has      Not applicable. Program has
               and Surveillance   detect, and track ballistic        just begun.                      just begun.
               System             missiles through a series of
                                  increasingly capable and
                                  interoperable satellites and
                                  ground infrastructure.
Source: GAO.



Congressional Actions                     In October 2000, the Congress directed the Air Force to transfer the
                                          program to the Ballistic Missile Defense Organization (now MDA). The
                                          Senate Committee on Armed Services directed a study of alternatives to
                                          SBIRS-low as part of the fiscal year 2002 budget authorization process.
                                          These alternatives were to include ground-, sea-, and air-based sensors
                                          such as radar systems. MDA is currently expected to complete this study
                                          in 2003. The Committee directed that the report contain (1) an analysis of
                                          essential national missile defense requirements that SBIRS-low would
                                          fulfill and what alternative systems could also fulfill such requirements; (2)
                                          a quantitative assessment of national missile defense system performance
                                          without SBIRS-low or any alternative system; (3) a quantitative assessment
                                          of the national missile defense system performance with SBIRS-low and
                                          with each alternative system; (4) yearly cost estimates for SBIRS-low and
                                          of each alternative system beginning with fiscal year 2002, including all



                                          Page 7                                                         GAO-03-597 Missile Defense
                       previous fiscal years and all fiscal years through deployment of a fully
                       operational system; (5) a risk assessment of SBIRS-low and of each
                       alternative system; and (6) a qualitative assessment of the strengths and
                       weaknesses of SBIRS-low and each alternative system.

                       In addition, the Congress denied the $385 million DOD requested for the
                       program for fiscal year 2002, but it provided $250 million for a satellite
                       sensor technology effort, of which STSS would be a part. MDA was also
                       directed by the House Appropriations Committee to take STSS out of the
                       acquisition process and manage it as a sustained and deliberate
                       technology development effort.


                       DOD believes the following capabilities are needed to have a space-based
Capabilities Remain    missile tracking system that can play a useful role in the overall missile
to Be Proven through   defense system. These capabilities have not yet been demonstrated in
                       space, although DOD has had successes in demonstrating some related on-
Testing in Space       orbit capabilities through experimental satellites.

                       •   Acquisition-to-track hand over:
                           The ability of one satellite to detect or “acquire” a missile launch and to
                           transmit this data to its internal tracking sensor. The tracking sensor
                           would then continue tracking the missile after the acquisition sensor
                           has completed its detection function.

                       •   Satellite-to-satellite hand over:
                           The ability of two or more low-earth orbiting satellites to pass along
                           missile tracking data through two-way cross-links. This is a challenging
                           capability to demonstrate given the low orbits and flight path geometry
                           of the satellites. DOD has no military flight experience linking two or
                           more low-earth orbiting satellites through two-way cross-links. The
                           Iridium System, a private network of low orbiting satellites, can
                           establish cross-links, but it does not have the timeliness and low bit
                           error rate requirement of STSS. Further, only voice data (versus
                           analytical data) is transmitted from one fixed user to another (in
                           comparison with a moving satellite’s speed), there are ground stations
                           to assist in the process, and dropped links are not mission-critical as
                           they would be for STSS. DOD’s Milstar communication satellites use
                           cross-links, but they operate in a much higher orbit in fixed positions




                       Page 8                                               GAO-03-597 Missile Defense
    relative to one another, so this experience also does not translate
    directly to STSS.3

•   Midcourse tracking:
    The ability to (1) accurately track cool objects from thousands of
    kilometers away, which depends on sensitive sensors and accurate
    pointing capabilities, and (2) stereo tracking, which requires the
    capability to transfer and fuse data from multiple sensors in space
    while viewing the target missile from differing ranges and angles.
    Midcourse stereo tracking (two satellites reporting tracking
    information on one missile’s flight) is more desirable because it results
    in more precise information on the missile’s location. Some missile
    tracking capabilities were demonstrated during DOD’s 1996 Midcourse
    Space Experiment (MSX), which launched a satellite that collected
    data on a missile launch using optical sensors. However, this satellite
    did not conduct the same kind of functions that STSS would be
    required to perform, nor did it demonstrate all of the same
    technologies.4

•   Dual mission data processing:
    The ability to process and analyze data from two satellites that view
    one event from two different angles and locations.

•   Missile defense system integration:
    The ability to transmit and fuse STSS data with data provided by other
    space-, air, land-, and sea-based sensors—including legacy and
    emerging systems belonging to DOD and U.S. allies—and to use the
    results effectively in missile defense operations.




3
  The Milstar satellite communications system provides secure, jam resistant, worldwide
communications to meet essential wartime requirements for high-priority military users.
4
 The MSX spacecraft had 5 primary sensors with a total of 11 optical sensors, precisely
aligned so that activity of various targets can be viewed simultaneously with multiple
sensors. Four months after its launch, MSX successfully observed and tracked a 20-minute
ballistic missile test flight. MSX collected more than 800 seconds of high-quality data on
this missile test. MSX tracked missiles by relying on a sensor that was cooled using a
passive technique whereas the sensor on-board STSS is to rely on an active and mechanical
approach.




Page 9                                                       GAO-03-597 Missile Defense
                    There are two capabilities that DOD believes do not need to be
                    demonstrated as quickly, but they would significantly enhance a space-
                    based missile tracking system. They are:

                    •   Autonomous operation:
                        The ability of each satellite to operate as a self-contained unit and to
                        perform some decision-making functions before downlinking the
                        results. Because satellites will be moving at speeds of more than 15,000
                        mph relative to one another and across different orbital planes, as well
                        as moving in and out of the target missile’s range, calculations and
                        decisions must be made and data passed between sensors and satellites
                        within seconds. This is a desired future capability.

                    •   Discrimination:
                        Countering more advanced and sophisticated threats will require DOD
                        to be able to detect and track multiple objects and differentiate the
                        threatening warhead from decoys. Given technical challenges, DOD
                        deferred plans to achieve this capability for STSS. However, it plans to
                        achieve this capability for the missile defense system as a whole before
                        2015.


                    MDA could demonstrate space-based missile tracking capabilities by
MDA’s Approach to   either continuing earlier efforts or developing new satellites. At the
Demonstrate STSS    beginning of the STSS program, MDA chose to combine both, focusing
                    first on assembling and launching existing satellites and second on
Capabilities        developing new satellites. MDA also sought to avoid the mistakes made in
                    previous space-based missile tracking efforts by adopting a more flexible,
                    knowledge-based development strategy and calling for competition in
                    aspects of satellite development. Recently, MDA decided to launch the
                    first two demonstration satellites in 2007 and launch the first next
                    generation satellite in 2011.




                    Page 10                                            GAO-03-597 Missile Defense
Potential Approaches   MDA could demonstrate space-based missile tracking capabilities by
Available to MDA       either relying on legacy satellites or developing new satellites or a
                       combination of both.5 Specifically, MDA could complete work on satellite
                       and ground components that were partially built during the Air Force’s
                       effort (SBIRS-low) and put into storage 4 years ago. The satellites were
                       intended to serve as precursors to a constellation of operational satellites.
                       The capabilities that were built into the legacy components include
                       acquisition to track hand over, satellite-to-satellite hand over, stereo
                       midcourse tracking, and a limited capability to discriminate the types of
                       missiles launched.

                       MDA could also develop more capable and more robust satellites based on
                       newer technology. The satellites could be equipped with more accurate
                       sensors, faster data processing capacity, and longer lasting components.
                       The new satellites could also be designed to include features not available
                       to the existing satellites, such as adding an autonomous operations
                       capability. As with any approach, a ground segment capable of supporting
                       future demonstration satellites would be needed.

                       At the beginning of the STSS program, MDA decided it would pursue a
                       combination of both approaches. Specifically, it would complete the
                       assembly of satellite and ground components already in storage and
                       launch them to coincide with broader missile defense tests that would
                       take place in 2006-2007. This would allow MDA to establish a basis for
                       making trade-off decisions between space-, sea-, and air-based missile
                       defense sensors (for example, radar systems). MDA also decided to
                       develop a newer design, including more robust technologies envisioned
                       for the target system.




                       5
                        MDA could supplement both approaches by incorporating knowledge from the results of
                       tests of other satellites with missile tracking capabilities, though there are limitations to the
                       knowledge that could be applied to STSS. These include past tests such as the 1996 MSX
                       test discussed earlier and upcoming tests such as one MDA will be conducting in the near
                       future with a satellite build by Spectrum Astro to collect infrared data on intercontinental
                       ballistic missiles during the boost phase. Data to be collected under the contract will be
                       used to verify MDA’s future selection of a kill vehicle and tracking sensors for missile
                       engagements during the boost and ascent phases. The data will also build the foundation
                       for developing guidance and homing algorithms for MDA’s ground-based, boost-, and
                       ascent-phase interceptors. The satellite will be designed for an on-orbit lifetime of at least 1
                       year, with the objective of a 2-year on-orbit life, and is planned for launch into low earth
                       orbit in June 2004.




                       Page 11                                                          GAO-03-597 Missile Defense
MDA Sought to Avoid Past   At the onset of STSS, MDA adopted a more flexible product development
Mistakes with a More       approach that would maximize competition. For example:
Flexible Strategy
                           •   As with all missile defense elements, MDA called for a strategy that
                               would evolve STSS over time, rather than trying to make a big leap in
                               its capability. This means that new technology would be incorporated
                               into subsequent increments so that the product’s capability would
                               evolve over time. Our work has shown this approach reduces risk
                               because it introduces less new content and technology into a program’s
                               design and development effort. An evolutionary strategy also enables
                               developers to deliver a series of interim capabilities to the customer
                               more quickly.

                           •   Under its evolutionary approach, MDA deferred requirements that were
                               too technically challenging or beyond its immediate missile defense
                               mission. For example, MDA deferred the requirement for a
                               discrimination capability and has not decided whether the next STSS
                               development block will perform discrimination. It also deferred
                               requirements for STSS missions beyond missile defense, including
                               technical intelligence and battlespace characterization. Instead, these
                               missions would be addressed only to the extent that inherent or
                               residual capabilities could satisfy them. Our work has also shown that
                               programs are more successful when customers are willing to defer
                               requirements that demand more time or unproven technologies to
                               succeeding versions of the product. In essence, this flexibility helps to
                               ensure the product can be developed within available resources.6

                           •   MDA called for competition in the development and production of the
                               sensors onboard the satellite that would detect a missile launch
                               (acquisition sensor) and track a missile flight (tracking sensor) so that
                               costs could be contained in the future and the best technical solution
                               could be pursued. Specifically, one satellite would host sensors from
                               one subcontractor and another satellite would host sensors from a
                               competing subcontractor. Since contractors may use different
                               materials to build the infrared sensors, different detector technologies,
                               and different production methods, performance could vary
                               considerably. In describing the STSS approach, the Director of MDA
                               stated that injecting competition into sensor development was



                           6
                            U.S. General Accounting Office, Best Practices: Better Matching of Needs and Resources
                           Will Lead to Better Weapon System Outcomes GAO-01-288 (Washington, D.C.: March 8,
                           2001).




                           Page 12                                                    GAO-03-597 Missile Defense
                               necessary to reduce risks, particularly since MDA planned to award a
                               single contract to a prime contractor.

                           •   MDA decided to fly “demonstration” satellites before developing and
                               producing them in larger numbers. This practice enables MDA to see
                               how components and subsystems work together as a system in a
                               realistic environment before a greater investment of procurement
                               funds is made. Our work has also shown this to be a practice used by
                               successful programs.7


Recent Decisions on STSS   After MDA laid the foundation for approaching STSS, it decided to
Strategy Provide More      complete development and testing of two satellites and ground station
Time to Learn from First   equipment it acquired under SBIRS-low; launch the first satellite in 2006
                           and the second in 2007. Then, beginning in 2003, MDA would pursue
Two Demonstration          development of new demonstration satellites with more robust technology
Satellites                 and launch them beginning in 2010. It would launch and demonstrate a
                           series of satellites until it arrived at a design that could be used to support
                           a bigger constellation of satellites for the missile defense system. The Air
                           Force signed a contract with Northrop Grumman in August 2002 valued at
                           $868.7 million to (1) design, manufacture, and deliver the satellites and test
                           and check out the satellites on orbit; (2) develop a ground system; and (3)
                           conduct preliminary engineering analyses on the new demonstration
                           satellites.

                           In late 2002, MDA made significant changes to its strategy after it decided
                           to allocate less funding to the STSS program in order to fund other missile
                           defense elements. Specifically, it decided to continue the STSS program by
                           integrating and testing the existing satellites, but launch them in tandem in
                           2007 instead of sequentially in 2006 and 2007. Work on a single new
                           satellite would begin in 2003, instead of a pair of satellites as had been
                           originally planned. The program office plans to define the capabilities for
                           the follow-on satellite in mid-2003 and until then, the design, technologies,
                           and specifications for the new satellite will not be known. Work on the
                           new satellite will be stretched out, but MDA hopes to launch the new
                           satellite in 2011, only 1 year later than planned. The STSS program office
                           has programmed about $1 billion to complete work, launch, and operate



                           7
                            U.S. General Accounting Office, Best Practices: A More Constructive Test Approach Is
                           Key to Better Weapon System Outcomes, GAO/NSIAD-00-199 (Washington, D.C.: July 31,
                           2000).




                           Page 13                                                    GAO-03-597 Missile Defense
the legacy satellites and $1.3 billion for fiscal years 2004 through 2009 for
work on the new, follow-on satellite effort.

This change has some benefits in that certain capabilities could be
demonstrated over a longer on-orbit period of time. Under the original
strategy, satellite-to-satellite hand over, midcourse tracking, and dual
mission data processing would only have been demonstrated for about 8
months since this would be the amount of time that the satellites would be
fully operational together. By contrast, under the new strategy, these same
capabilities could be demonstrated for as long as 2 years since the
satellites will be launched in tandem. Moreover, under the previous
strategy, only partial integration with the missile defense test bed could be
demonstrated because data from the 2006 satellite would be processed off
line. There could also be delays in processing data because the ground
segment may not be fully integrated with the missile defense test bed until
2008. (The completion of ground connectivity between the STSS ground
station and the missile defense system does not yet have a definitive
schedule.) Since satellites are expected to be fully operable for 2 years,
integration could be demonstrated during the latter part of the second
satellite’s life. But this would limit the extent to which MDA can assess
STSS functions in the context of the overall system.

MDA is using tools to measure the maturity of critical technologies on the
legacy satellites. Specifically, as the table below shows, MDA has assessed
critical technologies for the legacy satellites using technology readiness
levels (TRL). TRLs measure maturity along a scale of one to nine. TRL 1
characterizes the least mature technologies representing the point where
scientific research begins to be translated into technologies basic
properties. A TRL 9 represents the most mature, an actual application of
the technologies in its final form under mission conditions. DOD guidance
states that a TRL 7, which means the system has been demonstrated in an
operational environment, is desired but that a TRL 6 represents acceptable
risk for a space-related technology to enter product development. At a
TRL 6, the subsystem or system has been demonstrated in a relevant
environment. MDA expects critical technologies on the legacy satellites to
be at a TRL 6 by June 2006. Moving from a TRL 5 to a TRL 6 to a TRL 7
represents a significant investment.




Page 14                                              GAO-03-597 Missile Defense
                                        Table 2: TRL Assessment for Critical Technologies for Legacy Satellites

                                                                                                TRL at                Projected TRL
                                         Technology area                                      fall 2002                at June 2006
                                         Acquisition sensor                                           5                           6
                                         Tracking sensor                                              5                           6
                                         Single-stage cryocooler                                      5                           6
                                         Two-stage cryocooler                                         5                           6
                                         Satellite communication cross-links                          6                           6
                                         On-board processor                                           6                           6
                                        Source: Air Force.



                                        Table 3 highlights the main activities that must be done to complete work
                                        on the legacy satellites. MDA developed a schedule to support the original
                                        plan to launch in 2006 and 2007. It is in the process of establishing the
                                        dates that these activities would need to be done by in order to support
                                        the new tandem launch date of 2007.

Table 3: Work That Must Be Done on Legacy Satellites

Area                                     Work
Systems engineering and assessment       This includes validating specifications and configurations, establishing performance
                                         baselines, and assessing and integrating ground test data analyses. Work under this
                                         component also includes analyzing the performance of the satellites after launch.
Sensor development                       This includes redesign efforts to the track sensor, along with assembly, and integration
                                         and test. The track sensor is on the critical path, and all work on this sensor needs to
                                         be completed in time to allow for integration onto the spacecraft. Under the original
                                         strategy, this meant the work should be done by July 2004. While work progresses on
                                         the sensor, software in support of the sensor will also be developed.
Spacecraft development                   Efforts include developing the spacecraft test bed, harness, and software. Activities
                                         also include integration and test of the satellite before and after launch. Satellite
                                         integration and test is on the critical path and was to begin in February 2005 and be
                                         completed in November 2005 under the original strategy. At that time the satellite will
                                         be shipped for launch.
Ground segment development               Activities include designing and developing the ground systems, installing hardware in
                                         the ground facilities, and integrating and testing the systems. The ground segment will
                                         involve more software development than the other satellite segments. Also,
                                         operational procedures will be developed and training on ground systems is to take
                                         place.
System test and operations planning      This includes developing the system and flight test plans. Readiness reviews are to be
                                         complete by September 2005, under the original strategy. Other activities planned
                                         include training and rehearsals, operations crew test training, test operations, and site
                                         and satellite operations.
                                        Source: Air Force Space and Missile Systems Center.




                                        Page 15                                                           GAO-03-597 Missile Defense
                           MDA’s approach to STSS will limit its ability to achieve its original goals
MDA May Not Be             for the program as well as the knowledge that could be gained from its
Able to Achieve            satellite demonstrations.
Original Goals With        •   First, the program office decided to forego pursuing production of on-
Its Revised Strategy           board sensors from competing contractors, as originally planned. As a
                               result, MDA will not have the ability to benefit from competition.

                           •   Second, to stay within its budget, the STSS program office made a
                               trade-off decision to develop only one new satellite rather than two and
                               to delay work on the new satellite. This decision will delay MDA’s
                               ability to learn about new satellite designs and technologies needed for
                               an operational capability.

                           •   Third, MDA’s decision to launch in 2007 is not knowledge-based. At this
                               point, it does not know the extent of work that needs be done on the
                               legacy satellites since it has not completed its assessment of the
                               condition of the components that have been in storage for 4 years.
                               Moreover, it is uncertain as to whether some of the activities it does
                               know it must undertake in order to integrate and test legacy satellite
                               systems can be completed in time for the 2007 launch. To its credit,
                               MDA has set aside extra funds for tasks that present particular
                               scheduling risk. But until it knows more about the working condition
                               of the satellite hardware and software, it cannot be sure of its ability to
                               deliver on time.


Limiting Competition       MDA’s decision to tentatively fund the design, but not the production, of a
Could Increase Long-term   sensor from a competing contractor as part of the first effort will
Costs and Risks            potentially increase long-term costs and risks. Specifically, it will preclude
                           MDA from gaining knowledge about competing sensors and selecting the
                           one that offers the best capability. Moreover, it precludes another
                           contractor from gaining experience in building infrared sensors,
                           potentially hampering MDA’s ability to compete work in the future and
                           making the system more costly over the long term.

                           We recently reported, for example, that DOD’s effort to develop a new
                           generation of communication satellites (the Advanced Extremely High
                           Frequency (AEHF) satellite program) incurred significant cost and
                           scheduling problems partly because of its decision to consolidate
                           contractors into one team. In commenting on our findings, DOD admitted
                           that its major failing with the program was the acceptance of this team’s
                           proposed approach of an overly optimistic performance, schedule, and



                           Page 16                                              GAO-03-597 Missile Defense
                             profit baseline. Part of the reason DOD chose to limit competition was to
                             launch a new satellite as soon as possible. In hindsight, it recognized that
                             this worsened the situation because the contracting team could not follow
                             through on its original promises, and DOD had nowhere else to turn to for
                             a better solution.

                             Also, under the SBIRS-low program, the Air Force awarded a contract in
                             1995 for the development of two technology demonstration satellites to
                             the same contractor. It later recognized the need for competition for the
                             demonstration to lower costs and reduce schedule and technical risks
                             because otherwise only one contractor would gain the experience and
                             knowledge needed to build the full constellation of satellites. To prevent
                             similar problems, in 1996, the Air Force requested offers for an alternative
                             system concept from one or more contractors to demonstrate and validate
                             critical design issues. The goal of this effort was to stimulate competition,
                             resolve key technical and production risks, and create a plan to ensure
                             SBIRS-low deployment schedules can be executed.


Delaying Work on New         The decision to launch the two existing satellites in tandem in 2007,
Design Will Delay Work       instead of 2006 and 2007, will provide MDA with more time to assess the
Toward Operational           working condition of the satellites’ components and to complete work on
                             assembling the satellites. It will also offer more time for MDA to assess
Capability                   capabilities such as satellite-to-satellite hand over since the two satellites
                             will be operating as a pair for a longer period of time. However, MDA
                             already knows that it would like to pursue different designs and different
                             technologies for its target system as the legacy system is based on
                             technologies that are more than a decade old. Its recent decision to delay
                             work on a new satellite will merely delay the opportunity to learn more
                             about a design that could offer a future operational capability and a better
                             basis for making trade-off decisions among missile defense sensors. In
                             fact, for the next few years, resources will largely be devoted to work on
                             the legacy satellites versus the new satellites. In 2004, about 92 percent of
                             STSS funds will go to the legacy satellites versus 8 percent to the new
                             ones.


Decision to Launch in 2007   MDA has not yet completed its assessment of the working condition of
Is Not Knowledge-Based       satellite hardware and software, so it does not know the full extent of
                             work that needs to be done on the legacy satellites. Moreover, MDA
                             officials recognize that through the process of testing, assembling, and
                             integrating the hardware and software components, unforeseen problems
                             could arise that may make it more difficult to complete the satellites in


                             Page 17                                              GAO-03-597 Missile Defense
time for the 2007 launch. These problems could include completing ground
segment software and the infrared sensor software as well as integrating
payload hardware and software. Table 4 highlights these and other
activities MDA believes have costs and scheduling risks.

MDA and the STSS program office have set aside $47.2 million in funding
to address these potential problems and also drafted risk mitigation plans,
which are to be updated in 2003. This money will be used for independent
review teams that can help the program office assess what work needs to
be done as well as other resources (for example, more personnel) and
activities (for example, contingency planning) needed to prevent
scheduling delays. The program office has also identified the need for $26
million in additional funding to more fully address the risks involved with
hardware and software issues, which has been funded from the
contractor’s management reserves. Program officials said that if the costs
prove to be too high after the assessment of the working condition of the
satellites, they will terminate the legacy effort and move onto the new
demonstrators.




Page 18                                            GAO-03-597 Missile Defense
Table 4: Potential Risks for the Current STSS Strategy

                                                                                                                                  Risk mitigation
                                                                                                                                         funding
 Risk                                    Description                                    Potential effect                               (millions)
 1. Condition of the                     Hardware and software must be tested to        If hardware and software do not test as              $ 9.9
     satellite hardware                  determine their working condition after        expected, launch delays could be
     and software                        being in storage for 4 years.                  significant, particularly for the first
                                                                                        launch.
 2.     Completion of the                The schedule to design, develop, and test      If the 31-month schedule does not play               $ 5.0
        ground segment                   the ground software is aggressive and          out, software costs could increase, and
        software                         requires these tasks to be done                the ground segment might not meet
                                         concurrently. Also, the requirements for the   the scheduled launches for the existing
                                         ground segment software were significantly     satellites.
                                         increased.
 3.     Uncertainty of the               Integrated flight test scenarios and targets If problems surface, costs could                      $14.1
        STSS Block 2006                  have not been defined or analyzed; STSS      increase, schedules could be delayed,
        performance                      performance analyses are incomplete; and     and STSS could provide less on-orbit
                                         infrared sensor tests will not be completed  performance and utility for the ballistic
                                         for 2 more years.                            missile defense system test bed.
 4.     Completion of the                Software requirements are undefined and      If the issues are not resolved, costs                 $14.6
        infrared sensor                  software interface issues could require      could increase, and payload testing
        software                         software redesign.                           and delivery could be delayed, which
                                                                                      would delay the launch(es).
 5.     Integration of the               Many integration and test activities         If this work does not proceed as                       $ 2.9
        payload hardware                 conducted in serial must be successful. The planned, costs would increase and
        and software                     schedule has little slack for test equipment delivery of the payloads would be late,
                                         or component failures.                       which could delay the launch(es).
 6.     Thermal                          Thermal performance and its impact on        If lower than expected thermal                         $ 0.8
        modifications to the             long-wave infrared performance will not be performance occurs, costs would
        infrared sensor                  known until the sensor has been built and    increase and satellite-tracking
        payload                          tested.                                      capabilities would be reduced.
Source: Air Force Space and Missile Systems Center.



Risk 1: Working Condition of                             MDA will not know the extent of work needed on satellite hardware and
Satellite Hardware and                                   software until late 2004. When the Air Force canceled the flight
Software Has Not Been                                    demonstration system satellites in 1999, the development of hardware and
Assessed                                                 software was not completed, and the problem areas that had been
                                                         identified had not been fully documented, leaving a knowledge gap that
                                                         will need to be closed before MDA proceeds with further development.
                                                         The legacy components have been retrieved from storage; however, they
                                                         still need to be tested to determine their working condition. In November
                                                         2002, testing started on the first satellite’s payload components (including
                                                         the acquisition and tracking sensors). As part of the SBIRS-low effort, the
                                                         tracking sensor was tested last year and found to be in working order. This
                                                         testing should be completed in October 2003. The satellite’s spacecraft
                                                         hardware has been visually inspected, and it will be tested from May 2003
                                                         to September 2003. The spacecraft hardware for the second satellite has
                                                         also been visually inspected. Under MDA’s schedule for its original


                                                         Page 19                                                       GAO-03-597 Missile Defense
                                  strategy, testing for the second satellite was planned for September 2003
                                  through November 2003, and payload testing was planned for December
                                  2003 to August 2004.

Risk 2: Time for Completing       MDA expects that many tasks needed to design, develop, and test the
Ground Segment Software May       ground software will need to be done concurrently to meet the new
Be Insufficient                   schedule. The effort will also be complicated by the fact that the
                                  requirements for the ground software significantly changed in 2002, at the
                                  time of the program restructure. Specifically, the software will need to
                                  support the future generation and eventual larger constellation of
                                  satellites, whereas the ground software associated with the legacy
                                  satellites was originally supposed to support two demonstration satellites.
                                  Program officials acknowledged that they would not have a high level of
                                  confidence in the software cost estimate or software schedule until the
                                  preliminary design review for the software occurred, which occurred in
                                  March 2003.

Risk 3: Critical Tests for        Critical tests for assessing preflight performance will not be done and
Assessing Whether                 analyzed until MDA is close to the point where it needs to begin launch
Performance Is Acceptable Will    preparations. For example, integrated flight test scenarios and targets
Not Be Done Until Shortly         have not yet been defined or analyzed and, as originally scheduled,
Before Launch Preparations        infrared sensor performance tests will not be completed until mid-
                                  calendar year 2005—a few months before MDA would need to stop work
                                  on the satellites and begin launch preparations. Program officials pointed
                                  to other factors that will make preparing for performance tests difficult,
                                  including the fact that MDA had not yet identified interfaces with other
                                  missile defense elements or integrated test plans and schedules.

                                  Our reports have shown that pushing such testing to the latter stages of a
                                  development program is very risky. Specifically, it prevents programs from
                                  using test results to improve design. It also raises the risk that problems
                                  will not be discovered until a point where it becomes very costly and time-
                                  consuming to fix them. Moreover, our reports have also shown that when
                                  testing occurs at latter stages, the amount of testing that is actually
                                  conducted is significantly less than planned.8

Risks 4, 5, and 6: Considerable   Considerable work needs to be done on the infrared sensors and software
Work Remains to Be Done on        within a short period of time. Three activities are particularly critical: (1)
Infrared Sensors and Software     completing software development for the acquisition sensor, (2)


                                  8
                                      GAO/NSIAD-00-199.




                                  Page 20                                              GAO-03-597 Missile Defense
                       integrating payload hardware and software, and (3) modifying the tracking
                       sensor to accommodate requirements for long-wave infrared performance.

                       •     Software development for the infrared sensors is on the critical path
                             and must be completed in time to support integration and testing of the
                             sensors. Software originally developed for the legacy satellites’ sensors
                             was never completed, and the sensors’ software requirements are not
                             completely defined, which may delay software development. According
                             to the program office, the lack of time scheduled to perform early
                             software testing on sensors could result in a delay in detecting and
                             resolving errors. Most of the $14.6 million risk mitigation funding in
                             this area has been earmarked to address the lack of early software
                             testing. Also, software interface issues could require a redesign of the
                             software.

                       •     Integration of the payload hardware and software will be complex
                             because many serial integration and test activities must be successful,
                             and the schedule has little slack for test equipment or component
                             failures, according to the program office.

                       •     The tracking sensor needs additional modifications to accommodate
                             long-wave infrared performance requirements. However, the impact of
                             the modifications will not be known until the sensor has been built and
                             tested. (The sensor was originally designed for mid-wave infrared
                             performance. In the middle of the SBIRS-low program, the long-wave
                             infrared requirements were imposed on the program. While
                             modifications were made to meet this requirement, the sensor still
                             generates more heat than the satellite coolers were designed to handle.
                             As such, the sensor still needs thermal modifications to improve its
                             long-wave infrared performance.)

Software Development   Since software development is a risk in many areas of the STSS program,
Schedules Tend to Be   MDA faces an overriding challenge in accurately predicting what work will
Optimistic             need to be done in developing software related to the program. Reports
                       show that this is a significant problem for many space and other weapon
                       system programs—commercial and military. For example, in a series of
                       studies completed in the 1990s, the Standish Group9 found that the average
                       cost overrun was 189 percent, the average schedule overrun was 222
                       percent of the original estimate, and, on average, only 61 percent of the
                       projects were delivered with originally specified features or functions


                       9
                           The CHAOS Report, the Standish Group International, Inc. (West Yarmouth, Mass.: 1995).




                       Page 21                                                       GAO-03-597 Missile Defense
attributable to software development. In November 2000, the Defense
Science Board reported that the majority of problems associated with
DOD software development programs are a result of undisciplined
execution. The Board found that troubled programs lacked well thought-
out, disciplined program management and/or software development
processes. Meaningful cost, schedule, and requirement baselines were also
lacking, making it virtually impossible to track progress against them.10

We have stated in previous reports that software development schedules
were optimistic for DOD’s AEHF satellite system and SBIRS-high program.
For example, the Air Force originally estimated that the AEHF payload
and spacecraft bus required approximately 257,000 lines of software code,
but as the requirements and capabilities of the satellite system were better
understood, the estimate grew to approximately 466,000 lines of software
code. AEHF’s ground segment also increased from about 1.1 million lines
of software code to nearly 1.7 million. In early 2002, during the last SBIRS-
high program restructure, Air Force officials estimated that the amount of
ground segment software had grown 48 percent, while the amount of
space segment software had grown 28 percent.

MDA and the prime contractor recognize that software presents a risk
across the board for STSS. The prime contractor has decided to manage
the work on the STSS program based on milestones that are 6 months
earlier than the contractual satellite launch date milestones. This means
that delays attributable to software development up to 6 months will not
directly affect the satellite launch schedule.




10
 Report of the Defense Science Board Task Force on Defense Software, Office of the
Under Secretary of Defense for Acquisition and Technology (Washington, D.C.: Nov. 2000).




Page 22                                                     GAO-03-597 Missile Defense
                       There are other approaches MDA could pursue, but they have not been
Alternate Approaches   considered because they would not allow STSS to participate in 2006-2007
May Garner More        missile defense testing. These include (1) planning the launch of the legacy
                       satellites for 2008 and (2) canceling work on completing the existing
Knowledge              satellites and focusing solely on developing new satellites. These
                       approaches would enable MDA to inject competition into the STSS
                       program, reduce scheduling risks, and demonstrate more capabilities.
                       Both approaches also have drawbacks, primarily, they would delay MDA’s
                       ability to make informed trade-offs between STSS and other competing
                       surveillance and tracking capabilities, such as ground-, sea-, and air-based
                       radar systems. Table 5 compares these alternatives in terms of achieving
                       capabilities to MDA’s original and current strategies.




                       Page 23                                            GAO-03-597 Missile Defense
Table 5: Comparison of MDA’s Strategy and Alternatives Not Being Considered




Delaying Launches of                   One approach not being considered involves delaying the launch of the
Existing Satellites Could              legacy satellites until 2008. This approach offers several advantages over
Reduce Scheduling Risk                 the current strategy.
and Offer More Knowledge               •   First, it would reduce program risks by allowing more time to complete
                                           the development and testing of satellite hardware and software that
                                           have been in storage for 4 years, and to complete software
                                           development and testing for the ground segment.




                                       Page 24                                            GAO-03-597 Missile Defense
                           •   Second, it would allow time to complete integration of the ground
                               segment with the missile defense test bed (scheduled for 2008) and
                               ensure that both satellites would have enough on-orbit life remaining
                               so that the satellites and ground segment could be tested together
                               while fully integrated into the ballistic missile defense system test bed.
                           •   Third, it would allow MDA to fund both the design and production of a
                               competing contractor’s acquisition sensor. This would ensure that
                               competition remains viable for the development of future series of
                               satellites, and it is key to MDA getting the best prices and technical
                               solution.
                           •   Fourth, satellites launched in 2008 will likely still have some residual
                               capability when the new satellite is launched in 2010, allowing them to
                               interact together to provide increased knowledge.

                           One drawback is a potential delay in demonstrating capabilities and
                           technologies, since MDA’s current plan would begin to demonstrate some
                           capability in 2007. But more importantly, this approach will delay the
                           benefit of incorporating on-orbit lessons learned into the upgraded design,
                           because this newer design will be well underway by the time the satellites
                           are launched. Instead, MDA will have to wait for a future effort before it
                           can incorporate these lessons learned.


Focusing Solely on         MDA is also not considering focusing solely on developing and
Developing New             demonstrating new satellites that can offer operational capability once a
Demonstration Satellites   limited or full constellation is fielded. However, this approach could
                           demonstrate most capabilities needed for an operational system with at
Can Allow MDA to           least two satellites in orbit at the same time for some duration.
Develop More Robust        Specifically, satellite-to-satellite hand over could be demonstrated since
Satellites Quicker         the new satellites would be designed to be compatible. Additionally, the
                           new satellites could be fully integrated with the missile defense test bed.
                           Midcourse stereo tracking and STSS mission dual data processing would
                           also be demonstrated. Discrimination capability could be demonstrated,
                           depending on the design selected.

                           There are other benefits of pursuing the newer technologies beyond
                           meeting these capabilities. The newer technology satellites would have
                           increased lifetimes. In addition, the satellites’ sensors would likely be
                           more sensitive and able to detect cooler targets. Software upgrades would
                           continue to evolve to meet a newer generation of needs with the new
                           technology satellites. Moreover, this approach would allow more time to
                           test with the longer on-orbit life expected from these newer satellites.




                           Page 25                                              GAO-03-597 Missile Defense
                      Finally, MDA could reach a decision to field an operational capability
                      sooner than with any other approach.

                      There are also drawbacks to this approach. There could be at least a 2-
                      year delay in demonstrating capability. In addition, the technology risk
                      would be greater because the critical technology for the new satellites is
                      less mature. For example, the TRLs for the new satellites currently range
                      from 4 to 5, whereas the TRLs for the existing satellites, according to the
                      Air Force, range from 5 to 6.11


                      Over about the last 20 years, DOD has invested billions of dollars to
Conclusions           develop a missile tracking capability from space. Past efforts show that a
                      heavy focus on meeting schedules can debilitate an effort to the point of
                      failure. Yet DOD is at risk of repeating past mistakes because it has made
                      decisions that are largely focused on meeting its 2007 launch date rather
                      than making sure the satellites and ground station can work as intended
                      and that it can gain the maximum knowledge at the lowest cost. Given the
                      research and development nature of the program at this point, MDA has
                      the ability to study and consider alternative ways of moving forward with
                      the existing satellite components with greater emphasis on gaining
                      knowledge from its demonstration satellites. If research and development
                      is not the primary goal and operational capability is, MDA should stop its
                      investment in completing the existing satellites and concentrate on
                      developing new satellites.


                      To better ensure the Missile Defense Agency’s approach to validate space-
Recommendations for   based sensors and technologies for missile acquisition, tracking, and
Executive Actions     discrimination, we recommend that the Secretary of Defense direct the
                      Director, MDA to take the following actions.

                      •    Focus spending on its STSS contract to assessing the working
                           condition of the legacy satellites and what additional work is
                           necessary to develop, test, and launch the existing satellites so that
                           MDA has more knowledge on which to build cost and schedule
                           estimates.


                      11
                        This means that for the new satellites, components are testable but do not exist in their
                      final assembled configuration. For the existing satellite hardware, prototypes are available
                      that are very close in their final form, fit, and function, and performance has been
                      demonstrated in a relevant environment.




                      Page 26                                                        GAO-03-597 Missile Defense
                     •   Use this assessment to conduct a broader analysis of alternative
                         approaches, including the possibility of delaying launches to 2008 as
                         well as dropping the development of the existing satellite components
                         and focusing instead on developing demonstration satellites based on
                         later generation technology.
                     •   Further, use this assessment to find ways to ensure that competition at
                         the sensor level is part of all efforts to develop missile tracking
                         capabilities.
                     •   If this assessment concludes that MDA should follow a different path
                         for STSS, renegotiate the STSS contract to account for this change.


                     We received written comments on a draft of this report from the Director
Agency Comments      of Defense Systems within the Office of the Under Secretary of Defense
and Our Evaluation   for Acquisition, Technology, and Logistics. DOD partially concurred with
                     our first two recommendations and concurred with our third and fourth
                     recommendations. In response to the first two recommendations, DOD
                     noted that efforts to develop, test, and launch the legacy satellite hardware
                     is well understood and on contract, and that a delay in launching the first
                     STSS satellites is not prudent, given the overall missile defense sensor
                     assessments that are to be made. In response to the third and fourth
                     recommendations, DOD agreed that the sensor payload competition is
                     central for risk mitigation and that if DOD pursued a different strategy,
                     contract adjustments would be warranted. DOD also offered additional
                     corrections and suggestions to clarify our draft report, which we have
                     incorporated as appropriate. DOD’s comments appear in appendix I.

                     In responding to our first recommendation, DOD agreed with the need to
                     assess the working condition of the legacy satellite hardware, but did not
                     explicitly concur with the need to focus spending on this assessment. DOD
                     further commented that its efforts to develop, test, and launch the legacy
                     hardware is well understood. This comment, however, is based on the
                     assumption that all of the hardware will be found in working condition
                     and performing within acceptable technical parameters. Our point is that
                     the condition of the legacy hardware will not be known until after all of
                     the hardware checks have been conducted. Because its knowledge of the
                     condition of the legacy satellites is not complete, MDA’s decisions to
                     develop older technology versus pursuing new technology and to launch
                     legacy satellites in 2007 may not have the expected results. If key satellite
                     components are found to be in unacceptable working condition, MDA may
                     be forced to spend more time and money than currently estimated to
                     execute its strategy.




                     Page 27                                             GAO-03-597 Missile Defense
In responding to our second recommendation, DOD asserted that it had
already conducted a broader analysis of alternative approaches to
development of space-based sensor support to the missile defense system.
As noted in our report, however, this analysis did not include the
alternative of launching legacy satellites in 2008 instead of 2007 or
focusing solely on development of new technology. We also disagree with
DOD’s comment that a delay in launching the first STSS satellites is not
prudent, given the overall ballistic missile defense system sensor
assessments that are to be made. First, MDA is striving to launch the STSS
satellites in 2007 to support the 2007 test bed in order to allow DOD to
make informed decisions about the composition of the missile defense
sensor architecture. However, launching both legacy satellites in 2007 also
has some long-term affordability consequences. For example, to be able to
fund a launch in 2007, MDA has decided not to fund the procurement of a
satellite sensor from a competing contractor. Instead, MDA plans to fund
the development of an alternative sensor design from a competing
contractor, if the funds available are sufficient for meaningful design
work. By reducing competition, MDA may well face higher long-term costs
to develop STSS because it may have to rely on a single contractor.
Moreover, competition will enable MDA to pursue the best technical
solution for STSS. Second, a primary goal of the initial STSS satellites is to
demonstrate key capabilities that have never before been demonstrated
from space. By adopting a strategy designed to meet the target launch
date, however, MDA will be constrained in its ability to learn about these
capabilities. For example, it will not be able to fully assess how well STSS
will interact with other missile defense systems because the legacy
systems will only be partially integrated with the missile defense test bed.
Third, the history of the STSS program warrants a broader assessment of
alternative investment approaches. The legacy satellites that MDA is
relying on experienced technical and schedule difficulties as well as
significant cost growth when they were developed under the SBIRS-low
program, STSS’s precursor. The SBIRS-low program as a whole was also
schedule driven, it faced technical challenges, and although almost
$2 billion was spent on this program, not a single satellite was launched.
The demonstration portion of the program was eventually canceled in
1999. To avoid similar problems, we believe that MDA should examine
approaches that offer ways to maximize competition and reduce cost and
scheduling risks even if that means a delay in its assessment of STSS’s
participation in the missile defense test bed.




Page 28                                              GAO-03-597 Missile Defense
              To determine what capabilities DOD still needs to demonstrate in support
Scope and     of a missile tracking capability from space, we reviewed briefings of
Methodology   program goals, acquisition and test plans, management reports, and
              internal memoranda relevant to the development of STSS. Specifically, we
              reviewed the system element reviews, MDA’s Director Guidance, and the
              element capability specification, from MDA and the Air Force’s Space and
              Missile Systems Center. We also held discussions on STSS capabilities
              with officials at MDA, the Space and Missile Systems Center in Los
              Angeles, California, and Northrop-Grumman Space Technology in
              Redondo Beach, California. We also reviewed documentation from the
              Director, Operational Test and Evaluation, Office of the Secretary of
              Defense, regarding the SBIRS-low program and its risks.

              To identify and assess DOD’s prior and current approaches for
              demonstrating missile-tracking capabilities, we reviewed the STSS
              September and December 2002 System Element Reviews, program
              briefings, and the STSS contract. We also held discussions with officials at
              the Space and Missile Systems Center. In deriving the different
              approaches, we relied on program briefings and supplemented this
              information with our own institutional knowledge and experience in
              reviewing space systems. Through trial and error for estimation purposes,
              we extended launch dates, launched satellites in tandem, or both, to see
              which could result in increased knowledge to the program. Through
              briefings with officials from the Space and Missile Systems Center and our
              own assessments, we determined what the advantages and disadvantages
              would be to each approach.

              To determine the risk areas involved with the different alternatives for
              accomplishing the STSS mission, we reviewed our prior work on the STSS
              program and the STSS System Element Review and launch schedules and
              discussed with officials at the Missile Defense Agency, the Air Force Space
              and Missile Systems Center, and Northrop-Grumman Space Technology,
              the work needed to finish development and testing of the existing
              demonstration satellites. We reviewed schedule and funding information
              for developing both hardware and software for the demonstration
              satellites, to include whether components can be fully integrated, tested,
              and validated before launch. We also reviewed our prior reports and
              testimonies on practices characterizing knowledge-based acquisition
              processes.

              We performed our work from July 2002 through March 2003 in accordance
              with generally accepted government auditing standards.



              Page 29                                            GAO-03-597 Missile Defense
We plan to provide copies of this report to the Chairmen and Ranking
Minority Members of the Senate Committee on Armed Services; the Senate
Committee on Appropriations, Subcommittee on Defense; the House
Committee on Armed Services; and the House Committee on
Appropriations, Subcommittee on Defense; the Secretary of Defense; and
the Director, Missile Defense Agency. We will make copies available to
others upon request. In addition, the report will be available at no charge
on the GAO Web site at http://www.gao.gov/.

If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Key contributors to this report were Cristina
Chaplain, Art Gallegos, Tony Beckham, Joseph Dewechter, Dave Hubbell,
Sigrid McGinty, Karen Sloan, Jim Solomon, Hai Tran, and Randy Zounes.




Katherine V. Schinasi
Director, Acquisition and Sourcing Management




Page 30                                            GAO-03-597 Missile Defense
                   Appendix: Comments from the Department of
Appendix: Comments from the Department
                   Defense



of Defense




         Page 31                                               GAO-03-597 Missile Defense
          Appendix: Comments from the Department of
          Defense




Page 32                                               GAO-03-597 Missile Defense
                     Appendix: Comments from the Department of
                     Defense




(120156)
           Page 33                                               GAO-03-597 Missile Defense
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548