oversight

Homeland Security: DHS and TSA Face Challenges Overseeing Acquisition of Screening Technologies

Published by the Government Accountability Office on 2012-05-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            United States Government Accountability Office

GAO                         Testimony
                            Before the Committee on Oversight and
                            Government Reform and Committee on
                            Transportation and Infrastructure, House
                            of Representatives
                            HOMELAND SECURITY
For Release on Delivery
Expected at 1:00 p.m. EDT
Wednesday, May 9, 2012



                            DHS and TSA Face
                            Challenges Overseeing
                            Acquisition of Screening
                            Technologies
                            Statement of Steve Lord, Director
                            Homeland Security and Justice Issues




GAO-12-644T
                                              May 9, 2012

                                              HOMELAND SECURITY
                                              DHS and TSA Face Challenges Overseeing
                                              Acquisition of Screening Technologies
Highlights of GAO-12-644T, a testimony
before the Committees on Oversight and
Government Reform and Transportation and
Infrastructure, House of Representatives.



Why GAO Did This Study                        What GAO Found
Within DHS, TSA is responsible for            GAO’s past work has found that the Department of Homeland Security (DHS)
developing and acquiring new                  and the Transportation Security Administration (TSA) have faced challenges in
technologies to address transportation-       developing and meeting program requirements when acquiring screening
related homeland security needs.              technologies. GAO’s past work has demonstrated that program performance
TSA’s acquisition programs represent          cannot be accurately assessed without valid baseline requirements established
billions of dollars in life-cycle costs and   at the program start. In June 2010, GAO reported that more than half of the 15
support a wide range of aviation              DHS programs GAO reviewed awarded contracts to initiate acquisition activities
security missions and investments,            without component or department approval of documents essential to planning
including technologies used to screen
                                              acquisitions, setting operational requirements, or establishing acquisition
passengers and checked baggage
                                              program baselines. At the program level, in January 2012, GAO reported that
such as AIT and EDS, among others.
GAO’s testimony addresses three key
                                              TSA did not fully follow DHS acquisition policies when acquiring advanced
DHS and TSA challenges identified in          imaging technology (AIT)—commonly referred to as a full body scanner that
past work: (1) developing and meeting         identifies objects or anomalies on the outside of the body—which resulted in
technology program requirements, (2)          DHS approving full AIT deployment without full knowledge of TSA’s revised
overseeing and conducting testing of          specifications. In July 2011, GAO reported that in 2010 TSA revised its explosive
new screening technologies, and (3)           detection systems (EDS) requirements to better address current threats and
identifying acquisition program               planned to implement these requirements in a phased approach; however, GAO
baselines (or starting points), program       reported that some number of the EDSs in TSA’s fleet were configured to detect
schedules, and costs. This statement          explosives at the levels established in 2005 while the remaining ones were
will also discuss recent DHS and TSA          configured to detect explosives at 1998 levels and TSA did not have a plan with
efforts to strengthen TSA’s investment        time frames needed to deploy EDSs to meet the current requirements.
and acquisition processes. This
statement is based on reports and             GAO also reported DHS and TSA challenges in overseeing and testing new
testimonies GAO issued from October           technologies. For example, in January 2012, GAO reported that TSA began
2009 through April 2012 related to            deploying AIT before it received approval for how it would test AIT. Contrary to
TSA’s efforts to manage, test, and            DHS’s acquisition guidance, TSA approved AIT for deployment prior to DHS’s
deploy various technology programs.           approval of the AIT testing and evaluation plan. In July 2011, GAO also reported
                                              that TSA experienced challenges collecting data on the properties of certain
What GAO Recommends                           explosives needed by vendors to develop EDS detection software and needed by
GAO is not making any new                     TSA before testing EDS prior to procurement and deployment to airports. TSA
recommendations. In prior work, GAO           and the DHS Science and Technology Directorate experienced these challenges
made recommendations to address               because of problems safely handling and consistently formulating some
challenges related to deploying AIT,          explosives. The challenges related to data collection for certain explosives
EDS, and other screening technology           resulted in problems carrying out the EDS procurement as planned.
to meet requirements; overseeing and
                                              DHS and TSA have experienced challenges identifying acquisition program
conducting testing of AIT and EDS
technologies; and incorporating
                                              baselines, program schedules, and costs. GAO’s prior work has found that
information on costs and schedules,           realistic acquisition program baselines with stable requirements for cost,
among other things, in making                 schedule, and performance are among the factors that are important to
technology acquisition decisions. DHS         successful acquisitions delivering capabilities within cost and schedule. GAO
and TSA concurred and have actions            also found that program performance metrics for cost and schedule can provide
underway to address these                     useful indicators of the health of acquisition programs. In April 2012 GAO
recommendations.                              reported that TSA’s methods for developing life-cycle cost estimates for the
                                              Electronic Baggage Screening Program did not fully adhere to best practices for
                                              developing these estimates.
View GAO-12-644T. For more information,
contact Steve Lord at (202) 512-4379 or       DHS has efforts underway to strengthen oversight of technology acquisitions. In
lords@gao.gov.                                part due to the problems GAO highlighted in DHS’s acquisition process, the
                                              implementation and transformation of DHS remains on GAO’s high-risk list.
                                                                                     United States Government Accountability Office
Chairmen Issa and Mica, Ranking Members Cummings and Rahall, and
Members of the Committees:

I am pleased to be here today to discuss our past work examining the
Transportation Security Administration’s (TSA) progress and challenges
in developing and acquiring new technologies to address homeland
security needs. TSA acquisition programs represent billions of dollars in
life-cycle costs and support a wide range of aviation security missions
and investments, including technologies used to screen passengers,
checked baggage, and air cargo, among others. Within the Department of
Homeland Security (DHS), the Science and Technology Directorate
(S&T) has responsibility for coordinating and conducting basic and
applied research, development, demonstration, testing, and evaluation
activities relevant to DHS components, which also have responsibilities
for developing, testing, acquiring, and deploying such technologies. For
example, TSA is responsible for securing the nation’s transportation
systems and, with S&T, researching, developing, and deploying
technologies to, for example, screen airline passengers and their
property.

In recent years, we have reported that DHS has experienced challenges
in managing its multibillion-dollar acquisition efforts, including
implementing technologies that did not meet intended requirements and
were not appropriately tested and evaluated, and has not consistently
included completed analyses of costs and benefits before technologies
were implemented.

My testimony today focuses on the key findings of our prior work related
to TSA’s efforts to acquire and deploy new technologies to address
homeland security needs. Our past work has identified three key
challenges: (1) developing and meeting technology program
requirements, (2) overseeing and conducting testing of new screening
technologies, and (3) identifying acquisition program baselines—or
starting points, program schedules, and costs. This statement will also
discuss recent DHS and TSA efforts to strengthen its investment and
acquisition processes.

This statement is based on reports and testimonies we issued from
October 2009 through May 2012 related to TSA’s efforts to manage, test,




Page 1                                                         GAO-12-644T
             acquire, and deploy various technology programs. 1 In addition, we
             obtained updated information in May 2012 from TSA on the number of
             currently deployed AIT units and from DHS officials on the status of the
             current EDS acquisition. For our past work, we reviewed program
             schedules, planning documents, testing reports, and other acquisition
             documentation. For some of the programs we discuss in this testimony,
             we conducted site visits to a range of facilities, such as national
             laboratories, airports, and other locations to observe research,
             development, and testing efforts. We also conducted interviews with DHS
             component program managers and S&T officials to discuss issues related
             to individual programs. More detailed information on the scope and
             methodology from our previous work can be found within each specific
             report. We conducted this work in accordance with generally accepted
             government auditing standards.

             Since the department’s creation in 2003, we have designated the
Background   implementation and transformation of DHS as high risk because DHS had
             to combine 22 agencies—several with major management challenges—
             into one department, and failure to effectively address DHS’s
             management and mission risks could have serious consequences for
             U.S. national and economic security. 2 This high-risk area includes (1)
             challenges in strengthening DHS’s management functions—financial
             management, human capital, information technology (IT), and acquisition
             management—(2) the effect of those challenges on DHS’s mission
             implementation, and (3) challenges in integrating management functions
             within and across the department and its components. On the basis of our
             prior work, in September 2010 we identified and provided to DHS 31
             actions and outcomes that are critical to addressing the challenges within
             the department’s management areas and in integrating those functions
             across the department. These key actions and outcomes include, among
             others, validating required acquisition documents in accordance with a
             department-approved, knowledge-based acquisition process.




             1
                 See the related products list at the end of this statement.
             2
               See GAO, Highlights of a GAO Forum: Mergers and Transformations: Lessons Learned
             for a Department of Homeland Security and Other Federal Agencies, GAO-03-293SP
             (Washington, D.C.: Nov. 14, 2002) and Results-Oriented Cultures: Implementation Steps
             to Assist Mergers and Organizational Transformations, GAO-03-669 (Washington, D.C.:
             July 2, 2003).




             Page 2                                                                    GAO-12-644T
The Aviation and Transportation Security Act (ATSA) established TSA as
the federal agency with primary responsibility for securing the nation’s
civil aviation system, which includes the screening of all passengers and
property transported from and within the United States by commercial
passenger aircraft. 3 In accordance with ATSA, all passengers, their
accessible property, and their checked baggage are screened pursuant to
TSA-established procedures at more than 450 airports presently
regulated for security by TSA. These procedures generally provide,
among other things, that passengers pass through security checkpoints
where they and their identification documents, and accessible property,
are checked by transportation security officers (TSO), other TSA
employees, or by private-sector screeners under TSA’s Screening
Partnership Program. 4

TSA relies upon multiple layers of security to deter, detect, and disrupt
persons posing a potential risk to aviation security. These layers include
TSOs responsible for screening passengers and their carry-on baggage
at passenger checkpoints, using technologies that include x-ray
equipment, magnetometers, and Advanced Imaging Technology (AIT),
among others. In response to the December 2009 attempted terrorist
attack, TSA revised its procurement and deployment strategy for AIT,
commonly referred to as full-body scanners, increasing the number of AIT
units it planned to procure and deploy. TSA stated that AIT provides
enhanced security benefits compared with walk-through metal detectors,
such as enhanced detection capabilities for identifying nonmetallic threat
objects and liquids. AIT produces an image of a passenger’s body that a
screener interprets. The image identifies objects, or anomalies, on the
outside of the physical body but does not reveal items beneath the
surface of the skin, such as implants. As of May 2012, TSA has deployed
more than 670 AIT units to approximately 170 airports and reported that it
plans to deploy a total of about 1,250 AIT units. In January 2012, we
issued a classified report on TSA’s procurement and deployment of AIT
that addressed the extent to which (1) TSA followed DHS acquisition


3
  See Pub. L. No. 107-71, 115 Stat. 597 (2001). For purposes of this testimony,
“commercial passenger aircraft” refers to a U.S. or foreign-based air carrier operating
under TSA-approved security programs with regularly scheduled passenger operations to
or from a U.S. airport.
4
 Private-sector screeners under contract to and overseen by TSA, and not TSOs, perform
screening activities at the 16 airports currently participating in TSA’s Screening
Partnership Program. See 49 U.S.C. § 44920.




Page 3                                                                     GAO-12-644T
                    guidance when procuring AIT and (2) deployed AIT units are effective at
                    detecting threats. Another layer of security is checked-baggage
                    screening, which uses technology referred to as explosive detection
                    systems (EDS) and explosives trace detection (ETD). 5


                    Our past work has found that technology program performance cannot be
DHS and TSA Have    accurately assessed without valid baseline requirements established at
Experienced         the program start. Without the development, review, and approval of key
                    acquisition documents, such as the mission need statement and
Challenges in       operational requirements document, agencies are at risk of having poorly
Developing and      defined requirements that can negatively affect program performance and
                    contribute to increased costs. 6 For example, in June 2010, we reported
Meeting Key         that more than half of 15 DHS programs we reviewed awarded contracts
Performance         to initiate acquisition activities without component or department approval
Requirements for    of documents essential to planning acquisitions, setting operational
                    requirements, or establishing acquisition program baselines. 7 We
Various Screening   currently have ongoing work related to this area and we plan to report the
Technologies        results later this year. We made a number of recommendations to help
                    address issues related to these procurements as discussed below. DHS
                    has generally agreed with these recommendations and, to varying
                    degrees, has taken actions to address them.

                    In addition, our past work has found that TSA faces challenges in
                    identifying and meeting program requirements in some of its aviation
                    security programs. For example:



                    5
                      AIT screens passengers for metallic and nonmetallic threats including weapons,
                    explosives, and other objects concealed under layers of clothing. TSA primarily uses two
                    types of technology in the screening of checked baggage: (1) explosive detection systems
                    (EDS) which use X-rays with computer-aided imaging to automatically recognize the
                    characteristic signatures of threat explosives, and (2) explosives trace detection (ETD)
                    machines, in which a human operator (baggage screener) uses chemical analysis to
                    manually detect traces of explosive materials’ vapors and residue.
                    6
                     The mission need statement outlines the specific functional capabilities required to
                    accomplish DHS’s mission and objectives, along with deficiencies and gaps in these
                    capabilities. The operational requirements document includes key performance
                    parameters and describes the mission, capabilities, and objectives to provide needed
                    capabilities.
                    7
                     GAO, Department of Homeland Security: Assessments of Selected Complex
                    Acquisitions, GAO-10-588SP (Washington, D.C.: June 30, 2010). Three of 15 were TSA
                    programs.




                    Page 4                                                                       GAO-12-644T
•    We reported in January 2012 that TSA did not fully follow DHS
     acquisition policies when acquiring AIT, which resulted in DHS
     approving full AIT deployment without full knowledge of TSA’s revised
     specifications. 8 Specifically, DHS’s Acquisition Directive 102 required
     TSA to notify DHS’s Acquisition Review Board (ARB) if AIT could not
     meet any of TSA’s five key performance parameters (KPP) or if TSA
     changed a KPP during qualification testing. 9 Senior TSA officials
     acknowledged that TSA did not comply with the directive’s
     requirements, but stated that TSA still reached a “good decision” in
     procuring AIT and that the ARB was fully informed of the program’s
     changes to its KPPs. Further, TSA officials stated that the program
     was not bound by the directive because it was a new acquisition
     process and they believed that the ARB was not fully functioning at
     the time. 10 DHS officials stated that the ARB discussed the changed
     KPP but did not see the documents related to the change and
     determined that TSA must update the program’s key acquisition
     document, the Acquisition Program Baseline, before TSA could
     deploy AIT units. However, we reported that, according to a February
     2010 acquisition decision memorandum from DHS, the ARB approved
     TSA for full-scale production without reviewing the changed KPP.
     DHS officials stated that the ARB should have formally reviewed
     changes made to the KPP to ensure that TSA did not change it
     arbitrarily. According to TSA, it should have submitted its revised
     requirements for approval, but it did not because there was confusion
     as to whether DHS should be informed of all changes. We had
     previously reported that programs procuring new technologies with


8
 In January 2012, we issued a classified report on TSA’s procurement and deployment of
AIT, commonly referred to as full body scanners, at airport checkpoints.
9
 The ARB is the cross-component board within DHS that determines whether a proposed
acquisition has met the requirements of key phases in the acquisition life cycle framework
and is able to proceed to the next phase and eventual full production and deployment. Key
performance parameters (KPP) are system characteristics that are considered critical or
essential. Failure to meet a KPP could be the basis to reject a system solution.
10
   DHS’s Undersecretary for Management issued a memorandum on November 7, 2008,
requiring compliance with the directive at the program’s next formal decision point, but no
later than 6 months from the date of the directive (by May 2009). DHS acquisition officials
stated that enforcing compliance with the new policy took almost 1 year, but that it worked
with TSA to make the directive’s requirements known. However, DHS’s previous
Directive—Management Directive 1400—also required component agencies to follow a
similar process whereby programs were reviewed by DHS’s Investment Review Board. As
such, the Investment Review Board began reviewing TSA’s AIT program (at that time
called the Whole Body Imager) as early as 2008.




Page 5                                                                        GAO-12-644T
     fluctuating requirements will have a difficult time ensuring that the
     acquisition is meeting program needs. 11 DHS acquisition oversight
     officials agreed that changing key requirements is not a best practice
     for system acquisitions already under way. As a result, we found that
     TSA procured and deployed a technology that met evolving
     requirements, but not the initial requirements included in its key
     acquisition requirements document that the agency initially
     determined were necessary to enhance the aviation system. We
     recommended that TSA should develop a roadmap that outlines
     vendors’ progress in meeting all KPPs. DHS agreed with our
     recommendation.

•    In July 2011, we reported that TSA revised its EDS requirements to
     better address current threats, and plans to implement these
     requirements in a phased approach. 12 However, we reported that
     some number of EDS machines in TSA’s checked baggage screening
     fleet are configured to detect explosives at the levels established in
     the 2005 requirements. The remaining EDS machines are configured
     to detect explosives at 1998 levels. When TSA established the 2005
     requirements, it did not have a plan with the appropriate time frames
     needed to deploy EDSs to meet the requirements. To help ensure that
     TSA’s checked baggage screening machines are operating most
     effectively, we recommended that TSA develop a plan to deploy EDSs
     to meet the most recent explosive-detection requirements and ensure
     that the new machines, as well as machines deployed in airports, are
     operated at the levels in established requirements. 13 DHS concurred
     with our recommendation and has begun taking action to address it;
     for example, DHS reported that TSA has developed a plan to evaluate
     its current fleet of EDSs to determine the extent to which they comply
     with these requirements. However, our recommendation is intended to
     ensure that TSA operate all EDSs at airports at the most recent
     requirements. Until TSA develops a plan identifying how it will



11
  GAO, Defense Acquisitions: Managing Risk to Achieve Better Outcomes, GAO-10-374T
(Washington, D.C.: Jan. 20, 2010).
12
  GAO, Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for
Checked Baggage, but Additional Screening Actions Are Needed, GAO-11-740
(Washington, D.C.: July 11, 2011).
13
   Ibid. An EDS machine uses computed tomography technology to automatically measure
the physical characteristics of objects in baggage. The system automatically triggers an
alarm when objects that exhibit the physical characteristics of explosives are detected.




Page 6                                                                     GAO-12-644T
                       approach the upgrades for currently deployed EDSs—and the plan
                       includes such items as estimated costs and the number of machines
                       that can be upgraded—it will be difficult for TSA to provide reasonable
                       assurance that its upgrade approach is feasible or cost effective.
                   Our prior work has also shown that not resolving problems discovered
DHS and TSA Have   during testing can sometimes lead to costly redesign and rework at a later
Encountered        date. Addressing such problems before moving to the acquisition phase
                   can help agencies better manage costs. Specifically:
Challenges in
Overseeing and     •   In January 2012, we reported that TSA began deploying AIT before it
                       received approval for how it would test AIT. For example, DHS’s
Testing New            Acquisition Directive 102 required DHS to approve testing and
Screening              evaluation master plans—the documents that ensure that programs
                       are tested appropriately—prior to testing. However, we found that
Technologies           DHS did not approve TSA’s testing and evaluation master plan until
                       January 2010, after TSA had completed qualification and operational
                       tests and DHS had already approved TSA for full AIT deployment.
                       According to DHS, the DHS Director of Operational Testing and
                       Evaluation assessed the testing of AIT prior to the September 2009
                       ARB meeting and recommended approving the decision to procure
                       AIT at that meeting, even though the ARB did not approve its testing
                       plans. Additionally, we reported that DHS approved TSA’s AIT
                       deployment in September 2009, on the basis of laboratory-based
                       qualification testing results and initial field-based operational testing
                       results that were not completed until later that year. According to DHS
                       officials, the department initially had challenges providing effective
                       oversight to projects already engaged in procurement when the
                       directive was issued. For example, they noted that TSA had begun
                       conducting qualification testing in 2009, but DHS’s first AIT oversight
                       meeting under the new directive was not until later that year. As a
                       result, we reported that TSA procured AIT without DHS’s full oversight
                       and approval or knowledge of how TSA would test and evaluate AIT.

                   •   In July 2011, we reported that TSA revised the explosive detection
                       requirements for EDS checked baggage screening machines in 2005
                       though it did not begin operating EDS systems to meet these 2005
                       requirements until 2009. We also reported that TSA made additional
                       revisions to the EDS requirements in January 2010 but experienced
                       challenges in collecting explosives data on the physical and chemical
                       properties of certain explosives needed by vendors to develop EDS




                   Page 7                                                            GAO-12-644T
      detection software to meet the 2010 requirements. 14 These data are
      also needed by TSA for testing the machines to determine whether
      they meet established requirements prior to their procurement and
      deployment to airports. TSA and S&T have experienced these
      challenges because of problems associated with safely handling and
      consistently formulating some explosives, which have also resulted in
      problems carrying out the EDS procurement as planned. Further, TSA
      deployed a number of EDSs that had the software necessary to meet
      the 2005 requirements, but because testing to compare false-alarm
      rates had not been completed, the software was not activated,
      subsequently; these EDSs were detecting explosives at levels
      established in 1998. According to TSA officials, once completed, the
      results of this testing to compare false alarm rates would allow them
      to determine if additional staff are needed at airports to help resolve
      false alarms once the EDSs are configured to operate at a certain
      level of requirements. TSA officials told us that they planned to
      perform this testing as a part of the ongoing EDS acquisition. We
      recommended that TSA develop a plan to ensure that TSA has the
      explosives data needed for each of the planned phases of the 2010
      EDS requirements before starting the procurement process for new
      EDSs or upgrades included in each applicable phase. DHS stated that
      TSA modified its strategy for the EDS’s competitive procurement in
      July 2010 in response to the challenges in working with the explosives
      for data collection by removing the data collection from the
      procurement process. TSA’s plan to separate the data collection from
      the procurement process is a positive step, but to fully address our
      recommendation, a plan is needed to establish a process for ensuring
      that data are available before starting the procurement process for
      new EDSs or upgrades for each applicable phase.

•     In June 2011 we reported that S&T’s Test & Evaluation and
      Standards Office, responsible for overseeing test and evaluation of
      DHS’s major acquisition programs, reviewed or approved test and
      evaluation documents and plans for programs undergoing testing, and
      conducted independent assessments for the programs that completed
      operational testing. 15 DHS senior-level officials considered the office’s
      assessments and input in deciding whether programs were ready to


14
     GAO-11-740.
15
  GAO, DHS Science and Technology: Additional Steps Needed to Ensure Test and
Evaluation Requirements Are Met, GAO-11-596 (Washington, D.C.: June 15, 2011).




Page 8                                                                 GAO-12-644T
                               proceed to the next acquisition phase. However, the office did not
                               consistently document its review and approval of components’ test
                               agents—a government entity or independent contractor carrying out
                               independent operational testing for a major acquisition. We
                               recommended, among other things, that S&T develop mechanisms to
                               document its review of component acquisition documentation. DHS
                               concurred and reported actions underway to address them.

                         •     In October 2009, we reported that TSA deployed explosives trace
                               portals, a technology for detecting traces of explosives on passengers
                               at airport checkpoints, in January 2006 even though TSA officials
                               were aware that tests conducted during 2004 and 2005 on earlier
                               models of the portals suggested the portals did not demonstrate
                               reliable performance in an airport environment. 16 In June 2006, TSA
                               halted deployment of the explosives trace portals because of
                               performance problems and high installation costs. In our 2009 report,
                               we recommended that, to the extent feasible, TSA ensure that tests
                               are completed before deploying new checkpoint screening
                               technologies to airports. DHS concurred with the recommendation
                               and has taken action to address it, such as requiring more-recent
                               technologies to complete both laboratory and operational tests prior to
                               deployment.

                         We have found that realistic acquisition program baselines with stable
DHS and TSA Have         requirements for cost, schedule, and performance are among the factors
Experienced              that are important to successful acquisitions delivering capabilities within
                         cost and schedule. 17 Our prior work has found that program performance
Challenges Identifying   metrics for cost and schedule can provide useful indicators of the health
Acquisition Program      of acquisition programs and, when assessed regularly for changes and
Baselines, Program       the reasons that cause changes, such indicators can be valuable tools for
                         improving insight and oversight of individual programs as well as the total
Schedules, and Costs     portfolio of major acquisitions. 18 Importantly, program performance cannot
                         be accurately assessed without valid baseline requirements established



                         16
                           GAO, Aviation Security: DHS and TSA Have Researched, Developed, and Begun
                         Deploying Passenger Checkpoint Screening Technologies, but Continue to Face
                         Challenges, GAO-10-128 (Washington, D.C.: October 7, 2009).
                         17
                              GAO-10-588SP.
                         18
                           Defense Acquisitions: Measuring the Value of DOD’s Weapon Programs Requires
                         Starting with Realistic Baselines, GAO-09-543T (Washington, D.C.: April 1, 2009).




                         Page 9                                                                    GAO-12-644T
at the program start, particularly those that establish the minimum
acceptable threshold required to satisfy user needs. 19 According to DHS’s
acquisition guidance, the program baseline is the contract between the
program and departmental oversight officials and must be established at
program start to document the program’s expected cost, deployment
schedule, and technical performance. Establishing such a baseline at
program start is important for defining the program’s scope, assessing
whether all life-cycle costs are properly calculated, and measuring how
well the program is meeting its goals. By tracking and measuring actual
program performance against this baseline, management can be alerted
to potential problems, such as cost growth or changing requirements, and
has the ability to take early corrective action.

•     We reported in Aril 2012 that TSA has not had a DHS-approved
      acquisition program baseline since the inception of the EBSP program
      more than 8 years ago. Further, DHS did not require TSA to complete
      an acquisition program baseline until November 2008. According to
      TSA officials, they have twice submitted an acquisition program
      baseline to DHS for approval—first in November 2009 and again
      February 2011. An approved baseline will provide DHS with additional
      assurances that TSA’s approach is appropriate and that the
      capabilities being pursued are worth the expected costs. In November
      2011, because TSA did not have a fully developed life-cycle cost
      estimate as part of its acquisition program baseline, DHS instructed
      TSA to revise the life cycle cost estimates as well as its procurement
      and deployment schedules to reflect budget constraints. DHS officials
      told us that they could not approve the acquisition program baseline
      as written because TSA’s estimates were significantly over budget.
      TSA officials stated that TSA is currently working with DHS to amend
      the draft program baseline and plans to resubmit the revised
      acquisition program baseline before the next Acquisition Review
      Board meeting, which is currently planned for July 2012. Establishing
      and approving a program baseline, as DHS and TSA currently plan to
      do for the EBSP, could help DHS assess the program’s progress in
      meeting its goals and achieve better program outcomes.
In our 2010 report of selected DHS acquisitions, 12 of 15 selected DHS
programs we reviewed exhibited schedule delays and cost growth beyond
initial estimates. We noted that DHS acquisition oversight officials have



19
     GAO-10-588SP.




Page 10                                                          GAO-12-644T
raised concerns about the accuracy of cost estimates for most major
programs, making it difficult to assess the significance of the cost growth
we identified. Leading practices state that the success of a large-scale
system acquisition, such as the TSA’s EDS acquisition, depends in part
on having a reliable schedule that identifies: (1) when the program’s set of
work activities and milestone events will occur, (2) how long they will take,
and (3) how they are related to one another. 20 Leading practices also call
for the schedule to expressly identify and define the relationships and
dependencies among work elements and the constraints affecting the
start and completion of work elements. Additionally, best practices
indicate that a well-defined schedule also helps to identify the amount of
human capital and fiscal resources that are needed to execute an
acquisition.

•     We reported in January 2012 that TSA did not have plans to require
      vendors to meet milestones used during the AIT acquisition. We
      recommended that TSA should develop a roadmap that outlines
      vendors’ progress in meeting all KPPs because it is important that
      TSA convey vendors’ progress in meeting those requirements and full
      costs of the technology to decision makers when making deployment
      and funding decisions. TSA reported that it hoped vendors would be
      able to gradually improve meeting KPPs for AIT over time. We
      reported that TSA would have more assurance that limited taxpayer
      resources are used effectively by developing a roadmap that specifies
      development milestones for the technology and having DHS
      acquisition officials approve this roadmap. DHS agreed with our
      recommendation.

•     In July 2011, we reported that TSA had established a schedule for the
      acquisition of EDS machines but it did not fully comply with leading
      practices, and TSA had not developed a plan to upgrade its EDS fleet
      to meet the current explosives detection requirements. These leading
      practices state that the success of a large-scale system acquisition,
      such as TSA’s EDS acquisition, depends in part on having a reliable
      schedule that identifies when the program’s set of work activities and
      milestone events will occur, amongst other things. For example, the
      schedule for the EDS acquisition is not reliable because it does not
      reflect all planned program activities and does not include a timeline
      to deploy EDSs or plans to procure EDSs to meet subsequent phases


20
     GAO-11-740.




Page 11                                                           GAO-12-644T
                            of explosive detection requirements. We stated that developing a
                            reliable schedule would help TSA better monitor and oversee the
                            progress of the EDS acquisition. DHS concurred with our
                            recommendation to develop and maintain a schedule for the entire
                            EBSP in accordance with the leading practices identified by GAO for
                            preparing a schedule. DHS commented that TSA had already begun
                            working with key stakeholders to develop and define requirements for
                            a schedule and to ensure that the schedule aligns with the best
                            practices outlined by GAO.

                       •    In April 2012, we reported that TSA’s methods for developing life
                            cycle cost estimates for the EBSP did not fully adhere to best
                            practices for developing these estimates. As highlighted in our past
                            work, a high-quality, reliable cost estimation process provides a sound
                            basis for making accurate and well-informed decisions about resource
                            investments, budgets, assessments of progress, and accountability
                            for results and thus is critical to the success of a program. We
                            reported that TSA’s estimates partially met three characteristics and
                            minimally met one characteristic of a reliable cost estimate. 21 DHS
                            concurred with our recommendation that TSA ensure that its life cycle
                            cost estimates conform to cost estimating best practices, and
                            identified efforts underway to address it. DHS also acknowledged the
                            importance of producing life cycle cost estimates that are
                            comprehensive, well documented, accurate, and credible so that they
                            can be used to support DHS funding and budget decisions.

                       In part due to the problems we have highlighted in DHS’s acquisition
DHS Has Efforts        process, the implementation and transformation of DHS remains on our
Underway to            high-risk list. DHS currently has several plans and efforts underway to
                       address the high-risk designation as well as the more specific challenges
Strengthen Oversight   related to acquisition and program implementation that we have
of Technology          previously identified. For example, DHS initially described an initiative in
                       the January 2011 version of its Integrated Strategy for High Risk
Acquisitions           Management to establish a framework, the Integrated Investment Life
                       Cycle Model (IILCM), for managing investments across its components
                       and management functions; strengthening integration within and across
                       those functions; and ensuring mission needs drive investment decisions.


                       21
                         We reported that the estimate was partially comprehensive, partially documented,
                       partially accurate, and minimally credible when compared to the criteria in our Cost
                       Estimating and Assessment Guide.




                       Page 12                                                                      GAO-12-644T
The department seeks to use the IILCM to enhance resource decision
making and oversight by creating new department-level councils to
identify priorities and capability gaps, revising how DHS components and
lines of business manage acquisition programs, and developing a
common framework for monitoring and assessing implementation of
investment decisions. We reported in March 2012 that, from the time DHS
first reported on the IILCM initiative in January 2011 to its December 2011
revision of its high-risk strategy, the initiative had made little progress
though DHS plans to begin using the IILCM by the end of September
2012.

In October 2011, to enhance the department’s ability to oversee major
acquisition programs, DHS realigned the acquisition management
functions previously performed by two divisions within the Office of Chief
Procurement Officer to establish the Office of Program Accountability and
Risk Management (PARM). PARM, which is responsible for program
governance and acquisition policy, serves as the Management
Directorate’s executive office for program execution and works with DHS
leadership to assess the health of major acquisitions and investments. To
help with this effort, PARM is developing a database, known as the
Decision Support Tool, intended to improve the flow of information from
component program offices to the Management Directorate to support its
governance efforts. DHS reported in its December 2011 Integrated
Strategy for High Risk Management that senior executives are not
confident enough in the data to use the Decision Support Tool developed
by PARM to help make acquisition decisions. However, DHS’s plans to
improve the quality of the data in this database are limited. At this time,
PARM only plans to check the data quality in preparation for key
milestone meetings in the acquisition process. This could significantly
diminish the Decision Support Tool’s value because users cannot
confidently identify and take action to address problems meeting cost or
schedule goals prior to program review meetings.

We reported in March 2012 that DHS has made progress strengthening
its management functions, but the department faces considerable
challenges. Specifically, DHS has faced challenges overseeing the
management, testing, acquisition, and deployment of various technology
programs including AIT and EDS. Going forward, DHS needs to continue
implementing its Integrated Strategy for High Risk Management and show
measurable, sustainable progress in implementing its key management
initiatives and corrective actions and achieving outcomes including those
related to acquisition management. DHS reported that it plans to revise its
Integrated Strategy for High Risk Management in June 2012, which


Page 13                                                         GAO-12-644T
                  includes management initiatives and corrective actions to address
                  acquisition management challenges, among other management areas.
                  We will continue to monitor and assess DHS’s implementation and
                  transformation efforts through our ongoing and planned work, including
                  the 2013 high-risk update that we expect to issue in early 2013.


                  Chairmen Issa and Mica, Ranking Members Cummings and Rahall, and
                  members of the committees, this concludes my prepared statement. I
                  would be pleased to respond to any questions that you may have.


                  For questions about this statement, please contact Steve Lord at (202)
GAO Contact and   512-4379 or lords@gao.gov. Contact points for our Offices of
Staff             Congressional Relations and Public Affairs may be found on the last page
                  of this statement. Individuals making key contributions to this statement
Acknowledgments   include Dave Bruno, Assistant Director; Scott Behen, Analyst-in-Charge;
                  Emily Gunn, and Katherine Trimble. Other contributors include: David
                  Alexander, Tom Lombardi, Jason Lee, Linda Miller, and Jerry Seigler.
                  Key contributors for the previous work that this testimony is based on are
                  listed within each individual product.




                  Page 14                                                        GAO-12-644T
Related GAO Products
             Related GAO Products




             Checked Baggage Screening: TSA Has Deployed Optimal Systems at the
             Majority of TSA-Regulated Airports, but Could Strengthen Cost
             Estimates. GAO-12-266. Washington D.C.: April 27, 2012.

             Transportation Security Administration: Progress and Challenges Faced
             in Strengthening Three Key Security Programs. GAO-12-541T.
             Washington D.C.: March 26, 2012

             Aviation Security: TSA Has Made Progress, but Additional Efforts Are
             Needed to Improve Security. GAO-11-938T. Washington, D.C.:
             September 16, 2011.

             Department of Homeland Security: Progress Made and Work Remaining
             in Implementing Homeland Security Missions 10 Years after 9/11.
             GAO-11-881. Washington, D.C.: September 7, 2011.

             Homeland Security: DHS Could Strengthen Acquisitions and
             Development of New Technologies. GAO-11-829T. Washington, D.C.:
             July 15, 2011.

             Aviation Security: TSA Has Taken Actions to Improve Security, but
             Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13,
             2011.

             Aviation Security: TSA Has Enhanced Its Explosives Detection
             Requirements for Checked Baggage, but Additional Screening Actions
             Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.

             Homeland Security: Improvements in Managing Research and
             Development Could Help Reduce Inefficiencies and Costs.
             GAO-11-464T. Washington D.C.: March 15, 2011.

             High-Risk Series: An Update. GAO-11-278. Washington D.C.: February
             16, 2011.

             Department of Homeland Security: Assessments of Selected Complex
             Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.

             Aviation Security: Progress Made but Actions Needed to Address
             Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T.
             Washington, D.C.: June 30, 2010.




             Page 15                                                       GAO-12-644T
           Related GAO Products




           Aviation Security: TSA Is Increasing Procurement and Deployment of
           Advanced Imaging Technology, but Challenges to This Effort and Other
           Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.:
           March 17, 2010.

           Aviation Security: DHS and TSA Have Researched, Developed, and
           Begun Deploying Passenger Checkpoint Screening Technologies, but
           Continue to Face Challenges. GAO-10-128. Washington, D.C.: October
           7, 2009.




(441071)
           Page 16                                                     GAO-12-644T
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.