United States Government Accountability Office GAO Testimony Before the Subcommittee on Transportation Security, Committee on Homeland Security, House of Representatives AVIATION SECURITY For Release on Delivery Expected at 1:30 p.m. EDT Tuesday, June 19, 2012 Status of TSA’s Acquisition of Technology for Screening Passenger Identification and Boarding Passes Statement of Stephen M. Lord, Director Homeland Security and Justice Issues GAO-12-826T Chairman Rogers, Ranking Member Jackson Lee, and Members of the Committee: I am pleased to be here today to discuss our past work examining the Transportation Security Administration’s (TSA) progress and challenges in developing and acquiring technologies to address aviation security needs. TSA’s acquisition programs represent billions of dollars in life cycle costs and support a wide range of aviation security missions and investments. Within the Department of Homeland Security (DHS), the Science and Technology Directorate (S&T) and TSA have responsibilities for researching, developing, and testing and evaluating new technologies, including airport checkpoint screening technologies. Specifically, S&T is responsible for the basic and applied research and advanced development of new technologies, while TSA, through its Passenger Screening Program, identifies the need for new checkpoint screening technologies and provides input to S&T during the research and development of new technologies, which TSA then procures and deploys. TSA screens more than 600 million air passengers per year through approximately 2,300 security checkpoint lanes at about 450 airports nationwide, and must attempt to balance its aviation security mission with concerns about efficiency and the privacy of the traveling public. The agency relies upon multiple layers of security to deter, detect, and disrupt persons posing a potential risk to aviation security. Part of its checkpoint security controls include a manual review and comparison by a travel document checker of each person’s boarding pass and identification, such as passports or state-issued driver’s licenses. However, concerns have been raised about security vulnerabilities in this process. For example, in 2006, a university student created a website that enabled individuals to create fake boarding passes. In addition, in 2011, a man was convicted of stowing away aboard an aircraft after using an expired boarding pass with someone else’s name on it to fly from New York to Los Angeles. Recent news reports have also highlighted the apparent ease of ordering high-quality counterfeit driver’s licenses from China. We have previously reported on significant fraud vulnerabilities in the Page 1 GAO-12-826T passport issuance process and on difficulties in detecting fraudulent identity documentation, such as driver’s licenses. 1 In response to these vulnerabilities, and as part of its broader effort to improve security and increase efficiency, TSA began developing technology designed to automatically verify boarding passes and to better identify altered or fraudulent passenger identification documents. TSA plans for this technology, known as Credential Authentication Technology/Boarding Pass Scanning Systems (CAT/BPSS), to eventually replace the current procedure used by travel document checkers to detect fraudulent or altered documents. However, we have previously reported that DHS and TSA have experienced challenges in managing their acquisition efforts, including implementing technologies that did not meet intended requirements and were not appropriately tested and evaluated, and have not consistently included completed analyses of costs and benefits before technologies were implemented. 2 Since DHS’s inception in 2003, we have designated implementing and transforming DHS as high risk because DHS had to transform 22 agencies—several with major management challenges—into one department. 3 This high-risk area includes challenges in strengthening DHS’s management functions, including acquisitions; the effect of those challenges on DHS’s mission implementation; and challenges in integrating management functions within and across the department and its components. DHS currently has several plans and efforts under way to address the high-risk designation as well as the more specific challenges related to acquisition and program implementation that we have previously identified. For example, DHS provided us with its Integrated Strategy for High Risk Management in June 2012, which includes management initiatives and corrective actions to address acquisition management challenges, among other management areas. We will 1 GAO, State Department: Significant Vulnerabilities in the Passport Issuance Process, GAO-09-681T (Washington, D.C.: May 5, 2009), and Transportation Worker Identification Credential: Internal Control Weaknesses Need to Be Corrected to Help Achieve Security Objectives, GAO-11-657 (Washington, D.C.: May 10, 2011). We also have ongoing classified work looking at the effectiveness of the travel document checker at detecting fraudulent documents, which we expect to finalize later this summer. 2 For example, see GAO, Homeland Security: DHS and TSA Face Challenges Overseeing Acquisition of Screening Technologies, GAO-12-644T (Washington, D.C.: May 9, 2012). 3 GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: Feb.16, 2011). Page 2 GAO-12-826T continue to monitor and assess DHS’s implementation and transformation efforts through our ongoing and planned work, including the 2013 high- risk update that we expect to issue in early 2013. My statement today focuses on (1) the status of TSA’s CAT/BPSS acquisition and the extent to which the related life cycle cost estimate is consistent with best practices and (2) challenges we have previously identified in TSA’s acquisition process to manage, test, acquire, and deploy screening technologies. This statement also provides information on issues for possible congressional oversight related to CAT/BPSS. This statement is based on reports and testimonies we issued from October 2009 through May 2012 related to TSA’s efforts to manage, test, acquire, and deploy various technology programs. 4 In addition, we obtained updated information in June 2012 from TSA on the status of its efforts to implement our recommendations from these reports. For our past work, we reviewed program schedules, planning documents, testing reports, and other acquisition documentation. For some of the programs we discuss in this testimony, we conducted site visits to a range of facilities, such as national laboratories, airports, and other locations to observe research, development, and testing efforts. We also conducted interviews with DHS component program managers and DHS Science and Technology Directorate officials to discuss issues related to individual programs. More detailed information on the scope and methodology from our previous work can be found within each specific report. In addition, this statement contains new information we obtained from TSA in June 2012 on the status of its CAT/BPSS acquisition. We reviewed key acquisition documents—including the mission needs statement (September 2008), request for proposal (April 2011), operational requirements document (August 2011), life cycle cost estimate (November 2011), and acquisition program baseline (November 2011)— interviewed officials from TSA’s Office of Security Capabilities, and viewed a demonstration of the CAT/BPSS test units. We compared the 4 See the related products list at the end of this statement. Examples of these technology programs include advanced imaging technology (AIT)—commonly referred to as a full body scanner—that screens passengers for metallic and nonmetallic threats including weapons, explosives, and other objects concealed under layers of clothing; explosives detection systems, which use X-rays with computer-aided imaging to automatically recognize the characteristic signatures of threat explosives; and explosives trace detection machines, in which a human operator (e.g., a baggage screener) uses chemical analysis to manually detect traces of explosive materials’ vapors and residue. Page 3 GAO-12-826T life cycle cost estimate with best practices from our Cost Estimating and Assessment Guide to determine whether the official cost estimates were comprehensive (i.e., include all costs), accurate, well documented, and credible. 5 We conducted all of our work in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings based on our audit objectives. We discussed new information in this statement with TSA officials and incorporated their comments as appropriate. In summary, TSA has completed its initial testing of the CAT/BPSS technology and has begun operational testing at three airports. We found the project’s associated life cycle cost estimate to be reasonably comprehensive and well documented, although we are less confident in its accuracy due to questions about the assumed inflation rate. In addition, we could not evaluate its credibility because the current version does not include an independent cost estimate or an assessment of how changing key assumptions and other factors would affect the estimate. Our past work has identified three key challenges related to TSA’s efforts to acquire and deploy technologies to address homeland security needs: (1) developing and meeting technology program requirements, (2) overseeing and conducting testing of new screening technologies, and (3) developing acquisition program baselines to establish initial cost, schedule, and performance parameters. 5 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). Page 4 GAO-12-826T CAT/BPSS, which is part of TSA’s Passenger Screening Program, has CAT/BPSS Is in the undergone initial testing and is in the operational testing and evaluation Operational Testing phase of acquisition, according to TSA. The goal of CAT/BPSS is to deploy a computerized system that will read and analyze data and and Evaluation Phase, embedded security features on every passenger’s identification and some and the Life Cycle boarding passes, and to identify fraudulent credentials and boarding Cost Estimate Is Not passes. In 2011, TSA conducted qualification testing of this system at its System Integration Facility at Washington Reagan National Airport, Fully Consistent with including testing the systems against more than 530 genuine and Best Practices fraudulent documents, such as state-issued driver’s licenses, passports, and military identification cards, according to TSA. The technology is designed to automatically compare a passenger’s identification with a set of embedded security features to seek to identify indicators of fraud and concurrently ensure that the information on the identification and boarding pass matches. This system is intended to help ensure that identity credentials and boarding passes presented at the checkpoint have not been tampered with or fraudulently produced, and that the information on the boarding pass matches that of the identity credential. According to TSA, CAT/BPSS is to compare identity credentials with an internal database of more than 2,400 templates for various types of credentials and to check for certain embedded security features, then alert the operator of any discrepancies. In September 2011, TSA awarded contracts for approximately $3.2 million, which included the purchase of 30 units from three different vendors. 6 In April 2012, TSA began deploying units to three airports— George Bush Intercontinental in Houston, Luis Muñoz Marín International in San Juan, and Washington Dulles International—in preparation for initial operational testing. TSA officials said that those airports were selected, in part, because of their high passenger volume and experience with detecting fraudulent documents. In preparation for initial testing, TSA tested the performance of its current process for comparison purposes. TSA is also training personnel on the CAT/BPSS systems, collecting preliminary data on system performance and availability, and assessing the adequacy of the concept of operations and standard operating procedures. According to TSA officials, these efforts will allow travel document checkers at the three airports to test the three systems in an 6 According to TSA, the $3.2 million included costs for maintenance, database updates, and training, among other things. Page 5 GAO-12-826T operational environment and provide feedback on the systems’ performance. During operational testing, TSA plans to assess the systems’ performance against key performance parameters for detection, passenger throughput, and availability. Once operational testing is complete, TSA plans to produce a system evaluation report and recommend whether to move forward with the acquisition or make modifications. Vendors that successfully exit the operational testing phase will be eligible to compete for a contract to produce 1,400 units, according to TSA. According to the life cycle cost estimate for the Passenger Screening Program, of which CAT/BPSS is a part, the estimated 20-year life cycle cost of CAT/BPSS is approximately $130 million based on a procurement of 4,000 units. 7 As highlighted in our Cost Estimating and Assessment Guide, a reliable cost estimate has four characteristics—it is comprehensive, well documented, accurate, and credible. 8 We reviewed TSA’s November 2011 life cycle cost estimate for the Passenger Screening Program and compared it with the four characteristics. Based on our assessment, the life cycle cost estimate is reasonably comprehensive and well documented. Regarding accuracy, the cost estimate assumes a 1 percent inflation rate from fiscal years 2015 through 2029, as compared with the historic inflation rates calculated for fiscal years 2009 through 2014, which ranged from 3.3 to 4.5 percent. If a larger inflation rate were used, costs would be much higher than what are currently estimated. In addition, we cannot make a determination as to the credibility of the life cycle cost estimate as it does not include a risk and uncertainty analysis or an independent cost estimate. The risk assessment would quantify risks and identify effects of changing key cost driver assumptions and factors. 9 In the cost estimate, TSA indicates that it is pursuing the acquisition of risk analysis capability and plans on having such capabilities in time for the next life cycle cost estimate. Likewise, there is no evidence that an independent cost estimate was conducted by a group outside the acquiring organization to determine whether other 7 This includes an initial procurement of 1,400 units in fiscal year 2013, and an additional 2,600 replacement units by fiscal year 2029. 8 GAO-09-3SP. The DHS Cost Analysis Division has implemented our Cost Estimating and Assessment Guide as the standard for cost estimating at DHS. 9 DHS did not approve the life cycle cost estimate due to the lack of risk and sensitivity analysis, according to TSA. Page 6 GAO-12-826T estimating methods would produce similar results. TSA officials indicated that the agency is updating its life cycle cost estimate to include a risk and uncertainty analysis and independent cost estimate, but the document has not yet been approved. The agency plans to expand the CAT/BPSS deployment schedule following successful implementation and testing in the selected airport environments. As of June 2012, TSA officials estimated that this could occur as soon as the end of this calendar year, depending on the results of the operational testing and evaluation phase. Our past work has identified three key challenges related to TSA’s efforts Previously Identified to acquire and deploy technologies to address homeland security needs: Challenges TSA Faces (1) developing and meeting technology program requirements, (2) overseeing and conducting testing of new screening technologies, and (3) in Overseeing developing acquisition program baselines to establish initial cost, Acquisition of schedule, and performance parameters. Screening We have previously reported that DHS and TSA have faced challenges in Technologies developing and meeting program requirements when acquiring screening technologies, and that program performance cannot be accurately assessed without valid baseline requirements established at the program start. In June 2010, for example, we reported that more than half of the 15 DHS programs we reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, or establishing acquisition program baselines. 10 We made a number of recommendations to help address issues related to these procurements. DHS generally agreed with these recommendations and, to varying degrees, has begun taking actions to address them. We currently have ongoing work related to this area and we plan to report the results later this fall. 11 At the program level, in May 2012, we reported that TSA did not fully follow DHS acquisition policies when acquiring advanced imaging 10 GAO, Department of Homeland Security: Assessments of Selected Complex Acquisitions, GAO-10-588SP (Washington, D.C.: June 30, 2010). Three of 15 were TSA programs. 11 We are conducting this work at the request of the Senate Committee on Homeland Security and Governmental Affairs and the Subcommittee on Oversight, Investigations, and Management of the House Committee on Homeland Security. Page 7 GAO-12-826T technology (AIT), or body scanners, which resulted in DHS approving full AIT deployment without full knowledge of TSA’s revised specifications. 12 As a result, we found that TSA procured and deployed a technology that met evolving requirements, but not the initial requirements included in its key acquisition requirements document that the agency initially determined were necessary to enhance the aviation system. We recommended that TSA develop a road map that outlines vendors’ progress in meeting all key performance parameters. DHS agreed with our recommendation and has begun taking action to address it. We have also reported on DHS and TSA challenges in overseeing and testing new screening technologies, which can lead to costly redesign and rework at a later date. Addressing such problems before moving to the acquisition phase can help agencies better manage costs. For example, in October 2009, we reported that TSA had deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, in January 2006 even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. 13 In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. In our 2009 report, we recommended that, to the extent feasible, TSA ensure that tests are completed before deploying new checkpoint screening technologies to airports. DHS concurred with the recommendation and has taken action to address it, such as requiring more-recent technologies to complete both laboratory and operational tests prior to deployment. DHS and TSA have also experienced challenges identifying acquisition program baselines, which include program schedules and costs. Our prior work has found that realistic acquisition program baselines with stable requirements for cost, schedule, and performance are among the factors that are important to successful acquisitions delivering capabilities within cost and schedule. We also found that program performance metrics for 12 See GAO-12-644T, in which we publicly reported some of the findings and recommendations from our January 2012 classified report on TSA’s procurement and deployment of AIT, commonly referred to as full body scanners, at airport checkpoints. 13 GAO, Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges, GAO-10-128 (Washington, D.C.: Oct. 7, 2009). Page 8 GAO-12-826T cost and schedule can provide useful indicators of the health of acquisition programs. For example, we reported in April 2012 that TSA has not had a DHS-approved acquisition program baseline since the inception of the Electronic Baggage Screening Program (EBSP) more than 8 years ago. 14 Further, DHS did not require TSA to complete an acquisition program baseline until November 2008. According to TSA officials, they have twice submitted an acquisition program baseline to DHS for approval—first in November 2009 and again in February 2011. An approved baseline would provide DHS with additional assurances that TSA’s approach is appropriate and that the capabilities being pursued are worth the expected costs. In November 2011, because TSA did not have a fully developed life cycle cost estimate as part of its acquisition program baseline for the EBSP, DHS instructed TSA to revise the life cycle cost estimates as well as its procurement and deployment schedules to reflect budget constraints. DHS officials told us that they could not approve the acquisition program baseline as written because TSA’s estimates were significantly over budget. TSA officials stated that TSA is currently working with DHS to amend the draft program baseline and plans to resubmit the revised acquisition program baseline before the next Acquisition Review Board meeting, which is planned for July or August 2012. Establishing and approving a program baseline, as DHS and TSA plan to do for the EBSP, could help DHS assess the program’s progress in meeting its goals and achieve better program outcomes. Our prior work on TSA acquisition management identified oversight problems that have led to cost increases, delivery delays, and other operational challenges for certain assets, such as EBSP, but TSA has also taken several steps to improve its acquisition management. For example, while we continue to find that some TSA acquisition programs do not have key documents needed for properly managing acquisitions, CAT/BPSS has a DHS-approved mission needs statement, operational requirements document, and acquisition program baseline. 15 14 GAO, Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, but Could Strengthen Cost Estimates, GAO-12-266 (Washington, D.C.: Apr. 27, 2012). 15 The life cycle cost estimate was approved by TSA but not by DHS. Page 9 GAO-12-826T This hearing provides an opportunity for congressional stakeholders to Congressional focus a dialogue on how to continue a sufficient level of oversight of the Oversight Issues CAT/BPSS acquisition and implementation and other key components of the Passenger Screening Program. For example, relevant questions that could be raised include the following: • To what extent, if any, have key performance parameters changed during the course of the acquisition, and how will these changes affect security and efficiency at the checkpoint? What would be TSA’s strategy if vendors have difficulty meeting the key performance parameters? • How will TSA ensure that implementation of the system addresses the security vulnerabilities previously identified? • What confidence does TSA have in its cost estimates and how is the agency mitigating the risk of cost escalation or schedule delays? • In managing limited resources to mitigate a potentially unlimited range of security threats, how does CAT/BPSS fit into TSA’s broader aviation security strategy? What cost-benefit and related analyses, if any, are being used to guide TSA decision makers? These types of questions and related issues warrant ongoing consideration by TSA management and continued oversight by congressional stakeholders. Chairman Rogers, Ranking Member Jackson Lee, and Members of the Committee, this concludes my prepared statement. I look forward to responding to any questions that you may have. For questions about this statement, please contact Steve Lord at (202) GAO Contact and 512-4379 or firstname.lastname@example.org. Contact points for our Offices of Staff Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement Acknowledgment include Jessica Lucas-Judy, Assistant Director; Carissa Bryant; Jennifer Echard; Laurier Fish; Tom Lombardi; and Katherine Trimble. Key contributors for the previous work that this testimony is based on are listed within each individual product. Page 10 GAO-12-826T Related GAO Products Related GAO Products Homeland Security: DHS and TSA Face Challenges Overseeing Acquisition of Screening Technologies. GAO-12-644T. Washington, D.C.: May 9, 2012. Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, but Could Strengthen Cost Estimates. GAO-12-266. Washington, D.C.: April 27, 2012. Transportation Security Administration: Progress and Challenges Faced in Strengthening Three Key Security Programs. GAO-12-541T. Washington D.C.: March 26, 2012. Homeland Security: DHS and TSA Acquisition and Development of New Technologies. GAO-11-957T. Washington, D.C.: September 22, 2011. Aviation Security: TSA Has Made Progress, but Additional Efforts Are Needed to Improve Security. GAO-11-938T. Washington, D.C.: September 16, 2011. Department of Homeland Security: Progress Made and Work Remaining in Implementing Homeland Security Missions 10 Years after 9/11. GAO-11-881. Washington, D.C.: September 7, 2011. Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies. GAO-11-829T. Washington, D.C.: July 15, 2011. Aviation Security: TSA Has Taken Actions to Improve Security, but Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13, 2011. Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for Checked Baggage, but Additional Screening Actions Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011. Homeland Security: Improvements in Managing Research and Development Could Help Reduce Inefficiencies and Costs. GAO-11-464T. Washington D.C.: March 15, 2011. High-Risk Series: An Update. GAO-11-278. Washington D.C.: February 16, 2011. Page 11 GAO-12-826T Related GAO Products Department of Homeland Security: Assessments of Selected Complex Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010. Aviation Security: Progress Made but Actions Needed to Address Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T. Washington, D.C.: June 30, 2010. Aviation Security: TSA Is Increasing Procurement and Deployment of Advanced Imaging Technology, but Challenges to This Effort and Other Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.: March 17, 2010. Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges. GAO-10-128. Washington, D.C.: October 7, 2009. GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP. Washington, D.C.: March 2009. (441087) Page 12 GAO-12-826T This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no Obtaining Copies of cost is through GAO’s website (www.gao.gov). Each weekday afternoon, GAO Reports and GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, Testimony go to www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. Contact: To Report Fraud, Waste, and Abuse in Website: www.gao.gov/fraudnet/fraudnet.htm E-mail: email@example.com Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Katherine Siggerud, Managing Director, firstname.lastname@example.org, (202) 512- Congressional 4400, U.S. Government Accountability Office, 441 G Street NW, Room Relations 7125, Washington, DC 20548 Chuck Young, Managing Director, email@example.com, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Please Print on Recycled Paper.
Aviation Security: Status of TSA's Acquisition of Technology for Screening Passenger Identification and Boarding Passes
Published by the Government Accountability Office on 2012-06-19.
Below is a raw (and likely hideous) rendition of the original report. (PDF)