oversight

Information Technology: DHS Needs to Enhance Management of Cost and Schedule for Major Investments

Published by the Government Accountability Office on 2012-09-26.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Requesters




September 2012
                 INFORMATION
                 TECHNOLOGY
                 DHS Needs to
                 Enhance Management
                 of Cost and Schedule
                 for Major Investments




GAO-12-904
                                               September 2012

                                               INFORMATION TECHNOLOGY
                                               DHS Needs to Enhance Management of Cost and
                                               Schedule for Major Investments
Highlights of GAO-12-904, a report to
congressional requesters




Why GAO Did This Study                         What GAO Found
DHS has responsibility for the                 Approximately two-thirds of the Department of Homeland Security’s (DHS) major
development and management of the              information technology (IT) investments are meeting their cost and schedule
IT systems for the 22 federal agencies         commitments (i.e., goals). Specifically, out of 68 major IT investments in
and offices under its jurisdiction. Of its     development, 47 were meeting cost and schedule commitments. The remaining
363 IT investments, 68 are in                  21—which total about $1 billion in spending—had one or more subsidiary
development and are classified by              projects that were not meeting cost and/or schedule commitments (i.e., they
DHS as a “major” investment that               exceeded their goals by at least 10 percent, which is the level at which the Office
requires special management attention          of Management and Budget (OMB) considers projects to be at increased risk of
because of its mission importance.
                                               not being able to deliver planned capabilities on time and within budget.)
Given the size and significance of
these investments, GAO was asked to            The primary causes for the cost and schedule shortfalls were (in descending
determine the (1) extent to which DHS          order of frequency):
IT investments are meeting their cost
and schedule commitments, (2)                  •   inaccurate preliminary cost and schedule estimates,
primary causes of any commitment               •   technology issues in the development phase,
shortfalls, and (3) adequacy of DHS’s          •   changes in agency priorities,
efforts to address these shortfalls and        •   lack of understanding of user requirements, and
their associated causes.                       •   dependencies on other investments that had schedule shortfalls.
To address these objectives, GAO               Eight investments had inaccurate cost and schedule estimates. For example,
analyzed recent cost and schedule              DHS’s Critical Infrastructure Technology investment had a project where actual
performance for DHS’s major IT                 costs were about 16 percent over the estimated cost, due in part to project staff
investments, as reported to OMB. To
                                               not fully validating cost estimates before proceeding with the project. In addition,
identify the primary cause(s) of any
                                               six investments had technical issues in the development phase that caused cost
shortfalls and whether any corrective
efforts were being taken to address
                                               or schedule slippages. For example, DHS’s Land Border Integration investment
them, GAO analyzed project plans and           had problems with wireless interference at certain sites during deployment of
related documentation and interviewed          handheld devices used for scanning license plates, which caused a project to be
responsible DHS officials and                  about 2.5 months late. In past work on DHS investments, GAO has identified
compared the corrective efforts to             some of the causes of DHS’s shortfalls and made recommendations to
applicable criteria to assess their            strengthen management in these areas (e.g., cost estimating, requirements), and
adequacy.                                      DHS has initiated efforts to implement the recommendations.

What GAO Recommends                            DHS often did not adequately address shortfalls and their causes. GAO’s
                                               investment management framework calls for agencies to develop and document
GAO is recommending that the                   corrective efforts to address underperforming investments. DHS policy requires
Secretary of Homeland Security direct          documented corrective efforts when investments experience cost or schedule
the appropriate officials to address the       variances. Although 12 of the 21 investments with shortfalls had defined and
guidance shortcomings and develop              documented corrective efforts, the remaining 9 did not. Officials responsible for 3
corrective actions for all major IT            of the 9 investments said they took corrective efforts but were unable to provide
investment projects having cost and
                                               plans or any other related documentation showing such action had been taken.
schedule shortfalls. In commenting on
                                               Officials for the other 6 investments cited criteria in DHS’s policy that excluded
a draft of this report, DHS concurred
with GAO’s recommendations.
                                               their investments from the requirement to document corrective efforts. This
                                               practice is inconsistent with the direction of OMB guidance and related best
                                               practices that stress developing and documenting corrective efforts to address
                                               problems in such circumstances. Until DHS addresses its guidance shortcomings
                                               and ensures each of these underperforming investments has defined and
View GAO-12-904. For more information,         documented corrective efforts, these investments are at risk of continued cost
contact David A. Powner at (202) 512-9286 or   and schedule shortfalls.
pownerd@gao.gov.

                                                                                        United States Government Accountability Office
Contents


Letter                                                                                     1
               Background                                                                  3
               DHS Meeting Cost and Schedule Commitments for Most Major IT
                 Investments                                                             10
               Causes of Investment Cost and Schedule Shortfalls Vary                    13
               About Half of DHS’s Investments with Shortfalls Do Not Have Well-
                 Developed Corrective Efforts                                            19
               Conclusions                                                               23
               Recommendations for Executive Action                                      23
               Agency Comments and Our Evaluation                                        24

Appendix I     Objectives, Scope, and Methodology                                        25



Appendix II    List of DHS Major IT Investments that Are Meeting Cost and Schedule
               Commitments                                                               28



Appendix III   List of DHS Major IT Investments and Subsidiary Projects with Shortfalls
                                                                                      35



Appendix IV    Comments from the Department of Homeland Security                         42



Appendix V     GAO Contact and Staff Acknowledgments                                     44



Tables
               Table 1: DHS’s Principal Component Organizations and their
                        Missions                                                           5
               Table 2: DHS Major IT Investments with Cost and Schedule
                        Shortfalls (dollars in millions)                                 10
               Table 3: Primary Causes of Shortfalls Experienced by Major DHS
                        IT Investments (in descending order of frequency)                13
               Table 4: Extent to Which DHS Had Developed and Documented
                        Corrective Efforts for Investment Shortfalls                     19



               Page i                                     GAO-12-904 DHS Major IT Investments
         Table 5: DHS Major IT Investments and Subsidiary Projects and
                  Planned Project Costs (in millions)                                              28
         Table 6: List of DHS Major IT Investments with Cost and/or
                  Schedule Shortfalls and Subsidiary Projects (cost in
                  millions)                                                                        35


Figure
         Figure 1: DHS Organizational Structure                                                     4




         Abbreviations

         CBP               U.S. Customs and Border Protection
         CIO               chief information officer
         DHS               Department of Homeland Security
         FEMA              Federal Emergency Management Agency
         ICE               U.S. Immigration and Customs Enforcement
         IT                information technology
         NPPD              National Protection and Programs Directorate
         OMB               Office of Management and Budget
         SBInet            Secure Border Network
         TSA               Transportation Security Administration
         USCIS             U.S. Citizenship and Immigration Services
         USCG              U.S. Coast Guard
         USSS              U.S. Secret Service
         US-VISIT          United States Visitor and Immigrant Status Indicator
                           Technology


         This is a work of the U.S. government and is not subject to copyright protection in the
         United States. The published product may be reproduced and distributed in its entirety
         without further permission from GAO. However, because this work may contain
         copyrighted images or other material, permission from the copyright holder may be
         necessary if you wish to reproduce this material separately.




         Page ii                                              GAO-12-904 DHS Major IT Investments
United States Government Accountability Office
Washington, DC 20548




                                   September 26, 2012

                                   The Honorable Thomas R. Carper
                                   Chairman
                                   Subcommittee on Federal Financial Management, Government
                                   Information, Federal Services and International Security
                                   Committee on Homeland Security and Governmental Affairs
                                   United States Senate

                                   The Honorable Michael T. McCaul
                                   Chairman
                                   The Honorable William R. Keating
                                   Ranking Member
                                   Subcommittee on Oversight, Investigations, and Management
                                   Committee on Homeland Security
                                   House of Representatives

                                   As part of its mission to protect the homeland, the Department of
                                   Homeland Security (DHS) is responsible for 22 agencies and offices with
                                   related missions and for the development and management of the
                                   information technology (IT) investments and systems used by those
                                   organizations. For its approximately 363 ongoing IT investments for fiscal
                                   year 2012, DHS plans to spend about $5.6 billion. Of these 363
                                   investments, 68 are under development and are classified by DHS as a
                                   “major” investment 1 that requires special management attention because
                                   of its importance to DHS’s mission.

                                   Given the size and significance of these investments, you asked us to
                                   determine the (1) extent to which DHS IT investments are meeting their
                                   cost and schedule commitments, (2) primary causes of any commitment
                                   shortfalls, and (3) adequacy of DHS’s efforts to address these shortfalls
                                   and their associated causes. In addition to this report, we have also




                                   1
                                       DHS defines a major IT Investment as one whose cost is $50 million or more.




                                   Page 1                                               GAO-12-904 DHS Major IT Investments
recently issued a report focused on the management and performance of
DHS’s major acquisitions. 2

To address these objectives, we analyzed how each of these 68 major
investments was performing against its cost and schedules commitments 3
as reported by the department to the Office of Management and Budget
(OMB) as of March 2012. In doing this, we identified investments that had
one or more projects exceeding 10 percent of their cost and schedule
commitments. We focused on these investments and on their subsidiary
projects 4 because they are under development and because OMB
considers them to be at an increased level of risk of not being able to
deliver promised capabilities on time and within budget, and thus they
require special attention from management. For each of these
investments and associated projects experiencing such shortfalls, we
reviewed project plans and related documentation and interviewed
responsible DHS officials to identify the primary cause(s) for the shortfalls
and whether any corrective efforts had been developed and documented
to address the shortfalls. We also compared the corrective efforts to
applicable criteria to assess their adequacy.

We conducted this performance audit from October 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. Further


2
  In that report—entitled GAO, Homeland Security: DHS Requires More Disciplined
Investment Management to Help Meet Mission Needs, GAO-12-833 (Washington, D.C.:
Sept. 18, 2012)—we assessed the extent to which (1) DHS’s major acquisition programs
face challenges that increase the risk of poor outcomes; (2) DHS has policies and
processes in place to effectively manage individual acquisition programs; (3) DHS has
policies and processes in place to effectively manage its portfolio of acquisition programs
as a whole; and (4) DHS has taken actions to address the high-risk acquisition
management issues we have identified in previous reports. The report focused on, among
other things, the extent to which major acquisitions were achieving their cost, schedule,
and capability objectives and whether the department had policies in place to effectively
manage these acquisition programs.
3
  Commitments are cost and schedule goals or targets that the department defined for the
investment in its investment planning and approval documents, such as cost and schedule
baselines.
4
  Major IT investments—which are typically multiyear efforts— are made up of one or
more subsidiary projects. A project is a temporary effort (e.g., 9 months) to accomplish a
unique product or service such as adding enhancements to a system.




Page 2                                                GAO-12-904 DHS Major IT Investments
             details on our objectives, scope, and methodology are included in
             appendix I.


             DHS’s mission is to lead the unified national effort to secure America by
Background   preventing and deterring terrorist attacks and protecting against and
             responding to threats and hazards to the nation. DHS is also responsible
             for ensuring that the nation’s borders are safe and secure, that they
             welcome lawful immigrants and visitors, and that they promote the free
             flow of commerce.

             Created in 2002, DHS assumed control of about 209,000 civilian and
             military positions from 22 agencies and offices that specialize in one or
             more aspects of homeland security. The purpose behind the merger was
             to improve coordination, communication, and information sharing among
             these multiple federal agencies. Figure 1 shows DHS’s organizational
             structure and table 1 identifies DHS’s principal organizations and
             describes their missions.




             Page 3                                     GAO-12-904 DHS Major IT Investments
Figure 1: DHS Organizational Structure




                                         Page 4   GAO-12-904 DHS Major IT Investments
Table 1: DHS’s Principal Component Organizations and their Missions
                           a
Principal organizations                    Mission
U.S. Customs and Border Protection         Protects the nation’s borders to prevent terrorists and terrorist weapons from entering the
(CBP)                                      United States, while facilitating the flow of legitimate trade and travel.
Domestic Nuclear Detection Office          Protects the nation by detecting and reporting unauthorized attempts to import, possess,
                                           store, develop, or transport nuclear or radiological material for use against the nation.
Federal Emergency Management Agency        Prepares the nation for hazards, manages federal response and recovery efforts
(FEMA)                                     following any national incident, and administers the National Flood Insurance Program.
U.S. Immigration and Customs               Protects the nation’s borders by identifying and limiting vulnerabilities in the nation’s
Enforcement (ICE)                          border, economic, transportation, and infrastructure security.
Intelligence and Analysis                  Works closely with DHS components, as well as state, local, and tribal entities, to fuse
                                           nontraditional and traditional intelligence information streams into national threat
                                           assessments, and disseminates the resulting information to DHS and external homeland
                                           security customers.
Management Directorate                     Oversees department budgets and appropriations, expenditure of funds, accounting and
                                           finance, procurement, human resources, IT, facilities and equipment, and identifies and
                                           tracks performance measurements. Includes the Offices of the Chief Information Officer,
                                           Chief Financial Officer, and Chief Procurement Officer.
National Protection and Programs           Works with state, local, and private sector partners to identify threats, determine
Directorate (NPPD)                         vulnerabilities, and target resources where risk is greatest to safeguard the nation’s
                                           critical physical and cyber infrastructures.
Office of Health Affairs                   Protects the nation against biohazards through coordinated efforts with all levels of
                                           government and the private sector to develop and support a scientifically rigorous,
                                           intelligence-based biodefense and health preparedness architecture.
Office of Operations and Coordination      Monitors the security of the United States and coordinates activities within the department
Planning                                   and with governors, homeland security advisors, law enforcement partners, and critical
                                           infrastructure operators in all 50 states and more than 50 major urban areas nationwide.
Transportation Security Administration     Protects the nation’s transportation systems to ensure freedom of movement for people
(TSA)                                      and commerce.
U.S. Citizenship and Immigration Services Administers immigration and naturalization adjudication functions and establishes
(USCIS)                                   immigration services policies and priorities.
U.S. Coast Guard (USCG)                    Protects the public, the environment, and U.S. economic interests in the nation’s ports
                                           and waterways, along the coast, in international waters, and in any maritime region as
                                           required to support national security.
U.S. Secret Service (USSS)                 Protects the President and other high-level officials and investigates counterfeiting and
                                           other financial crimes, including financial institution fraud, identity theft, computer fraud;
                                           and computer-based attacks on our nation’s financial, banking, and telecommunications
                                           infrastructure.
                                           Source: GAO analysis of DHS data.
                                           a
                                            This table does not show the organizations that fall under each of the directorates. It also does not
                                           show all organizations that report directly to the DHS Secretary and Deputy Secretary, such as
                                           executive secretary, legislative and intergovernmental affairs, public affairs, chief of staff, inspector
                                           general, and general counsel.


                                           Within the department’s Management Directorate, headed by the Under
                                           Secretary for Management, is the Office of the Chief Information Officer



                                           Page 5                                                         GAO-12-904 DHS Major IT Investments
                      (CIO). The CIO’s responsibilities include setting departmental IT policies,
                      processes, and standards, and ensuring that IT acquisitions comply with
                      DHS IT management processes, technical requirements, and approved
                      enterprise architecture, among other things. Additionally, the CIO chairs
                      DHS’s Chief Information Officer Council (CIO Council), which is
                      responsible for ensuring the development of IT resource management
                      policies, processes, best practices, performance measures, and decision
                      criteria for managing the delivery of IT services and investments, while
                      controlling costs and mitigating risks.

DHS IT Investments    DHS spends billions of dollars each year on IT investments to perform
Support Its Mission   both mission-critical and support functions that frequently must be
                      coordinated among components, as well as among external entities. Of
                      the $5.6 billion that DHS plans to spend on 363 IT-related investments in
                      fiscal year 2012, $4.4 billion is planned for the 83 the agency considers to
                      be a major investment; namely, costly, complex, and/or mission critical.

                      Of these 83 major IT investments, 68 are under development and have
                      planned fiscal year 2012 costs of approximately $4 billion. Examples of
                      major investments under development that are being undertaken by DHS
                      and its components include:

                      •   U. S. Customs and Border Protection—Automated Commercial
                          Environment/International Trade Data System will incrementally
                          replace existing cargo processing technology systems with a single
                          system for land, air, rail, and sea cargo and serve as the central data
                          collection system for federal agencies needing access to international
                          trade data in a secure, paper-free, web-enabled environment.

                      •   Immigration and Customs Enforcement —TECS Modernization is
                          to replace the legacy mainframe system developed by the U.S.
                          Customs Service in the 1980s to support its inspections and
                          investigations. Following the creation of DHS, those activities were
                          assigned to CBP and ICE, respectively. CBP and ICE are now
                          working to modernize their respective portions of the system in a
                          coordinated effort with separate funding and schedules. ICE’s portion
                          of the investment will include modernizing the investigative case
                          management and related support modules of the legacy system.

                      •   National Protection and Programs Directorate—National
                          Cybersecurity Protection System, also referred to as EINSTEIN, is
                          an integrated system that includes intrusion detection, analytics,
                          intrusion prevention, and information sharing capabilities that are used



                      Page 6                                      GAO-12-904 DHS Major IT Investments
                               to defend the federal executive branch civilian agencies’ IT
                               infrastructure from cyber threats. It consists of the hardware, software,
                               supporting processes, training, and services that are being developed
                               and acquired to support DHS’s mission requirements.


Best Practices and OMB     The success of major IT investments are judged by, among other things,
Guidance Stress            the extent to which they deliver promised system capabilities and mission
Importance of Keeping IT   benefits on time and within cost. Consequently, our best practices
                           research and extensive experience at federal agencies, as well as OMB
Investments within Cost    guidance, stress the importance of federal IT investments meeting cost
and on Schedule            and schedule milestones.

                           GAO’s Information Technology Investment Management guidance 5
                           highlights the need to regularly determine each IT project’s progress
                           toward cost and schedule milestones using established criteria and calls
                           for corrective efforts when milestones are not being met. The guidance
                           also calls for such corrective efforts to be defined and documented.
                           OMB plays a key role in helping federal agencies manage their
                           investments by working with them to better plan, justify, and determine
                           how much they need to spend on projects and how to manage their
                           approved projects. In December 2010, OMB issued its 25 Point
                           Implementation Plan to Reform Federal Information Technology
                           Management, 6 a plan to change IT management throughout the federal
                           government by strengthening the role of investment review boards to
                           enable them to more adequately manage agency IT portfolios, redefining
                           the role of agency CIOs and the Federal CIO Council 7 to focus on
                           portfolio management, and implementing face-to-face reviews to identify
                           IT investments that are experiencing performance problems and to select



                           5
                             GAO, Information Technology Investment Management: A Framework for Assessing and
                           Improving Process Maturity, GAO-04-394G (Washington, D.C.: Mar. 1, 2004).
                           6
                             Office of Management and Budget, 25 Point Implementation Plan to Reform Federal
                           Information Technology Management (Washington, D.C.: Dec. 9, 2010).
                           7
                             The CIO Council is comprised of the CIOs and Deputy CIOs of 28 agencies and is
                           chaired by OMB’s Deputy Director for Management. It is the principal interagency forum
                           for improving agency practices related to the design, acquisition, development,
                           modernization, use, sharing, and performance of federal information resources and is
                           responsible for developing recommendations for overall federal IT management policy,
                           sharing best practices, including the development of performance measures, and
                           identifying opportunities and sponsoring cooperation in using information resources.




                           Page 7                                              GAO-12-904 DHS Major IT Investments
                         them for a TechStat session—a review of selected IT investments
                         between OMB and agency leadership that is led by the Federal CIO.
                         In addition, OMB provides agencies with tools to measure how effectively
                         investments are meeting established cost and schedule parameters.
                         Specifically, OMB requires federal agencies to provide information on
                         their IT investments as a part of their yearly budget submissions, and to
                         do so using an exhibit 53, in which they list all of their IT investments and
                         their associated costs, and an exhibit 300, also called the Capital Asset
                         Plan and Business Case, which includes an investment’s cost and
                         schedule commitments.

                         Further, in June 2009, OMB deployed the IT Dashboard, a website that
                         displays near real-time information on, among other things, the cost and
                         schedule performance of all of an agency’s major IT investments. The IT
                         Dashboard provides, among other things, a cost and schedule
                         performance rating for each major IT investment’s subsidiary project.
                         These ratings are based on the extent to which the project is meeting its
                         cost and schedule commitments. For example, projects experiencing a 10
                         percent or greater cost and/or schedule variance are considered to be at
                         an elevated risk of not delivering promised capabilities on time and within
                         budget, and, as such, require management attention.


Prior GAO Reports        We have previously reported on the cost and schedule challenges
Highlighted Cost and     associated with major DHS IT investments, such as those with CBP’s
Schedule Shortfalls of   Secure Border Network (SBInet) and NPPD’s United States Visitor and
                         Immigrant Status Indicator Technology (US-VISIT). For example, in 2007
Major DHS Investments    we reported that the Secure Border Network had experienced significant
                         cost and schedule shortfalls due, in part, to the project not having fully
                         defined activities. In addition, in May 2010, we reported that continued
                         delays to the investment were likely because, among other things, it had
                         not developed a reliable integrated master schedule and the schedule did
                         not adequately capture all necessary activities. 8 In these reports, we
                         made recommendations to strengthen the program weaknesses to keep
                         the investment on schedule and within cost.



                         8
                          GAO, Secure Border Initiative: SBInet Expenditure Plan Needs to Better Support
                         Oversight and Accountability, GAO-07-309 (Washington, D.C.: Feb. 15, 2007) and Secure
                         Border Initiative: DHS Needs to Reconsider Its Proposed Investment in Key Technology
                         Program, GAO-10-340 (Washington, D.C.: May 5, 2010).




                         Page 8                                            GAO-12-904 DHS Major IT Investments
With regard to the US-VISIT investment, we noted in a November 2009
report that officials had not adopted an integrated approach to scheduling,
executing, and tracking the work that needed to be accomplished to
deliver the Comprehensive Exit project to more than 300 ports of entry on
schedule and within cost. 9 Accordingly, we recommended that DHS
strengthen management of the project by ensuring that it develop and
maintain integrated scheduling plans in accordance with applicable key
practices; DHS concurred with the recommendations.

Further, in 2011, as a part of our High Risk series, 10 we reported that,
because of acquisition weaknesses, major investments, such as the
recently canceled SBInet, continued to be challenged in meeting
capability, benefit, cost, and schedule expectations. Based on our prior
work, we identified and provided to DHS key actions and outcomes
critical to addressing this and other challenges.

Most recently, we reported 11 in July 2012 that DHS was making progress
in developing and implementing a new IT governance process. We found
that DHS had developed a new governance framework and that the
associated policies and procedures were generally consistent with recent
OMB guidance and with best practices for managing projects and
portfolios identified in GAO’s Information Technology Investment
Management framework; however, the agency had not yet finalized most
policies and procedures and was not fully using best practices for the
implementation. Accordingly, we made recommendations to DHS to,
among other things, strengthen its new governance process and related
IT management capabilities; the agency agreed to implement the
recommendations.




9
 GAO, Homeland Security: Key US-VISIT Components at Varying Stages of Completion,
but Integrated and Reliable Schedule Needed, GAO-10-13 (Washington, D.C.: Nov. 19,
2009).
10
     GAO, High Risk Series: An Update, GAO-11-278 (Washington, D.C.: February 2011).
11
  GAO, Information Technology: DHS Needs to Further Define and Implement Its New
Governance Process, GAO-12-818 (Washington, D.C.: July 2012).




Page 9                                             GAO-12-904 DHS Major IT Investments
                                         As discussed previously, our best practices research and experience at
DHS Meeting Cost                         federal agencies as well as OMB guidance stress the importance of
and Schedule                             investments meeting their cost and schedule commitments. OMB requires
                                         agencies to report to the IT Dashboard information on the cost and
Commitments for                          schedule performance of all their major IT investments.
Most Major IT
Investments                              Our analysis of the cost and schedule performance for DHS’s 68 major IT
                                         investments shows that approximately two-thirds of these investments
                                         and their subsidiary projects were meeting cost and schedule
                                         commitments; the remaining one-third had at least one subsidiary project
                                         that was not meeting its commitments. Specifically, out of the 68 major
                                         investments under development, 47 were meeting their cost and schedule
                                         commitments. (See app. II for a listing of the 47 investments and
                                         subsidiary projects that are meeting their commitments.) The remaining
                                         21 investments had one or more subsidiary projects that were not
                                         meeting cost and/or schedule commitments; the total planned cost for all
                                         projects in development for the 21 investments is approximately $1 billion.
                                         Table 2 lists the investments experiencing cost and/or schedule shortfalls,
                                         and the total planned project cost for each investment. A list of the
                                         investments and their subsidiary projects experiencing cost and/or
                                         schedule shortfalls is included in appendix III.




Table 2: DHS Major IT Investments with Cost and Schedule Shortfalls (dollars in millions)

                                                     One or more        One or more     One or more projects
                                                   projects with a    projects with a        with a cost and     Total planned
                                                                                                                              a
Component         Investment                        cost shortfall schedule shortfall     schedule shortfall      project cost




                                         Page 10                                            GAO-12-904 DHS Major IT Investments
                                                           One or more        One or more     One or more projects
                                                         projects with a    projects with a        with a cost and     Total planned
                                                                                                                                    a
Component           Investment                            cost shortfall schedule shortfall     schedule shortfall      project cost
CBP
                    Automated Commercial                                                                                    $124.26
                    Environment/International
                    Trade Data System
                    Land Border Integration                                                                                    20.9
                    Non-intrusive Inspection                                                                                  332.3
                    Systems Program
                    Northern Border, Remote                                                                                      8.2
                    Video Surveillance System
                    TECS Modernization                                                                                        43.03
DHS
Office of the
Chief Information
Officer
                    Human Resources IT                                                                                         8.52
FEMA
                    Disaster Assistance                                                                                      50.5
                    Improvement Plan
ICE
                    Detention and Removal                                                                                      8.62
                    Operations Modernization
NPPD
                    Critical Infrastructure                                                                                   20.55
                    Technology and Architecture
                    Infrastructure Security                                                                                   72.76
                    Compliance-Chemical
                    Security Assessment Tool
                    National Cybersecurity                                                                                    262.6
                    Protection System
                    Next Generation Networks                                                                                  63.06
                    Priority Services
                    US-VISIT: Arrival and                                                                                      7.18
                    Departure Information
                    System
                    US-VISIT: Automated                                                                                       33.24
                    Biometric Identification
                    System
TSA
                    Air Cargo Security                                                                                       4.09
                    Federal Air Marshal Service                                                                                5.43
                    Mission Scheduling and
                    Notification System



                                               Page 11                                            GAO-12-904 DHS Major IT Investments
                                                  One or more        One or more                    One or more projects
                                                projects with a    projects with a                       with a cost and     Total planned
                                                                                                                                          a
Component   Investment                           cost shortfall schedule shortfall                    schedule shortfall      project cost
            Hazmat Threat Assessment                                                                                                  4.09
            Program
            Security Technology                                                                                                      27.99
            Integrated Program
USCG
            CG Business Intelligence                                                                                                      .86
            Naturalization: CLAIMS 4                                                                                                  2.36
USSS
            Information Integration and                                                        
            Technology Transformation                                                                                                 43.61
                                                                                                                                             b
TOTAL                                    21                         5                          18                      2        $1,144.14
                                    Source: GAO analysis of OMB’s federal IT Dashboard data.
                                    a
                                        These are the total planned costs of all investment projects in development as of March 8, 2012.
                                    b
                                        Differences in total are rounded off.


                                    Of the 21 investments with a shortfall, 5 had one or more subsidiary
                                    project with a cost shortfall, 18 had one or more project with a schedule
                                    shortfall, and 2 had a project with both a cost and schedule shortfall.
                                    These shortfalls potentially impact the total cost of investments and can
                                    delay the implementation of key systems. For example:

                                    •       TSA’s Federal Air Marshal Service Mission Scheduling and
                                            Notification System: project to modernize the core scheduling
                                            software component of the system, which, among other things,
                                            determines the allocation of federal air marshals to flights and
                                            coordinates and communicates mission assignments, was delayed.

                                    •       NPPD’s Critical Infrastructure Technology and Architecture
                                            investment: project to develop an information-sharing application to be
                                            used by federal, state, and local stakeholders to increase their
                                            capability to combat terrorist use of improvised explosive devices had
                                            cost overruns of approximately 16 percent ($296,000).

                                    •       CBP’s Northern Border, Remote Video Surveillance System
                                            investment: project to incorporate IT Security improvements to the
                                            remote video surveillance systems in Buffalo, New York, and Detroit,
                                            Michigan, was delayed by approximately 2 months.

                                    •       FEMA’s Disaster Assistance Improvement Plan: a subsidiary
                                            project—site usability enhancements—that included enhancements to



                                    Page 12                                                             GAO-12-904 DHS Major IT Investments
                                                the DisasterAssistance.gov website to improve usability by making it
                                                easier and more intuitive for users to apply for and find information
                                                about disaster assistance from federal, state, local, tribal, and private
                                                nonprofit organizations was delayed.

                                            The primary causes of the shortfalls in cost and schedule associated with
Causes of Investment                        DHS’s 21 major IT investments were (in descending order of frequency):
Cost and Schedule                           inaccurate preliminary cost and schedule estimates, technical issues in
                                            the development phase, changes in agency priorities, lack of
Shortfalls Vary                             understanding of user requirements, and dependencies on other
                                            investments that had schedule shortfalls. A summary of these causes by
                                            investment and the associated component are shown in table 3 and are
                                            followed by (1) our analysis of these causes by category and (2)
                                            discussion of our past work on the department’s major investments and
                                            related IT management processes where we identified some of these
                                            same causes and made recommendations to strengthen management in
                                            these areas.

Table 3: Primary Causes of Shortfalls Experienced by Major DHS IT Investments (in descending order of frequency)

                                             Inaccurate    Technical                      Lack of
                                            preliminary     issues in    Changes understanding      Dependencies
                                          cost/schedule development     in agency         of user        on other     Other
Causes                                        estimates        phase     priorities requirements     investments     causes
Component Investment
CBP            Automated Commercial                                
               Environment /
               International Trade Data
               System
               Land Border Integration                             
               Non-Intrusive Inspection               
               Systems Program
               Northern Border, Remote                
               Video Surveillance
               System
               TECS Modernization                                                              
DHS Office     Human Resources IT                     
of the Chief
Information
Officer
FEMA           Disaster Assistance                                                                           
               Improvement Plan
ICE            Detention and Removal                                                          
               Operations Modernization




                                            Page 13                                        GAO-12-904 DHS Major IT Investments
                                         Inaccurate    Technical                                     Lack of
                                        preliminary     issues in                   Changes understanding      Dependencies
                                      cost/schedule development                    in agency         of user        on other     Other
Causes                                    estimates        phase                    priorities requirements     investments     causes
NPPD     Critical Infrastructure                       
         Technology and
         Architecture
         Infrastructure Security                                                                                                     
         Compliance: Chemical
         Security Assessment Tool
         National Cybersecurity                        
         Protection System
         Next Generation                                                                 
         Networks Priority Services
         US-VISIT: Arrival and                                                            
         Departure Information
         System
         US-VISIT: Automated                                                              
         Biometric Identification
         System
TSA      Air Cargo Security                                                                                              
         Federal Air Marshal                                                                                                     
         Service Mission
         Scheduling and
         Notification System
         Hazmat Threat                                 
         Assessment Program
         Security Technology                                                   
         Integrated Program
USCG     CG Business Intelligence                                                         
USCIS    Naturalization-CLAIMS 4                                                                          
USSS     Information Integration                                                                                          
         and Technology
         Transformation
Totals                                                 8                       6           4              3                3          2
                                        Source: GAO analysis of agency data.



                                        Specifically, our analysis of these causes by category showed:

                                        Inaccurate preliminary cost and schedule estimates: Inaccurate cost
                                        and schedule estimates in eight investments resulted in significant cost
                                        and schedule increases. For example:

                                        •     Preliminary schedule estimates for a project under CBP’s Non-
                                              Intrusive Inspection Systems Program investment—which supports
                                              the detection and prevention of contraband from entering the


                                        Page 14                                                       GAO-12-904 DHS Major IT Investments
    country—were inaccurate due to underestimating the time needed to
    complete a key task. Specifically, project officials did not accurately
    estimate how long it would take to complete an environmental
    assessment because they did not consider all requirements in their
    initial planning, thus resulting in a schedule delay of approximately 2
    months.

•   The NPPD investment called Critical Infrastructure Technology and
    Architecture had a project—integral to developing an information
    sharing application to be used by federal, state, and local
    stakeholders to increase their capability to combat terrorist use of
    improvised explosive devices—where actual costs for completing
    critical tasks were about 16 percent over the cost estimated at project
    initiation. According to investment officials, this was due in part to
    project staff developing the cost estimates very quickly and not fully
    validating them before proceeding with the project.

•   TSA’s Hazmat Threat Assessment Program (which performs a threat
    assessment on commercial truck drivers who transport hazardous
    materials to determine the threat status to transportation security) had
    a schedule shortfall with a project, because, in part, the time needed
    to modify a contract was not accurately estimated, which led to a
    schedule delay of nearly 3 months.
Technology issues in the development phase: Technical issues in the
development phase caused cost or schedule slippages in six
investments. Examples include:

•   Changes made to one part of ICE’s Detention and Removal
    Operations Modernization investment, which is designed to upgrade
    IT capabilities to support efficient detention and removal of non-U.S.
    citizens, created a cascading effect, leading to changes to other parts
    of the system and contributed to delays of more than a month.

•   Issues in establishing a testing and development environment that
    matched the production environment delayed project testing in several
    projects under FEMA’s Disaster Assistance Improvement Plan
    investment (which is to ease the burden on disaster survivors by
    providing them with a mechanism to access and apply for disaster
    assistance).

•   Technical complications during deployment caused the schedule to
    slip by 79 days on a project under CBP’s Land Border Integration
    investment, which assists with the processing of inbound and



Page 15                                      GAO-12-904 DHS Major IT Investments
    outbound travel at border patrol checkpoints nationwide. Specifically,
    the handheld devices used for scanning license plates used a
    wireless spectrum that had interference problems at certain sites, and
    resolving this issue took more time than had been planned for.

Changes in agency priorities: Four investments experienced cost and
schedule slippages due to changing priorities at the agency level. In
particular,

•   The schedules were delayed for two NPPD US-VISIT investments:
    the Arrival and Departure Information System, which collects arrival
    and departure information on non-U.S. citizens traveling to the United
    States as well as current immigration status updates for each traveler,
    and the Automated Biometric Identification System, a fingerprint
    repository and biometric-matching system. Delays were due to a
    management decision to focus on accelerating the development of
    other investments or projects, which took resources (i.e., personnel)
    away from the investment. Consequently, the Arrival and Departure
    Information System’s fiscal year 2011 maintenance release project
    was delayed approximately 3 months, and the Automated Biometric
    Identification System’s fiscal year 2011 product support project was
    delayed by approximately 7 months.

•   A critical subsidiary project to deliver predictive analytical capabilities
    under USCG’s Business Intelligence investment, which is designed to
    reduce organizational uncertainty and risk in decision making, had a
    schedule delay of approximately 3 months due to changing priorities.
    Project officials said that Coast Guard management directed
    resources to other projects with a higher priority, thus limiting the
    ability to work on the predictive analytics capability project.

Lack of understanding user requirements: Three investments had
slippages resulting from misunderstanding or inadequately developed
user requirements and expectations.

•   A project under USCIS’s Claims 4 investment, which is a processing
    system for the adjudication of naturalization applications, was delayed
    by 2 weeks because inadequate user requirements led to a design
    flaw that required additional time to address.

•   Customer priorities and expectations for ICE’s Detention and Removal
    Operations Modernization investment changed over time, which
    contributed to schedule delays of more than a month.



Page 16                                       GAO-12-904 DHS Major IT Investments
•   The schedule for a CBP TECS Modernization investment project,
    which supports the screening of travelers entering the United States,
    was delayed due to users requesting that the application in
    development interface with a separate system. The project was
    delayed by 3 months while program officials developed new
    requirements.

Dependencies on other component’s investments that had schedule
shortfalls: Investments also encountered schedule slippages when
interdependent investments encountered delays. For example:

•   USSS’s Information Integration and Technology Transformation
    investment to provide advanced security measures to electronically
    send, receive, and track access to USSS’s unclassified and classified
    information was delayed approximately 6 months due to a
    component’s project being delayed.

•   Costs for a project under FEMA’s Disaster Assistance Improvement
    Plan investment rose approximately 27 percent ($210,000) due, in
    part, to the delayed deployment of another investment.

Other causes of cost and schedule slippages that were cited by
department officials included delays in receiving funding and gaps in
leadership due to key management turnover. Specifically,

•   The schedule for three projects under TSA’s Federal Air Marshal
    Service Mission Scheduling and Notification investment was delayed
    due to delays in receiving full funding. DHS had provided the
    investment with partial funding, and thus investment officials produced
    an investment plan based on that funding level; when full funding was
    subsequently restored, the plan had to be updated, which resulted in
    delays.

•   The costs for a key subsidiary project of NPPD’s Infrastructure
    Security Compliance, Chemical Security Assessment Tool, which is to
    provide for the electronic submission of chemical facility data and
    controlled use of such data, rose approximately 20 percent ($719,000)
    due, in part, to multiple director-level program changes, which led to
    corresponding changes in the investment’s vision and direction.

In our past work on DHS’s investments and related IT management
processes, we have identified some of these same causes and made
recommendations to strengthen management in these areas. For



Page 17                                     GAO-12-904 DHS Major IT Investments
example, with regard to cost estimating, we reported that forming a
reliable estimate of costs provides a sound basis for measuring against
actual cost performance and that the lack of such a basis contributes to
variances. 12 To help agencies establish such a capability, we issued a
guide in March 2009 that was based on the practices of leading
organizations. 13 In a July 2012 report examining how well DHS is
implementing these practices, we reported that the department had
weaknesses in cost estimating. 14 Accordingly, we made
recommendations to DHS to strengthen its cost estimating capabilities,
and the department has plans and efforts under way to implement our
recommendations.

We have also reported 15 that developing sufficient requirements is key to
effectively delivering systems on time and within budget and that DHS
has experienced project delays and cost overruns resulting from initial
requirements not being defined properly. To address this challenge, DHS
has begun, as part of defining and implementing a new IT governance
process, to establish centers of excellence to provide investment officials
with expert assistance in requirements development and other essential
IT management disciplines. 16 However, we also reported that DHS had
yet to fully define and implement these initiatives; accordingly, we made
recommendations to do so, and the agency agreed to implement the
recommendations and has efforts planned and under way to complete
implementation by September 2013.




12
 GAO, Information Technology Cost Estimation: Agencies Need to Address Significant
Weaknesses in Policies and Practices, GAO-12-629 (Washington, D.C.: July 2012).
13
  GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and
Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009).
14
     GAO-12-629.
15
   GAO, Department of Homeland Security: Assessments of Selected Complex
Acquisitions, GAO-10-588SP (Washington, D.C.: June 2010).
16
     GAO-12-818.




Page 18                                          GAO-12-904 DHS Major IT Investments
                                          A variety of best practices exist to guide the successful acquisition of IT
About Half of DHS’s                       investments, including how to develop and document corrective actions
Investments with                          for projects experiencing cost and schedule shortfalls. In particular,
                                          GAO’s Information Technology Investment Management framework calls
Shortfalls Do Not                         for agencies to develop and document corrective efforts for
Have Well-Developed                       underperforming projects. It also states that agencies are to ensure that,
Corrective Efforts                        as projects develop and costs rise, the project continues to meet mission
                                          needs at the expected levels of cost and risk; if projects are not meeting
                                          expectations or if problems have arisen, agencies are to quickly take
                                          steps to address the deficiencies. In addition, DHS policy 17 requires
                                          corrective actions when cost or schedule variances exceed 8 percent.

                                          DHS developed and documented corrective efforts for 12 of the 21 major
                                          investments with a shortfall, but the remaining 9 did not have documented
                                          corrective efforts. Table 4 depicts the investments with shortfalls and
                                          whether corrective efforts had been developed and documented.

Table 4: Extent to Which DHS Had Developed and Documented Corrective Efforts for Investment Shortfalls

Adequately developed and                                                                                          Yes    No
documented corrective efforts?
Component                         Investment
CBP                               Automated Commercial Environment / International Trade Data System                 
                                  Land Border Integration                                                            
                                  Non-Intrusive Inspection Systems Program                                           
                                  Northern Border, Remote Video Surveillance System                                  
                                  TECS Modernization                                                                 
DHS                               Human Resources IT                                                                        
Office of the Chief Information
Officer
FEMA                              Disaster Assistance Improvement Plan                                               
ICE                               Detention and Removal Operations Modernization                                     
NPPD                              Critical Infrastructure Technology and Architecture                                
                                  Infrastructure Security Compliance: Chemical Security Assessment Tool              
                                  National Cybersecurity Protection System                                                  




                                          17
                                            Department of Homeland Security, Acquisition Management Directive, Directive
                                          Number 102-01 (Jan. 20, 2010) and Department of Homeland Security, Capital Planning
                                          and Investment Control Guide, version 7.2 (October 2011).




                                          Page 19                                           GAO-12-904 DHS Major IT Investments
        Next Generation Networks Priority Services                                                 
        US-VISIT: Arrival and Departure Information System                                         
        US-VISIT: Automated Biometric Identification System                                        
TSA     Air Cargo Security                                                                         
        Federal Air Marshal Service Mission Scheduling and Notification System                     
        Hazmat Threat Assessment Program                                                    
        Security Technology Integrated Program                                              
USCG    Coast Guard Business Intelligence                                                          
USCIS   Naturalization-CLAIMS 4                                                                    
USSS    Information Integration and Technology Transformation                               
Total                                                                                      12       9
                Source: GAO analysis of DHS data.


                DHS took corrective actions to address investment shortfalls in 12
                investments. Actions for the 12 included:

                •     CBP: Automated Commercial Environment /International Trade Data
                      System had schedule shortfalls due to an inadequate testing and
                      development environment; they were resolved by leveraging CBP’s
                      disaster recovery site to perform the testing.

                •     CBP: Land Border Integration investment schedule delays due to
                      technical complications were resolved through the use of risk
                      management processes (e.g., identification, assessment, tracking,
                      and mitigation of risks) identified in the investment’s July 2011 risk
                      management plan, which addressed the cause of the investment
                      shortfalls.

                •     CBP: Non-Intrusive Inspection Systems Program, which had schedule
                      shortfalls due to inaccurate estimates, tracked the project status and
                      risks via project health status reports and other mitigation strategies.

                •     CBP: The Remote Video Surveillance System investment officials
                      developed and documented an investment rebaseline to address the
                      investment’s schedule shortfall, which was due to an inaccurate initial
                      project schedule estimate.

                •     CBP: The TECS Modernization investment—which had schedule
                      shortfalls due to system development being delayed due to questions
                      about whether planned enhancements duplicated functions performed
                      by another agency system—program officials briefed key




                Page 20                                            GAO-12-904 DHS Major IT Investments
    management on the differences between the system functions and
    development was allowed to continue.

•   FEMA: Disaster Assistance Improvement Plan cost and schedule
    shortfalls were due to, among other things, dependencies on other
    investments. DHS developed a remediation plan for each shortfall to
    limit the negative impact.

•   ICE: Detention and Removal Operations Modernization investment
    schedule shortfalls were due in part to a lack of understanding of user
    requirements; to address these issues, investment officials worked
    with key stakeholders to engage users to more thoroughly identify
    user requirements.

•   NPPD: Critical Infrastructure Technology and Architecture
    investment’s cost shortfalls, which were due to inaccurate initial cost
    estimates, were resolved by investment officials through several
    corrective efforts, including completing the project’s life cycle cost
    estimate.

•   NPPD: Infrastructure Security Compliance, Chemical Security
    Assessment Tool investment schedule was delayed due to multiple
    changes in leadership and in the investment’s direction. Project
    officials developed and documented an investment rebaseline, which
    was approved in February 2012. It was intended to, among other
    things, develop a more accurate schedule.

•   TSA: The Hazmat Threat Assessment Program investment schedule
    was delayed because the time needed to adjust a contract had not
    been accurately estimated. In response, investment officials
    documented an investment rebaseline, which was approved in March
    2012.

•   TSA: The Security Technology Integrated Program investment had
    schedule shortfalls from inaccurate estimates of the time needed to
    revise a contract. To resolve these issues, officials developed and
    documented an initiative to improve methods used to identify and
    track risks and resolve the schedule shortfalls. This effort is intended
    to help the investment avoid additional schedule changes.

•   USSS: The Information Integration and Technology Transformation
    investment had one project with schedule shortfalls due to
    dependencies on another component’s investment that had schedule




Page 21                                      GAO-12-904 DHS Major IT Investments
     slippages; issues were addressed by following the mitigation actions
     detailed in the investment’s risk management plan.

With regard to the remaining nine investments, three were unable to
provide us with documentation, even though project officials stated that
they had developed some corrective efforts, and six did not engage in
corrective efforts to address shortfalls. Of the three investments, officials
from TSA’s Federal Air Marshal Service Mission Scheduling and
Notification System investment, for example, reported that they had
addressed the project’s schedule shortfall—which was due, in part, to a
support contractor not having adequate staffing—by performing the work
within the agency instead of relying on the contractor. Further, according
to TSA officials, the cost and schedule shortfalls on the Air Cargo Security
investment, which were due to technical complications and dependencies
on other investments, were addressed by establishing a new cost and
schedule baseline. Nonetheless, this lack of documentation is
inconsistent with the direction of DHS’s guidance and related best
practices, and it shows a lack of process discipline and attention to key
details, which raises concern about the thoroughness of corrective efforts.

Of the six investments without any corrective efforts, officials from these
investments (namely, the Office of the Chief Information Officer’s Human
Resources IT investment, NPPD’s US-VISIT Automated Biometric
Identification System and Arrival and Departure Information System
investments, USCG’s Business Intelligence investment, NPPD’s National
Cybersecurity Protection System, and USCIS’s Claims 4 investment),
stated that they did not develop and document corrective efforts because
they believed DHS’s guidance does not call for it in their circumstances.
Specifically, the officials said that although DHS’s guidance 18 calls for
corrective actions to be developed and documented when an investment
or its projects experiences a life cycle cost or schedule variance of 8
percent or greater, the variances on their project activities thus far were
not large enough to constitute such a life cycle variance.

The impact of this is that multiple projects can continue to experience
shortfalls—which increases the risk that investments will experience
serious lifecycle cost and schedule variances—without having to develop


18
  Department of Homeland Security, Acquisition Management Directive, Directive
Number 102-01 (Jan. 20, 2010) and Department of Homeland Security, Capital Planning
and Investment Control Guide, version 7.2 (October 2011).




Page 22                                          GAO-12-904 DHS Major IT Investments
                      and document corrective actions and thus alert top management about
                      potential problems and associated risks. This is inconsistent with the
                      direction of OMB, which requires agencies to report (via the IT
                      Dashboard) on the cost and schedule performance of their projects and
                      considers those projects with a 10 percent or greater variance to be at an
                      increased level of risk of not being able to deliver promised capabilities on
                      time and within budget, and thus they require special attention from
                      management. It is also inconsistent with our best practices research and
                      experience at federal agencies, which stresses that agencies report to
                      management when projects are not meeting expectations or when
                      problems arise and quickly develop and document corrective efforts to
                      address the problems. Further, our research and work at agencies has
                      shown that waiting to act until significant life cycle variances occur can
                      sometimes be risky and costly, as life cycle schedules are typically for
                      multiyear periods, allowing the potential for underperforming projects to
                      continue to vary from their cost and schedule goals for an extended
                      amount of time without any requirement for corrective efforts.
                      Consequently, until these guidance shortcomings are addressed and
                      each underperforming project has defined and documented corrective
                      actions, the department’s major investments these projects support will
                      be at an increased risk of cost and schedule shortfalls.


                      Most of the projects comprising DHS’s 68 major IT investments are
Conclusions           meeting their cost and schedule commitments, but 21 major
                      investments—integral to DHS’s mission and costing approximately $1
                      billion—have projects that are experiencing significant cost and schedule
                      shortfalls. These shortfalls place these investments at increased risk of
                      not delivering promised capabilities on time and within budget, which, in
                      turn, pose a risk to DHS’s ability to fully meet its mission of securing the
                      homeland. DHS guidance does not require projects experiencing
                      significant cost and schedule shortfalls to develop and document
                      corrective efforts until they cause a life cycle cost and schedule variance.
                      This increases risk and is contrary to effective IT investment practices.
                      Given that DHS is currently establishing and implementing new IT
                      governance processes, the department is positioned to address the
                      guidance shortfalls.


                      We recommend that the Secretary of Homeland Security direct the
Recommendations for   appropriate officials to:
Executive Action


                      Page 23                                      GAO-12-904 DHS Major IT Investments
                     •   Establish guidance that provides for developing corrective efforts for
                         major IT investment projects that are experiencing cost and schedule
                         shortfalls of 10 percent or greater, similar to those identified in this
                         report.

                     •   Ensure that major IT investment projects with shortfalls of 10 percent
                         or greater have defined and documented corrective efforts.

                     In its written comments signed by the Director for the Departmental GAO-
Agency Comments      OIG Liaison Office and reprinted in appendix IV, DHS concurred with our
and Our Evaluation   recommendations and estimated that it would implement the first
                     recommendation by September 30, 2013, and the second one
                     immediately. It also commented that the department was pleased that the
                     report positively acknowledged that DHS (1) is meeting cost and schedule
                     commitments for most of its major IT investments and (2) has plans and
                     efforts under way to improve cost estimating capabilities and implement a
                     center of excellence for requirements engineering. The department also
                     provided technical comments, which we have incorporated where
                     appropriate.

                     We are sending copies of this report to interested congressional
                     committees and the Secretary of Homeland Security. In addition, the
                     report is available at no charge on the GAO website at
                     http://www.gao.gov.

                     If you or your staffs have any questions on the matters discussed in this
                     report, please contact me at (202) 512-9286 or pownerd@gao.gov.
                     Contact points for our Offices of Congressional Relations and Public
                     Affairs may be found on the last page of this report. GAO staff who made
                     major contributions to this report are listed in appendix V.




                     David A. Powner
                     Director, Information Technology
                     Management Issues




                     Page 24                                      GAO-12-904 DHS Major IT Investments
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              The objectives of our review were to determine the (1) extent to which
              Department of Homeland Security (DHS) IT investments are meeting their
              cost and schedule commitments, (2) primary causes of any commitment
              shortfalls, and (3) adequacy of DHS’s efforts to address these shortfalls
              and causes.

              To address our first objective, we analyzed how each of DHS’s 68 major
              investments under development was performing against its cost and
              schedules commitments, as reported by the agency to the Office of
              Management and Budget (OMB) for inclusion on OMB’s federal IT
              Dashboard. More specifically, we analyzed the extent to which each of
              these investments had met or exceeded, as of March 2012, cost and
              schedule commitments established when the investment was initiated. In
              doing this, we identified investments that had a project exceeding 10
              percent of its cost and schedule commitments. We focused on these
              investments and their subsidiary projects because OMB considers them
              to be at an increased level of risk of not being able to deliver promised
              capabilities on time and within budget, and thus requiring special attention
              from management.

              To assess the reliability of the IT Dashboard data we analyzed, we
              corroborated the data by interviewing investment and other DHS officials
              to determine whether the information on the dashboard was consistent
              with that reported by DHS. In addition, we followed up on the status of
              implementation of previous GAO recommendations to improve the quality
              of information on OMB’s federal IT Dashboard. 1 Specifically, we analyzed
              plans and related documentation describing efforts by DHS to increase
              the scrutiny and quality of data submitted to the IT Dashboard. As part of
              this, we also interviewed department officials including those from the
              Office of the Chief Information Officer who are responsible for reviewing
              and submitting DHS’s investment cost and schedule data to the federal IT
              Dashboard. The documentation and interviews provided us a level of
              assurance that the data we used for this engagement were, in fact,
              reliable.

              For our second objective, we used a structured interview instrument to
              survey the DHS and component officials responsible for the investments


              1
                GAO, Information Technology: OMB Has Made Improvements to Its Dashboard, but
              Further Work Is Needed by Agencies and OMB to Ensure Data Accuracy, GAO-11-262
              (Washington, D.C.; Mar. 15, 2011).




              Page 25                                        GAO-12-904 DHS Major IT Investments
Appendix I: Objectives, Scope, and
Methodology




experiencing cost and schedule shortfalls in order to identify the causes
of the shortfalls. As part of surveying these officials, we analyzed project
and related documentation to corroborate the causes reported to us via
the survey. We then analyzed these causes for commonalities, grouped
them accordingly, and tallied the frequency of each cause by investment.
In addition, we compared the causes to our prior reports on major DHS
investments and related IT management processes to identify the extent
to which we had made recommendations to address the causes
associated with the department’s investment cost and schedule shortfalls.

To address our third objective, we initially identified and reviewed relevant
criteria on developing and documenting corrective actions to address
investment shortfalls. Specifically, these criteria included

    •     DHS’s Acquisition Directive 102 (AD-102), 2
    •     DHS’s Capital Planning and Investment Control Guide, 3 and
    •     GAO’s Information Technology Investment Management guide. 4

We then used a structured interview instrument to survey DHS and
component officials responsible for those investments experiencing
shortfalls; we used the survey to identify whether any corrective actions
had been developed and documented to address investment shortfalls.
We also reviewed investment planning and execution documentation
(e.g., project plans, project status reports, program meeting minutes, and
acquisition program baselines) to corroborate information provided by the
officials during the survey process. We then compared these corrective
efforts to the criteria to identify any gaps and in those cases where there
were, we reviewed documentation and interviewed agency officials to
assess the reason for the gaps and any negative impacts.

We conducted this performance audit from October 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to


2
 Department of Homeland Security, Acquisition Management Directive, Directive Number
102-01 (Jan. 20, 2010).
3
 Department of Homeland Security, Capital Planning and Investment Control Guide,
version 7.2 (October 2011).
4
  GAO, Information Technology Investment Management: A Framework for Assessing and
Improving Process Maturity, GAO-04-394G (Washington, D.C.: Mar. 1, 2004).




Page 26                                           GAO-12-904 DHS Major IT Investments
Appendix I: Objectives, Scope, and
Methodology




obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives.




Page 27                                     GAO-12-904 DHS Major IT Investments
Appendix II: List of DHS Major IT
                                            Appendix II: List of DHS Major IT Investments
                                            that Are Meeting Cost and Schedule
                                            Commitments


Investments that Are Meeting Cost and
Schedule Commitments
                                            Table 5 lists the DHS major IT investments that were meeting their cost
                                            and schedule commitments.

Table 5: DHS Major IT Investments and Subsidiary Projects and Planned Project Costs (in millions)

                                                                                                                             Planned
                                                                                                                              project
                                                                                                                                   a,b
Component                  Investment Title                              Project Name                                        costs
Analysis and
Operations

                           Common Operational Picture                    DHS Common Operational Picture
                                                                         Infrastructure deployment at DHS Data
                                                                         Center 2
                                                                                                                                $0.50
                                                                         DHS Common Operational Picture
                                                                         enhanced functionality capabilities                     0.10
                           Homeland Security Information Network         Homeland Security Information Network 3.0
                                                                         DME                                                    22.34
                                                                         HS SLIC Consolidation                                   1.60
                                                                         Two Factor Authentication in Legacy                     0.31
CBP
                           Advance Passenger Information System          Advance Passenger Information System                    3.70
                           Analytical Framework for Intelligence         Analytical Framework for Intelligence
                                                                         System                                                 29.35
                                                                         Homeland Secure Data Network                            4.29
                           Automated Targeting System                    Passenger Iteration                                    15.79
                           Maintenance                                   Cargo Iteration                                        17.76
                           Block 1                                       Operations Task Order                                   5.00
                                                                         Maintenance Task Order                                  4.40
                           Infrastructure                                Windows 7                                               6.25
                           Integrated Fixed Towers                       Acquisition Planning                                    2.28
                                                                         Construct and Deployment of Integrated
                                                                         Fixed Towers System to Nogales Areas of
                                                                         Responsibility
                                                                         Construct and Deployment of Integrated
                                                                         Fixed Towers System to Sonoita Areas of
                                                                         Responsibility
                                                                         Construct and Deployment of Integrated
                                                                         Fixed Towers System to Casa Grande Areas
                                                                         of Responsibility
                           Mobile Surveillance Capabilities              First Article Test of vehicles                          1.45
                                                                         Deployment to Arizona




                                            Page 28                                                GAO-12-904 DHS Major IT Investments
                                          Appendix II: List of DHS Major IT Investments
                                          that Are Meeting Cost and Schedule
                                          Commitments




                                                                                                                          Planned
                                                                                                                           project
                                                                                                                                a,b
Component                   Investment Title                           Project Name                                       costs
                                                                       Vehicle Maintenance
                            Remote Video Surveillance Systems          Remote Video Surveillance Systems                     26.76
                            SAP                                        SAP Budget Solution Upgrade                           13.38
                                                                       SAP Support Patches                                    1.35
                            Tactical Communications Modernization      Houlton                                               60.78
                                                                       El Paso                                                7.74
                                                                       Rio Grande Valley                                     33.85
                                                                       Digital in Place                                      19.50
DHS - Office of the Chief
Information Officer
                            Homeland Secure Data Network               State and Local Fusion Center Build Out or
                                                                       Relocation                                             3.22
                                                                       Homeland Secure Data Network
                                                                       Enhancements                                           1.80
                            Infrastructure Transformation Program      Data Center                                          161.90
                                                                       Network                                               21.61
                                                                       Email                                                  2.14
                                                                       Wireless                                               3.20
FEMA
                            Disaster Management E-Government           Disaster Management PMO                                3.80
                            Initiative                                 FEMA.gov migration to Drupal Content
                                                                       Management System                                      4.70
                                                                       Text Messaging and Mobile Platform                     1.50
                                                                       RightNow CRM Integration                               1.50
                                                                       Web Usability and Disaster Survivor Studies            1.00
                                                                       DHS Enterprise Hosting and Environment                 2.69
                            Integrated Public Alert & Warning System   Emergency Alert System Primary Entry Point            30.17
                                                                       Expansion
                                                                       Emergency Alert System Primary Entry Point             7.84
                                                                       Site Modernization
                                                                       Integrated Public Alert & Warning System -             1.40
                                                                       OPEN Sustainment and Enhancement
                                                                       Emergency Alert System Primary Entry Point             3.70
                                                                       Operation and Maintenance
                            Logistics Supply Chain Management          Acquire Initial Operational Capability                 2.16
                                                                       Commercial Off The Shelf Upgrade                      29.20
                                                                       SOA GSS Integration                                    0.63
                                                                       Data Center Migration                                  3.36




                                          Page 29                                               GAO-12-904 DHS Major IT Investments
                            Appendix II: List of DHS Major IT Investments
                            that Are Meeting Cost and Schedule
                            Commitments




                                                                                                           Planned
                                                                                                            project
                                                                                                                 a,b
Component   Investment Title                             Project Name                                      costs
             National Flood Insurance Program            SAR Remediation
            Information Technology Systems &
            Services                                                                                           2.09
ICE
            Criminal Alien Identification Initiatives    Alien Criminal Response Information
                                                         Management System Modernization                       7.31
                                                         Status Determination Support                          5.58
                                                         Automated Threat Prioritization                      11.79
            DRO Electronic Health Record System          eHR Acquisition & Validation Study                    1.42
                                                         eHR Acquisition & Deployment
            Enforcement Information Sharing              Data Pattern Information Collection System-
                                                         2                                                     0.88
                                                         Law Enforcement Information Sharing
                                                         Service                                               0.34
            IT Infrastructure (Atlas)                    Data Center Services                                 10.50
                                                         Desktop Services                                      5.40
                                                         Email Services                                        1.56
                                                         Network Services                                      2.29
                                                         Single Sign-On Services                               1.76
                                                         Video Services                                        4.50
            Student and Exchange Visitor Information Student and Exchange Visitor Information
            System                                   System                                                    7.96
                                                         Student and Exchange Visitor Information
                                                         System II                                             4.70
            TECS Modernization                           Core Case Management Release 1.0                     21.55
NPPD
            FPS Risk Assessment and Management           Risk Assessment and Management Program
            Program                                      Maintenance Support                                   3.43
                                                         FPS Modified Infrastructure Survey Tool
                                                         Development                                           2.50
                                                         FPS Gateway Development and Deployment                3.35
                                                         Follow-on Assessment Tool Development
                                                         and Deployment
TSA
            Crew Vetting                                 IBM Oasis WO#46                                       3.90
            FAMSNet (Common Operating                    Security Updates                                      0.09
            Environment)
                                                         Software Patches                                      0.09
            Information Technology Infrastructure        TSA Enterprise Integrated Test Environment            6.09
            Program                                      TSA Enterprise Desktop Refresh                       50.04




                            Page 30                                              GAO-12-904 DHS Major IT Investments
                            Appendix II: List of DHS Major IT Investments
                            that Are Meeting Cost and Schedule
                            Commitments




                                                                                                             Planned
                                                                                                              project
                                                                                                                   a,b
Component   Investment Title                             Project Name                                        costs
                                                         TSA Enterprise Secure Mobile Computing                  3.97
                                                         TSA Enterprise IT Network Switches                     12.00
            Performance Management Information           Create Development Environment                          0.87
            System                                       Commercial Off The Shelf Maintenance                    0.65
                                                         Performance Management Information                      1.33
                                                         System and PIMS Server/Application
                                                         Migration
                                                         Performance Management Information
                                                         System, AIM, and PIMS Software Releases
                                                         Annual Customer Satisfaction Survey
            Secure Flight                                Functional Releases                                     6.77
                                                         Secure Flight User Interface Releases                   3.59
                                                         Report Management System Releases                       3.77
                                                         Maintenance Releases BY 11 Q4                           0.20
                                                         Maintenance Releases BY 12                              2.45
            Technology Infrastructure Modernization      Service Oriented Architecture Foundation               16.00
            Program                                      Service Oriented Architecture Platform
            Transportation Worker Identification         4th Quarter Transportation Worker                       1.80
            Credential                                   Identification Credential Maintenance
                                                         Pre-Transition and Formal Transition work               2.00
                                                         2012 Transportation Worker Identification              16.60
                                                         Credential Maintenance
                                                         Transportation Worker Identification                    1.10
                                                         Credential Reader Certification
            TSA Operating Platform                       Enterprise Applications                                 0.23
                                                         Applications Development                                2.21
                                                         Cloud Computing                                         0.06
USCG
            Asset Logistics Management Information       Asset Logistics Management Information                  8.15
            System                                       System Project
            C4ISR                                        Segment 2                                              56.48
                                                         Prototype Segment 2                                    22.70
                                                         Video and Mission Processor Redesign
                                                         FRC Information Assurance/Authority To
                                                         Operate                                                 1.50
                                                         National Security Cutters C4ISR Certification
                                                         and Accreditation                                       5.00
                                                         National Security Cutters 3-4 Operational
                                                         Testing Support                                         2.50




                            Page 31                                                GAO-12-904 DHS Major IT Investments
                            Appendix II: List of DHS Major IT Investments
                            that Are Meeting Cost and Schedule
                            Commitments




                                                                                                           Planned
                                                                                                            project
                                                                                                                 a,b
Component   Investment Title                             Project Name                                      costs
            Coast Guard-LIMS                             Segment 1                                             6.88
            Direct Access                                Direct Access Technical Refresh                       7.51
                                                         Direct Access Maintenance                             3.09
                                                         CG-1 PMO Support                                      0.77
            Interagency Operations Centers               Segment 1                                             2.04
                                                         Segment 2                                             3.53
            Nationwide Automatic Identification          Increment 2                                          11.77
            System                                       Maintenance Support                                   6.49
            Ports and Waterways Safety System - IT       AIS Recapitalization                                  1.34
                                                         INVS Recapitalization                                 0.32
            Rescue 21                                    Project Management                                   25.49
                                                         Engineering                                          30.40
                                                         System Support and Maintenance                       37.36
                                                         Rescue 21 Deployment                                257.12
                                                         VSAT AC&I                                             0.63
                                                         R21 Western Rivers                                    5.64
                                                         R21 Alaska                                            9.50
                                                         Vessels
USCIS
            Benefits Provision - Verification         Verification Information System                          2.58
            Information System/Employment Eligibility Verification Information System
            Verification
            Immigration - CLAIMS 3.0                     CLAIMS 3 - C&A                                        0.67
                                                         CLAIMS MF DC Migration Deploy                         0.10
                                                         CLAIMS 3 MF R.3.14                                    0.35
                                                         CLAIMS 3 MFAS R1.6                                    0.30
                                                         CLAIMS 3 FD258                                        0.30
                                                         CLAIMS 3 ICMS R1.18                                   0.27
                                                         CLAIMS 3 CRISI R 4.9                                  0.45
                                                         CLAIMS 3 FBASI R1.7                                   0.45
                                                         CLAIMS 3 BRU
                                                         CLAIMS 3 ICMS R1.19
                                                         CLAIMS 3 MF R3.15
                                                         CLAIMS 3 MFAS R1.7
                                                         CLAIMS 3 Quarterly Table Updates                      1.34
                                                         CLAIMS 3 DBA Support                                  1.44




                            Page 32                                              GAO-12-904 DHS Major IT Investments
                            Appendix II: List of DHS Major IT Investments
                            that Are Meeting Cost and Schedule
                            Commitments




                                                                                                                      Planned
                                                                                                                       project
                                                                                                                            a,b
Component   Investment Title                                        Project Name                                      costs
                                                                    LAN R6.21 Testing & Deploy                            0.03
                                                                    LAN R6.22-Development                                 0.16
                                                                    LAN R6.23                                             0.35
                                                                    LAN R6.24                                             0.35
                                                                    LAN R6.25
                                                                    ICMS R1.17 - Testing                                  0.03
                                                                    FBASEI R1.6-Development                               0.16
            Infrastructure (Enterprise)                             IT Physical Plant/HVAC/Electrical                     4.58
                                                                    IT Data and Voice Systems (hardware
                                                                    Upgrades)                                             7.02
                                                                    Video Upgrade Project                                 0.57
                                                                    Windows 7 and Office 2010                             0.01
                                                                    Video End-of-Life                                     0.03
                                                                    Voice End-of-Life                                     0.03
                                                                    Application & Network Monitoring                      0.01
                                                                    Telework Expansion (Citrix Enterprise
                                                                    Telework Project)                                     0.24
                                                                    Data Server Suites                                    0.50
                                                                    Data & Voice circuits                                 7.05
            Integrated Document Production                          Redesigned EAD card                                   0.06
                                                                    Card Personalization System Technology
                                                                    Refreshment Release 1.1                               0.42
                                                                    Optical Card Consumables Contract                     7.20
                                                                    Non-Optical Card Consumables Contract                 2.30
                                                                    Card Personalization System Technology
                                                                    Refreshment                                           2.40
                                                                    Card Operations Services                              2.50
            Transformation                                          Core Account and Case Management                    756.28
USSS
            IT Infrastructure                                       End User Systems & Support Maintenance
                                                                    Activities                                            3.15
                                                                    Mainframe and Servers Services & Support
                                                                    Maintenance Activities                                0.12
                                                                    Telecommunications Systems & Support
                                                                    Maintenance Activities                                6.57
                                                                    Datacenter Migration                                 18.00
                                                                                                                              c
Totals      47                                                                                                      $2,135.29
                            Source: GAO analysis of OMB’s federal IT dashboard data.




                            Page 33                                                         GAO-12-904 DHS Major IT Investments
Appendix II: List of DHS Major IT Investments
that Are Meeting Cost and Schedule
Commitments




a
    This is the sum of the investment’s project costs as of March 8, 2012.
b
  Some projects do not have project costs, because DHS has not yet provided the costs to OMB, and
the projects are not yet in development.
c
    Differences in the total are due to rounding.




Page 34                                                       GAO-12-904 DHS Major IT Investments
Appendix III: List of DHS Major IT
                                             Appendix III: List of DHS Major IT Investments
                                             and Subsidiary Projects with Shortfalls



Investments and Subsidiary Projects with
Shortfalls
                                             Table 6 lists the DHS major IT investments with cost and/or schedule
                                             shortfalls, their costs and subsidiary projects, whether or not they had a
                                             cost and/or schedule shortfall, and their planned costs.

Table 6: List of DHS Major IT Investments with Cost and/or Schedule Shortfalls and Subsidiary Projects (cost in millions)

                                               Total
                                               planned
                                               project                                         Cost      Schedule Planned
                                                     a                                                                          b
Component Investment title                     costs         Subsidiary projects               shortfall shortfall project costs
CBP
             1.   Automated Commercial      $124.26          M1: e-Manifest Rail and Sea
                  Environment/International
                  Trade Data System                                                                                       $10.22
                                                             Cargo Control and Release
                                                             Planning/ADE 2a/2b Approval
                                                             Phase                                                                  3
                                                             Task Order PROD                                                 22.85
                                                             Task Order ECOM                                                 28.14
                                                             Automated Commercial
                                                             Environment Infrastructure
                                                             Support                                                         60.05
             2.   Land Border Integration      20.89         Land Border Integration Phase 1                                 20.9
             3.   Non-intrusive Inspection     332.3         Large Scale FY08
                  Systems Program                                                                                            17.44
                                                             Large Scale ARRA                                                61.93
                                                             Small Scale ARRA                                                 0.78
                                                             Large-scale FY 2010                                            31.41
                                                             Non-Intrusive Inspection
                                                             Maintenance FY11                                                58.18
                                                             Non-Intrusive Inspection
                                                             Acquisition FY11                                                46.45
                                                             Non-Intrusive Inspection
                                                             Maintenance FY12                                              116.11
             4.   Northern Border, Remote      8.2           Northern Border Project Remote
                  Video Surveillance                         Video Surveillance System
                  System
                                                                                                                              3.1
                                                             Northern Border Project Remote
                                                             Video Surveillance System                                         5.1
             5.   TECS Modernization           43.05         Secondary Inspection                                             8.33
                                                             High Performance Primary Query
                                                             and Manifest Processing                                          8.65
                                                             Travel Documents and Encounter
                                                             Data                                                            11.16




                                             Page 35                                            GAO-12-904 DHS Major IT Investments
                                                Appendix III: List of DHS Major IT Investments
                                                and Subsidiary Projects with Shortfalls




                                                  Total
                                                  planned
                                                  project                                            Cost      Schedule Planned
                                                        a                                                                             b
Component Investment title                        costs         Subsidiary projects                  shortfall shortfall project costs
                                                                Lookout Record Data Services                                        9.98
                                                                Secondary Inspection
                                                                Maintenance                                                        4.93
DHS - Office of the Chief Information Officer
              6.   Human Resources IT             8.52          EmpowHR Deployment                                                   0.4
                                                                Reporting Tool Development                                          1.79
                                                                PAS                                                                  0.9
                                                                HQ Survey Tool                                                      0.03
                                                                NFC VPN                                                             0.18
                                                                Balanced Workforce Survey Tool                                      1.19
                                                                USAJOBS 3.0                                                         2.57
                                                                Human Capital Segment
                                                                Architecture                                                       1.46
FEMA
              7.   Disaster Assistance            50.5          Surge IOC
                   Improvement Plan                                                                                                1.86
                                                                ForSee Satisfaction Survey
                                                                Integration                                                         0.04
                                                                SMS Pilot                                                           0.15
                                                                HUD SOA Interface                                                 0.78
                                                                Sustainment                                                         3.05
                                                                Site Usability Improvements                                        5.78
                                                                Development Support                                                 6.68
                                                                Louisiana DSNAP                                                     1.09
                                                                Address Locator Service                                             0.41
                                                                DAIP Mobile Applicant Inquiry                                      0.25
                                                                WFA: Income and Insurance
                                                                Verification                                                       0.59
                                                                Other Needs Assistance                                             0.36
                                                                Usability and Portal Consolidation                                  2.71
                                                                Centralized Application                                            0.59
                                                                Common State Application                                           0.59
                                                                SMS Phase II                                                       0.19
                                                                Interface Enhancements FY 2012                                     1.36
                                                                Coordinated Assistance Network
                                                                Interface Enhancements FY 2013




                                                Page 36                                               GAO-12-904 DHS Major IT Investments
                                          Appendix III: List of DHS Major IT Investments
                                          and Subsidiary Projects with Shortfalls




                                            Total
                                            planned
                                            project                                           Cost      Schedule Planned
                                                  a                                                                            b
Component Investment title                  costs         Subsidiary projects                 shortfall shortfall project costs
                                                          Stakeholder Outreach                                               3.34
                                                          Program Support                                                   20.69
ICE
           8.   Detention and Removal    8.62             Risk Classification Assessment
                Operations Modernization                                                                                    2.62
                                                          EARM 4.0                                                                 4
                                                          EARM 3.0 Release 2                                                       2
NPPD
           9.   Critical Infrastructure     20.55         Protected Critical Infrastructure
                Technology and                            Information Management System:
                Architecture                              Development, Modification, and
                                                          Enhancement                                                        0.58
                                                          Protected Critical Infrastructure
                                                          Information Management System:
                                                          Maintenance                                                         1.5
                                                          Infrastructure Protection
                                                          Gateway: Development,
                                                          Modification, & Enhancement                                         1.5
                                                          Infrastructure Protection
                                                          Gateway: Maintenance                                               3.26
                                                          Infrastructure Information
                                                          Collection System: Development,
                                                          Modification, & Enhancement                                        2.46
                                                          Infrastructure Information
                                                          Collection System: Maintenance                                     4.56
                                                          Infrastructure Protection Share:
                                                          Maintenance                                                        0.84
                                                          Automated Critical Asset
                                                          Management System:
                                                          Maintenance                                                        1.75
                                                          Technical Resource for Incident
                                                          Prevention Wire: Main                                             1.86
                                                          Web Emergency Operations
                                                          Center: Maintenance                                                1.38
           10. Infrastructure Security  72.76             Ammonium Nitrate Analysis of IT
               Compliance-Chemical                        Requirements FY11 Q4 Close
               Security Assessment Tool                   Out                                                                0.29
                                                          Case Management FY11 Q4
                                                          close out                                                          1.62
                                                          Chemical Security Assessment
                                                          Tool Enhancements FY 2011; Q4
                                                          Close Out                                                         3.65




                                          Page 37                                              GAO-12-904 DHS Major IT Investments
                                          Appendix III: List of DHS Major IT Investments
                                          and Subsidiary Projects with Shortfalls




                                            Total
                                            planned
                                            project                                           Cost      Schedule Planned
                                                  a                                                                            b
Component Investment title                  costs         Subsidiary projects                 shortfall shortfall project costs
                                                          Production System Update FY11
                                                          Q4 close out EA                                                    1.19
                                                          Ammonium Nitrate DME                                               5.34
                                                          Case Management DME                                                6.07
                                                          Chemical Security Assessment
                                                          Tool DME                                                          10.14
                                                          Develop staging environment for
                                                          Development & Testing                                              3.81
                                                          Production Environment
                                                          Migration to DHS DC
                                                          Maintenance Support FY11 Q4
                                                          Close Out                                                          7.31
                                                          Maintenance Support                                               33.35
           11. National Cybersecurity       262.6         National Cybersecurity Protection
               Protection System                          System Block 2.0                                                  0.77
                                                          National Cybersecurity Protection
                                                          System Block 2.1                                                  2.58
                                                          National Cybersecurity Protection
                                                          System Block 2.2
                                                          National Cybersecurity Protection
                                                          System Block 3.0                                                  96.45
                                                          National Cybersecurity Protection
                                                          System Maintenance                                                162.8
           12. Next Generation Networks 63.06             Core SP1
               Priority Services                                                                                            6.36
                                                          Core SP2                                                          20.19
                                                          Core SP3                                                          25.89
                                                          Common                                                            10.62
           13. US-VISIT: Arrival and        7.18          Arrival and Departure Information
               Departure Information                      System FY 2011 Maintenance
               System                                     Release Project                                                   1.98
                                                          Arrival and Departure Information
                                                          System FY 2012 Maintenance
                                                          Release Project                                                    5.2
                                                          ADIS FY13 Maintenance Release
                                                          Project
           14. US-VISIT: Automated          33.24         Automated Biometric
               Biometric Identification                   Identification System FY 2011
               System                                                                                                      14.56
                                                          IDENT FY 2012                                                     18.68
                                                          IDENT FY 2013



                                          Page 38                                              GAO-12-904 DHS Major IT Investments
                                     Appendix III: List of DHS Major IT Investments
                                     and Subsidiary Projects with Shortfalls




                                       Total
                                       planned
                                       project                                         Cost      Schedule Planned
                                             a                                                                          b
Component Investment title             costs         Subsidiary projects               shortfall shortfall project costs
TSA
           15. Air Cargo Security      4.09          Known Shipper Management
                                                     System 1.3                                                      0.55
                                                     Known Shipper Management
                                                     System 1.4                                                       0.74
                                                     Known Shipper Management
                                                     System 1.5
                                                     Indirect Air Carrier Management
                                                     System 2.5                                                       0.44
                                                     Indirect Air Carrier Management
                                                     System 2.6                                                       0.55
                                                     Freight Assessment System 3.1                                   0.33
                                                     Freight Assessment System 4.0                                     0.6
                                                     Freight Assessment System 4.1
                                                     Freight Assessment System 3.9                                  0.19
                                                     Indirect Air Carrier Management
                                                     System 2.7                                                       0.68
           16. Federal Air Marshal     5.43          Scheduling Software
               Service Mission                       Modernization
               Scheduling and
               Notification System                                                                                   0.02
                                                     DC2 Migration                                                   0.76
                                                     Hardware Modernization                                           1.45
                                                     IRIS Production Integration
                                                     ARES Production Integration
                                                     Portal Modernization                                             1.2
                                                     FAMIS Modernization                                                    2
           17. Hazmat Threat           4.09          Hazardous Materials
               Assessment Program                    Endorsement SG Maintenance                                       0.74
                                                     Hazardous Materials
                                                     Endorsement Comparability                                        1.4
                                                     Hazardous Materials
                                                     Endorsement SG Maintenance                                       1.95
           18. Security Technology     27.99         TSE Data Management FY2011
               Integrated Program                                                                                     3.69
                                                     TSE Data Management FY 2012                                      5.54
                                                     Remote Monitoring and
                                                     Maintenance/Maintenance Ticket
                                                     Application FY 2011                                             0.35




                                     Page 39                                            GAO-12-904 DHS Major IT Investments
                                          Appendix III: List of DHS Major IT Investments
                                          and Subsidiary Projects with Shortfalls




                                            Total
                                            planned
                                            project                                         Cost      Schedule Planned
                                                  a                                                                          b
Component Investment title                  costs         Subsidiary projects               shortfall shortfall project costs
                                                          Remote Monitoring and
                                                          Maintenance /Maintenance Ticket
                                                          Application FY2012                                               4.51
                                                          Security Equipment
                                                          Modernization FY 2011                                           3.27
                                                          Security Equipment
                                                          Modernization FY 2012                                          10.64
USCG
           19. Coast Guard Business         0.86          CGBI 3.0
               Intelligence                                                                                                0.16
                                                          Cognos 10 Upgrade                                                0.25
                                                          Predictive Analytics Capability                                 0.25
                                                          Data Quality Capability                                           0.2
USCIS
           20. Naturalization: CLAIMS 4     2.36          CLAIMS 4 Release 8.5                                              0.9
                                                          CLAIMS 4 Release 8.6                                              0.9
                                                          CLAIMS 4 Release 8.7
                                                          CLAIMS 4 Security                                                 0.1
                                                          Release 8.5.0.1                                                   0.2
                                                          Lean Agile Release 8.5.1.0                                       0.08
                                                          Lean Agile Release 8.5.2.0                                      0.08
                                                          Lean Agile Release 8.5.3.0                                       0.08
                                                          Release 8.5.2.1                                                  0.02
                                                          Release 8.5.3.1                                                  0.02
USSS
           21. Information Integration      43.61         IT Modernization
               and Technology
               Transformation                                                                                             12.68
                                                          Cyber Security                                                         5
                                                          Database Architecture &
                                                          Maintenance                                                     6.303
                                                          Information Assurance                                           3.327
                                                          COLD2                                                             10
                                                          Cross Domain/Multi-Level
                                                          Security                                                         4.5
                                                          Communications Capabilities                                       1.8

                                                                                                                                 c
Total      21                               $1,144.14                                              6        34       $1,143.35




                                          Page 40                                            GAO-12-904 DHS Major IT Investments
Appendix III: List of DHS Major IT Investments
and Subsidiary Projects with Shortfalls




Source: GAO analysis of OMB’s federal IT Dashboard data.
a
    This is the sum of the investment’s project costs as of March 8, 2012.
b
  Some projects do not have project costs, because DHS has not yet provided the costs to OMB, and
the projects are not yet in development.
c
    Differences in the total are due to rounding.




Page 41                                                       GAO-12-904 DHS Major IT Investments
Appendix IV: Comments from the
             Appendix IV: Comments from the Department
             of Homeland Security



Department of Homeland Security




             Page 42                                     GAO-12-904 DHS Major IT Investments
Appendix IV: Comments from the Department
of Homeland Security




Page 43                                     GAO-12-904 DHS Major IT Investments
Appendix V: GAO Contact and Staff
                  Appendix V: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  David A. Powner at (202) 512-9286 or pownerd@gao.gov
GAO Contact
                  In addition to the contact named above, the following staff also made key
Staff             contributions to this report: Gary Mountjoy (assistant director), Scott
Acknowledgments   Borre, Camille Chaires, and Nancy Glover.




(311259)
                  Page 44                                    GAO-12-904 DHS Major IT Investments
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.