oversight

Customs Service Modernization: Automated Commercial Environment Progressing, but Further Acquisition Management Improvements Needed

Published by the Government Accountability Office on 2003-02-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to Congressional Committees




February 2003
                CUSTOMS SERVICE
                MODERNIZATION
                Automated
                Commercial
                Environment
                Progressing, but
                Further Acquisition
                Management
                Improvements Needed




GAO-03-406
                a
                                               February 2003

                                               CUSTOMS SERVICE MODERNIZATION

                                               Automated Commercial Environment
Highlights of GAO-03-406, a report to          Progressing, but Further Acquisition
congressional committees
                                               Management Improvements Needed



The U.S. Customs Service is                    Customs’ November 2002 ACE expenditure plan, the fourth in a series of
conducting a multiyear, multibillion           legislatively required plans, provides for certain project management tasks
dollar project, the Automated                  as well as the definition, design, and development of the first release of the
Commercial Environment (ACE), a                second of four planned ACE increments. GAO’s analysis of the plan shows
new trade processing system that is            that it meets the legislative conditions imposed by the Congress.
planned to support effective and
efficient movement of goods into
the United States. By congressional            In its series of reports on Customs’ management of ACE, GAO has made a
mandate, Customs’ expenditure                  number of recommendations, which Customs is currently addressing.
plans for ACE must meet certain                However, Customs has been slow to correct weaknesses in two areas
conditions, including being                    fundamental to effective acquisition management—people and processes.
reviewed by GAO. This study                    These weaknesses increase the risk that ACE will be late, cost more than
addresses whether Customs’ latest              necessary, and not perform as intended.
plan satisfies these conditions and
provides observations about the                •   While the Customs Modernization Office (CMO) has developed a human
plan and Customs’ efforts to                       capital strategy, neither the strategy nor supporting documentation
implement GAO’s open                               identifies how the commitments made in the strategy will be met,
recommendations for improving
ACE management.
                                                   including what steps will be taken and what resources are needed to
                                                   execute the steps. Further, based on the strategy’s timeline, it will be
                                                   over a year before the CMO fully implements the strategy.
                                               •   Customs’ has made slow progress in implementing key acquisition
To ensure that Customs is                          practices, such as project office management and acquisition risk
positioned to effectively manage                   management, which GAO first recommended Customs do in 1999. Since
the acquisition of ACE, GAO is                     that time many practices have been developed, but almost none of them
making recommendations to the                      have been implemented, as shown in the figure below.
commissioner aimed at improving
Customs’ acquisition management                Additionally, Customs’ ACE contractor is conducting system tests in
capabilities. Customs concurred                accordance with best practices, initial test results are positive, and
with GAO’s recommendations and
                                               indicators suggest the contractor is delivering a quality product. However,
described specific actions that it is
taking to respond to each.                     Customs is not employing independent verification and validation (IV&V) in
                                               overseeing ACE testing, which is one way to mitigate the acquisition
                                               weaknesses cited above. Without IV&V, Customs states that it is relying on
                                               the contractor’s reputation and maturity level as guarantors of system
                                               quality, which is not adequate for a complex and risky program like ACE.

                                               Status of Customs’ Implementation of Acquisition Management Key Practices




www.gao.gov/cgi-bin/getrpt?GAO-03-406.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Randolph C.
Hite at (202) 512-3439 or hiter@gao.gov.
Contents



Letter                                                                                                                 1
                            Recommendations for Executive Action                                                       4
                            Agency Comments and Our Evaluation                                                         5


Appendixes
             Appendix I:    Customs’ Fourth Automated Commercial Environment
                            (ACE) Expenditure Plan                                                                     7
             Appendix II:   Comments from the U.S. Customs Service                                                     51




                            Abbreviations

                            ACE          Automated Commercial Environment
                            CIO          Chief Information Officer
                            CMM          Capability Maturity Model
                            CMO          Customs Modernization Office
                            IV&V         independent verification and validation
                            OMB          Office of Management and Budget
                            SA-CMM       Software Acquisition Capability Maturity Model
                            SAT          system acceptance tests
                            SEI          Software Engineering Institute

                             This is a work of the U.S. Government and is not subject to copyright protection in the
                             United States. It may be reproduced and distributed in its entirety without further
                             permission from GAO. It may contain copyrighted graphics, images or other materials.
                             Permission from the copyright holder may be necessary should you wish to reproduce
                             copyrighted materials separately from GAO’s product.




                            Page i                                         GAO-03-406 Customs Service Modernization
A
United States General Accounting Office
Washington, D.C. 20548



                                    February 28, 2003                                                                               Leter




                                    The Honorable Ben Nighthorse Campbell
                                    Chairman
                                    The Honorable Byron L. Dorgan
                                    Ranking Minority Member
                                    Subcommittee on Treasury and General Government
                                    Committee on Appropriations
                                    United States Senate

                                    The Honorable Ernest J. Istook, Jr.
                                    Chairman
                                    The Honorable Steny H. Hoyer
                                    Ranking Minority Member
                                    Subcommittee on Transportation, Treasury, Postal Service,
                                      and General Government
                                    Committee on Appropriations
                                    House of Representatives

                                    In November 2002, the U.S. Customs Service submitted to Congress its
                                    fourth expenditure plan, seeking release of $314 million for its Automated
                                    Commercial Environment (ACE) project. ACE is to be Customs’ new trade
                                    processing system and the first project under the Customs Modernization
                                    Program. As required by Customs’ fiscal year 2002 appropriation and
                                    continuing resolutions,1 we reviewed the expenditure plan. Our objectives
                                    were (1) to determine whether the fourth ACE expenditure plan satisfies
                                    certain legislative conditions, (2) to determine whether the plan is
                                    consistent with our open ACE recommendations, and (3) to provide
                                    observations about the plan and Customs’ management of ACE.

                                    On December 27, 2002, we briefed your offices on the results of this review.
                                    This report transmits the results of our work. The full briefing, including
                                    our scope and methodology, is reprinted as appendix I.


                                    1
                                     For example, P. L. 108-2 appropriated such amounts as may be necessary under the
                                    authority and conditions provided in the applicable appropriations act for fiscal year 2002
                                    for continuing projects or activities that were conducted in fiscal year 2002, at a rate for
                                    operations not exceeding the current rate, and for which appropriations, funds, or other
                                    authority was made available in the Treasury and General Government Appropriations Act,
                                    2002, P. L. 107-67, 115 Stat. 514, 520 (2001). These appropriations shall be available to the
                                    extent and in the manner which would be provided by the fiscal year 2002 Treasury
                                    Appropriations Act.




                                    Page 1                                          GAO-03-406 Customs Service Modernization
Concerning our first objective, Customs’ expenditure plan satisfies the
legislative conditions specified in Customs’ appropriations act. That is, the
plan provides for (1) meeting the capital planning and investment control
review requirements of the Office of Management and Budget (OMB);
(2) complying with Customs’ enterprise architecture; 2 and (3) complying
with federal acquisition rules, requirements, guidelines, and systems
acquisition management practices. Further, the plan was reviewed and
approved by the Joint Capital Investment Review Board3 and OMB.

Concerning our second objective, Customs is making progress in
addressing our open recommendations, which are as follows:

• Justify and make investment decisions incrementally.

• Before building each ACE release, certify to Customs’ appropriations
  subcommittees that Customs’ enterprise architecture has been
  sufficiently extended and updated.

• Develop and implement a rigorous and analytically verifiable cost
  estimating program that embodies the tenets of effective estimating as
  defined in the Software Engineering Institute’s (SEI) institutional and
  project-specific estimating guidance.

• Immediately develop and implement a human capital management
  strategy for the Customs Modernization Office (CMO).

• Limit future expenditure plan requests for management reserve funds to
  10 percent of the total funds requested for the program, or adequately
  justify any management reserve requests in excess of 10 percent.




2
 An enterprise architecture is an institutional blueprint for guiding and constraining
investments in business process change and systems.
3
To expedite reviews of ACE expenditure plans, the Customs and Treasury Investment
Review Boards were consolidated to form the Joint Capital Investment Review Board.




Page 2                                           GAO-03-406 Customs Service Modernization
• Develop and implement process controls consistent with SEI’s Software
  Acquisition Capability Maturity Model (SA-CMM4), and by September 30,
  2002, assess and report to Customs’ appropriation subcommittees on
  the maturity of CMO’s software acquisition process.

• Address the risks associated with the accelerated ACE acquisition
  strategy and report on the strategy going forward and plans for
  mitigating the risks associated with this strategy.

While Customs has taken actions to address each of these
recommendations, its actions to develop and implement a human capital
strategy and acquisition management process controls were only partially
consistent with our recommendations. Specifically, while the CMO
developed a human capital strategy that specified high-level commitments
and satisfied some best practices, there are some notable omissions,
including how these commitments will be met. Further, based on the
timeline in the strategy, it will be over a year before key practices are
implemented.

Also, while Customs reported to its House and Senate appropriations
subcommittees on its development of SA-CMM processes, it did not report
on implementation. Further, while many of the acquisition processes have
been developed, Customs has not yet implemented most of them.

Finally, we made the following observations:

• System integration and acceptance testing performed by the ACE
  contractor is being conducted in accordance with best practices. The
  contractor has established a test organization and built an infrastructure
  for test activities. It has completed system integration and system
  acceptance tests (SAT) for ACE increment 1, release 1.1, and these test
  activities were consistent with test management best practices.

• The results of SAT performed by the ACE contractor are positive. All but
  seven ACE increment 1, release 1.1 requirements successfully passed


4
 Capability Maturity Model (CMM) is a service mark of Carnegie Mellon University, and
CMM is registered in the U.S. Patent and Trademark Office. The SA-CMM identifies key
process areas that are necessary to effectively manage software-intensive system
acquisitions. Level 2 is the second level of the SA-CMM’s five-level scale; achieving this level
means that an organization has the software acquisition rigor and discipline to repeat
project successes.




Page 3                                            GAO-03-406 Customs Service Modernization
                         their test cases for SAT. For those that did not pass, only noncritical
                         system defects resulted, and acceptable workarounds were identified.

                      • Data provided by the ACE contractor on all unresolved system defects
                        for ACE increment 1, release 1.1, show a recent downward trend,
                        suggesting that the contractor is delivering a quality product.

                      • Customs’ oversight of contractor testing does not provide for
                        independent verification and validation (IV&V). Industry best practices
                        include IV&V as a part of effective managerial oversight and control.

                      • The potential for ACE infrastructure to support other Department of
                        Homeland Security applications could affect both cost and schedule for
                        ACE. Customs is one of several agencies to be merged into the
                        Department of Homeland Security, and it has proposed that the ACE
                        infrastructure be used by these agencies. Because a meaningful
                        understanding of the effect of the new department’s use of the ACE
                        infrastructure is not yet known, there is the potential for existing ACE
                        cost and schedule commitments to change.



Recommendations for   To ensure that Customs has the requisite capability to manage its ACE
                      acquisition, we recommend that the Customs Service Commissioner
Executive Action      designate strengthening CMO human capital and acquisition processes as
                      priority matters. To this end, we recommend that the Commissioner direct
                      the Chief Information Officer (CIO) to immediately

                      • develop and implement each of the missing SEI SA-CMM practices for
                        the key process areas discussed in this report, and until this is
                        accomplished, report to its appropriations subcommittees quarterly on
                        the progress of its efforts to do so;

                      • develop and implement the missing human capital management
                        practices discussed in this report, and until this is accomplished, report
                        to its appropriations committees quarterly on the progress of its efforts
                        to do so; and

                      • establish an IV&V function to assist Customs in overseeing contractor
                        efforts, such as testing.

                      Additionally, we recommend that the Commissioner take steps, as
                      appropriate in light of Customs’ merger into the Department of Homeland



                      Page 4                                  GAO-03-406 Customs Service Modernization
                      Security, to have future ACE expenditure plans specifically address any
                      proposals or plans, whether tentative or approved, for extending and using
                      ACE infrastructure to support other homeland security applications,
                      including any impact on ACE of such proposals and plans.



Agency Comments and   In written comments on a draft of this report signed by the acting director,
                      Office of Planning, Customs stated that it was pleased with the report,
Our Evaluation        agreed with our recommendations, and described specific actions that are
                      being taken to implement each. Customs’ comments are reprinted in
                      appendix II.

                      Customs also provided additional information regarding our
                      recommendation that it establish an IV&V function to assist in overseeing
                      contractor efforts, such as testing. Specifically, Customs stated that it
                      wanted to modify its earlier statement that it relies on the contractor’s
                      reputation and maturity level as guarantors of system quality. Customs said
                      that its modified position is based on a number of practices that it believes
                      are elements of an integrated approach to IV&V. However, the practices
                      that Customs described as elements of IV&V, while additive to system
                      quality, are not independent from program cost and schedule pressures,
                      and therefore do not constitute IV&V.


                      We are sending copies of this report to the Chairmen and Ranking Minority
                      Members of other Senate and House committees and subcommittees that
                      have authorization and oversight responsibilities for the Customs Service.
                      We are also sending copies to the Secretary of Homeland Security, the
                      Secretary of the Treasury, the Commissioner of the Customs Service, and
                      the Director of OMB. We also will make copies available to others upon
                      request. In addition, the report will be available at no charge on the GAO
                      Web site at http://www.gao.gov.




                      Page 5                                  GAO-03-406 Customs Service Modernization
Should you or your staff have any questions on matters discussed in this
report, please contact me at (202) 512-3439. I can also be reached by E-mail
at HiteR@gao.gov. Key contributors to this report were Mark Bird, Harold
Brumm, Jr., Barbara Collier, Scott Farrow, Joanne Fiorino, Tamra
Goldstein, Michael Holland, Freda Paintsil, Madhav Panwar, Karen Richey,
Randolph Tekeley, and Aaron Thorne.




Randolph C. Hite
Director, Information Technology Architecture
  and Systems Issues




Page 6                                 GAO-03-406 Customs Service Modernization
Appendix I

Customs’ Fourth Automated Commercial                                                               Appendx
                                                                                                         ies




Environment (ACE) Expenditure Plan                                                                  Append
                                                                                                         x
                                                                                                         Ii




    Customs’ Fourth Automated Commercial
     Environment (ACE) Expenditure Plan

                               Briefing for the staffs of the
                   Subcommittee on Treasury and General Government
                           Senate Committee on Appropriations
                                            and
             Subcommittee on Treasury, Postal Service, and General Government
                           House Committee on Appropriations
                                        December 27, 2002




                               Page 7                       GAO-03-406 Customs Service Modernization
                                Appendix I
                                Customs’ Fourth Automated Commercial
                                Environment (ACE) Expenditure Plan




                                                                                   Briefing Contents
• Introduction

• Objectives, Scope, and Methodology

• Results in Brief

• Background

• Results

• Conclusions

• Recommendations

• Agency Comments




                                                                                                         2




                                Page 8                                 GAO-03-406 Customs Service Modernization
                                              Appendix I
                                              Customs’ Fourth Automated Commercial
                                              Environment (ACE) Expenditure Plan




                                                                                                                                        Introduction
The U.S. Customs Service is acquiring a new trade processing system known as the Automated Commercial
Environment (ACE). ACE is intended to
  • promote more efficient movement of legitimate trade and more effective enforcement of trade laws,

  • strengthen border security operations, and

  • include a single system interface between the trade community and the federal government, known as the
    International Trade Data System (ITDS), which is to reduce the data reporting burden placed on the trade
    community and facilitate the collection and dissemination of trade data for agencies with trade-related missions.

                      ACE/ITDS
                                       ACE
                                                                  ITDS
                                                                                                   Functions unique to
                                                                                                      a single other
                                         Functions unique           Functions common                 federal agency
                                           to Customs                 to Customs and
                                            operations             other federal agencies
                                                                                                  Functions common to
                                                                                                  other federal agencies




                      Common interface or “front end” (e.g., Web portal) for Customs, other federal agencies, and trade organizations




                        Brokers and
                      trade advisors
                                       Importers and
                                         exporters
                                                       Carriers          Customs       Agency 1        Agency 2
                                                                                                                   …        Agency
                                                                                                                              104

                                   Trade community                                              Other federal agencies
                     Source: GAO.


                                                                                                                                                    3




                                              Page 9                                                              GAO-03-406 Customs Service Modernization
                                                      Appendix I
                                                      Customs’ Fourth Automated Commercial
                                                      Environment (ACE) Expenditure Plan




                                                                                                                                               Introduction
                                                         1
Customs is required in its appropriations not to obligate funds for ACE until it submits to the Committees on
Appropriations ACE expenditure plans that
    • meet the capital planning and investment control review requirements of the Office of Management and Budget
      (OMB), including security planning;

    • comply with the Customs enterprise architecture;

    • comply with federal acquisition rules, requirements, guidelines, and systems acquisition management practices;

    • are reviewed and approved by the Customs Investment Review Board, Treasury, and OMB; and

    • are reviewed by GAO.
Since 1999, we have conducted four reviews of Customs’ ACE project, including our reviews of the first three
                  2
expenditure plans. In particular, in May 2002, we reported that ACE was a high-risk project and made
recommendations for strengthening Customs’ ACE management. Customs has implemented some of our
recommendations and continues to work on others.




1
  Public Law 107-294 is the current continuing resolution providing funding for ACE through January 11, 2003. The continuing resolution appropriates such amounts
as may be necessary under the authority and conditions provided in the applicable appropriations act for fiscal year 2002 for continuing projects or activities that
were conducted in fiscal year 2002, at a rate for operations not exceeding the current rate, and for which appropriations, funds, or other authority was made
available in the Treasury and General Government Appropriations Act, 2002. The continuing resolution further states that these appropriations shall be available "to
the extent and in the manner which would be provided by" the FY 2002 Treasury Appropriations Act.
2
  U.S. General Accounting Office, Customs Service Modernization: Third Expenditure Plan Meets Legislative Conditions, but Cost Estimating Improvements
Needed, GAO-02-908 (Washington, D.C.: Aug. 9, 2002); Customs Service Modernization: Management Improvements Needed on High-Risk Automated Commercial
Environment Project, GAO-02-545 (Washington, D.C.: May 13, 2002); Customs Service Modernization: Results of Review of First Automated Commercial
Environment Expenditure Plan, GAO-01-696 (Washington, D.C.: June 5, 2001); Customs Service Modernization: Serious Management and Technical Weaknesses
Must Be Corrected, GAO/AIMD-99-41 (Washington, D.C.: Feb. 26, 1999).
                                                                                                                                                                     4




                                                      Page 10                                                       GAO-03-406 Customs Service Modernization
                                                      Appendix I
                                                      Customs’ Fourth Automated Commercial
                                                      Environment (ACE) Expenditure Plan




                                                                                                       Objectives, Scope and Methodology
Objectives
As agreed, our objectives were to
    • determine whether the fourth ACE expenditure plan satisfies the legislative conditions,

    • determine whether the plan is consistent with our open ACE recommendations, and

    • provide observations about the plan and Customs’ management of ACE.

Scope and Methodology
To accomplish our objectives, we analyzed the plan and supporting documents and activities, comparing them to
relevant federal requirements and guidance and applicable best practices. In particular, we compared the
    • ACE Increment 2 Cost Benefit Analysis to the Office of Management and Budget’s (OMB) policy for cost benefit
      analysis of federal programs;
    • Customs Modernization Office (CMO) Human Capital Management Strategy to key principles from private
                           3
      sector organizations; and

    • CMO’s acquisition management improvement program to the Software Engineering Institute’s (SEI) Software
                                                     4
      Acquisition Capability Maturity Model (SA-CMM); and
    • test plans, test results, testing activities, and independent verification and validation (IV&V) activities to
                                                                      5
      industry best practices as discussed in GAO’s testing guide.


3
  U.S. General Accounting Office, Human Capital: Key Principles from Nine Private Sector Organizations, GAO/GGD-00-28 (Washington, D.C.: Jan. 31, 2000).
4
  Capability Maturity ModelSM is a service mark of Carnegie Mellon University, and CMM is registered in the U.S. Patent and Trademark Office. The SA-CMM
identifies key process areas that are necessary to effectively manage software-intensive system acquisitions. Level 2 is the second level of the SA-CMM’s five-level
scale; achieving this level means that an organization has the software acquisition rigor and discipline to repeat project successes.
5
  U.S. General Accounting Office, Year 2000 Computing Crisis: A Testing Guide GAO/AIMD-10.1.21 (Washington, D.C.: November 1998).
                                                                                                                                                                       5




                                                      Page 11                                                       GAO-03-406 Customs Service Modernization
                                     Appendix I
                                     Customs’ Fourth Automated Commercial
                                     Environment (ACE) Expenditure Plan




                                                                       Objectives, Scope and Methodology
Scope and Methodology (cont.)
We also interviewed ACE program management, support contractors, and modernization contractor officials to
understand the scope and content of plans, clarify information in supporting documentation, and seek explanations
for deviations from federal or commercial criteria.
As agreed, we did not independently validate the project content and status information that Customs provided.
However, we used Customs’ provided data and the CostXpert software cost estimating model to check the
reasonableness of the software development cost estimate contained in the expenditure plan.
We performed our work from November 2002 through December 2002 in accordance with generally accepted
government auditing standards.




                                                                                                                    6




                                     Page 12                                     GAO-03-406 Customs Service Modernization
                                                     Appendix I
                                                     Customs’ Fourth Automated Commercial
                                                     Environment (ACE) Expenditure Plan




                                                                                                                 Results in Brief: Objective 1
Customs’ expenditure plan satisfies the legislative conditions.

 Legislative conditions                                                                                                Satisfies                Does not
                                                                                                                                                 satisfy
 1.    Meets OMB capital planning and investment control review requirements,                                               9
                                           6
       including OMB Circular A-11, part 3
 2.    Complies with the Customs enterprise architecture                                                                    9

 3.    Complies with federal acquisition rules, requirements, guidelines, and                                               9
       systems acquisition management practices
 4.    Is reviewed and approved by the Customs Investment Review Board,                                                     9
       Treasury, and OMB
 5.    Is reviewed by GAO                                                                                                   9




6
  Customs provided us with a draft of the expenditure plan before it was submitted to the appropriations subcommittees. We determined that the draft plan did not
satisfy the requirements of OMB Circular A-11, part 3 (July 2001). After we discussed our analysis with Customs and support contractor officials, Customs amended
the expenditure plan to satisfy the requirements before submitting it to the appropriations subcommittees.
                                                                                                                                                                  7




                                                     Page 13                                                      GAO-03-406 Customs Service Modernization
                                       Appendix I
                                       Customs’ Fourth Automated Commercial
                                       Environment (ACE) Expenditure Plan




                                                                                  Results in Brief: Objective 2
Customs’ expenditure plan is consistent with our open recommendations.

Open recommendations                                                    Consistent                       Status
                                                                   Fully       Partially      Complete       In progress

1.   Justify and make investment decisions incrementally.           9                                             9

2.   Before building each ACE release, certify to Customs’          9                                             9
     appropriations subcommittees that Customs’ enterprise
     architecture has been sufficiently extended and updated.

3.   Develop and implement a rigorous and analytically              9                                             9
     verifiable cost estimating program that embodies the
     tenets of effective estimating as defined in the Software
     Engineering Institute’s (SEI) institutional and project-
     specific estimating guidance.

4.   Immediately develop and implement a Customs                                  9                               9
     Modernization Office (CMO) human capital management
     strategy.




                                                                                                                           8




                                       Page 14                                       GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                         Results in Brief: Objective 2 (cont.)


Open recommendations                                                  Consistent                       Status
                                                                 Fully        Partially     Complete       In progress

5.   Limit future expenditure plan requests for management        9                                             9
     reserve funds to 10 percent of the total funds requested
     for the program, or adequately justify any management
     reserve requests in excess of 10 percent.

6.   Develop and implement process controls for SEI’s SA-                          9                            9
     CMM, and by September 30, 2002, assess and report to
     Customs’ appropriation subcommittees on CMO’s
     software acquisition process maturity.

7.   Address the risks associated with the accelerated ACE        9                                             9
     acquisition strategy and report on the strategy going
     forward and plans for mitigating the risks associated
     with this strategy.




                                                                                                                         9




                                      Page 15                                      GAO-03-406 Customs Service Modernization
                                       Appendix I
                                       Customs’ Fourth Automated Commercial
                                       Environment (ACE) Expenditure Plan




                                                                                 Results in Brief: Objective 3
Other observations:
Contractor-performed system integration and acceptance testing is conducted in accordance with best practices.
Contractor-performed system acceptance test results are positive.
System defect trends suggest contractor is delivering a quality product.
Customs’ oversight of contractor testing does not provide for independent verification and validation (IV&V).
Potential for ACE infrastructure to support other Department of Homeland Security applications could effect ACE
cost and schedule.


Summary of recommendations:
To ensure that Customs is positioned to effectively perform its role as the acquirer of ACE, we are making
recommendations to the Customs Commissioner to expeditiously implement overdue people and process capability
improvements and to keep its appropriations committees informed about ACE acquisition matters.




                                                                                                                   10




                                       Page 16                                    GAO-03-406 Customs Service Modernization
                                                       Appendix I
                                                       Customs’ Fourth Automated Commercial
                                                       Environment (ACE) Expenditure Plan




                                                                                                                                                    Background
ACE Acquisition Strategy
On April 27, 2001, Customs awarded a 5-year, indefinite-delivery, indefinite-quantity contract to International Business
Machines (IBM) Global Services. IBM and its subcontractors are collectively called the e-Customs Partnership (eCP).
Through a series of contract task orders, Customs plans to acquire and implement ACE in four increments. The first
two increments each have two releases.
The following table summarizes primary ACE functions by increment and release.
Functions                                                                                                        ACE increment7
                                                                                            1                         2                         3                4
                                                                               Release 1        Release 2    Release 1       Release 2
Enterprise portal and IT infrastructure                                           Initial       Additional    Additional     Additional      Additional     Remaining
Data processing (e.g., reference library and data query capabilities)             Initial       Additional    Additional     Additional      Additional     Remaining
Trade agreement (e.g., manage visas, quotas; implement trade                                                                                   Initial      Remaining
agreements)
Integration of ITDS functionality                                                                 Initial     Additional     Additional     Additional      Remaining
Account management (e.g., create, access accounts)                                Initial       Additional    Additional     Additional     Remaining
Account revenue (e.g., generate billing statements, accept                                        Initial     Additional     Additional     Additional      Remaining
payments)
Enforcement (e.g., determine violation severity, resolve compliance                               Initial     Additional     Additional     Remaining
violations)
Selectivity criteria for assessing noncompliance risk, screening                                                Initial      Additional      Additional     Remaining
incoming transactions
Cargo release and tracking                                                                        Initial     Additional     Additional      Additional     Remaining
Ability to process multi-modal goods, full export functions,                                                                                                   All
                                                             8
decommissioning of Automated Commercial System (ACS)
Source: GAO.


7
  All means all capabilities within this function are provided in this increment or release; initial means a beginning set of capabilities is provided, and the number
and type of users may be limited for certain capabilities; additional means an increased set of capabilities is provided, and the number and type of users may be
expanded for specific capabilities; remaining means that the final set of capabilities is provided to intended users.
8
  This system tracks, controls, and processes all goods imported into the United States. ACE is expected to provide all functions currently provided by this system.
                                                                                                                                                                       11




                                                       Page 17                                                        GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                                    Background
The following tables summarize complete and ongoing ACE contract task orders. The first table describes tasks
associated with managing the Customs modernization program and the ACE project. The second describes tasks
associated with developing and implementing ACE.


Management Task         Start            Status          Description
Program Management      August 2001      Ongoing         Apply the Customs enterprise life cycle management
                                                         process to the various modernization program activities.
Enterprise              August 2001      Ongoing         Updates enterprise architecture and develops Web portal
Architecture and                                         and plan for transitioning to the new modernization
Engineering                                              environment.
Enterprise Process      February 2002    Ongoing         Supports integrating all process improvement efforts
Improvement Support                                      across the enterprise and develops enterprise quality
                                                         assurance program.
ACE/ITDS Integration    February 2002    Ongoing         Establishes a strategy for integrating ITDS with ACE, and
Support                                                  secures stakeholder buy-in.
Source: GAO.




                                                                                                                    12




                                      Page 18                                   GAO-03-406 Customs Service Modernization
                                   Appendix I
                                   Customs’ Fourth Automated Commercial
                                   Environment (ACE) Expenditure Plan




                                                                                                Background

ACE Increment Task   Start            Status          Description
ACE Requirements and August 2001      Completed       Updates and develops the requirements and desired
Planning—Increment 1                  February 2002   business results and developed the concept of operations
                                                      and project plan for ACE increment 1.
ACE Increment 1,     February 2002    Ongoing         Designs, develops, and implements ACE increment 1,
Release 1                                             release 1.
Implementation
ACE Increment 1,     August 2002      Ongoing         Designs, develops, and implements ACE increment 1,
Release 2                                             release 2.
Implementation
ACE Requirements and February 2002    Ongoing         Updates requirements and desired business results and
Planning—Increment 2                                  develops concept of operations and project plan for ACE
                                                      increment 2.
Source: GAO.




                                                                                                                13




                                   Page 19                                  GAO-03-406 Customs Service Modernization
                                                     Appendix I
                                                     Customs’ Fourth Automated Commercial
                                                     Environment (ACE) Expenditure Plan




                                                                                                                                              Background
Customs’ Fourth Expenditure Plan Summary
On November 22, 2002, Customs submitted its fourth ACE expenditure plan, seeking release of $314 million.
Activities                                                                                                                                Amount $ millions
Increment 2, release 1 definition, design and development and release 2 definition                                                                      $185.9
Increment 3 requirements and planning                                                                                                                     $14.8
Increment 1 operations and maintenance                                                                                                                      $6.1
eCP ACE program management                                                                                                                                $11.9
Enterprise architecture and engineering                                                                                                                   $15.7
CMO costs                                                                                                                                                 $37.5
ITDS development and integration into ACE                                                                                                                   $5.4
                                                                                                                    9
National Customs Automation Program (NCAP) and Free and Secure Trade (FAST)                                                                                 $4.5
  operations and maintenance
     10
SAP implementation support                                                                                                                                  $1.5
    Subtotal                                                                                                                                            $283.4
                             11
Management reserve                                                                                                                                        $30.6
    Total                                                                                                                                               $314.0



9
   NCAP is the ACE prototype system that has served as a pilot for demonstrating certain ACE functionality, including account-based import processing. FAST is an
enhancement to NCAP that allows expedited border processing for shipments involving importers, carriers, and commercial drivers who are predetermined to be
low risk. ACE funding is being used to operate and maintain both NCAP and FAST.
10
   The SAP Implementation Project, a joint project of the Office of Finance and the Office of Information Technology, is intended to provide automated enterprise
resource planning. The SAP product provides multiple modules, each performing separate, but integrated business functions. Some are used to support Customs’
financial management, and others may be used to support ACE functions, such as customer relationship management and supply chain management.
11
   Customs has accumulated $34.4 million in management reserve from prior funding releases. As of December 11, 2002, Customs reports that $0.7 million of the
$34.4 million has been obligated.
                                                                                                                                                                  14




                                                     Page 20                                                      GAO-03-406 Customs Service Modernization
                                                    Appendix I
                                                    Customs’ Fourth Automated Commercial
                                                    Environment (ACE) Expenditure Plan




                                                                                                                                            Background
ACE Increment 2 Release 1
Customs’ fourth expenditure plan includes acquisition activities associated with the second of four ACE increments.
Specifically, the plan requests $185.9 million for the definition of increment 2, releases 1 and 2, and the design and
development of increment 2, release 1. Increment 2, release 1, is to be deployed at all ports (truck, rail, sea, and air).
Increment 2, release 1, is intended to
     • expand account and revenue management capability to high-volume carriers and brokers;

     • enhance query capability and provide a data warehouse;

     • provide consolidated selectivity, targeting, and enforcement functionality;

     • provide access to trade data for four federal agencies; and
                                                                                 12




     • establish a link to Customs Automated Export System.
                                                                                  13




12
   The four federal agencies are the Food and Drug Administration (Center for Food Safety and Nutrition), Department of Agriculture (Animal and Plant Health
Inspection Service), Immigration and Naturalization Service (crew release data only), and Census Bureau.
13
   This system allows electronic transmissions of key export data from exporters, exporters’ agents, and outbound carriers.
                                                                                                                                                               15




                                                    Page 21                                                      GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                                       Background
Chronology of Prior ACE Expenditure Plans and Funding Releases
In March 2001, Customs’ appropriations committees released to Customs $5 million in stopgap funding for the ACE
program to sustain CMO operations.
On March 26, 2001, Customs submitted its first ACE expenditure plan to its appropriations committees, seeking
release of $45 million to award the modernization contract, sustain CMO operations, and provide the support needed
to manage the modernization program. Customs’ appropriations committees subsequently approved the use of $45
million, bringing the total ACE funding to $50 million.
On February 1, 2002, Customs submitted its second ACE expenditure plan to its appropriations committees seeking
release of $206.9 million to sustain CMO operations; define, design, develop, and deploy increment 1, release 1; and
identify increment 2 requirements. Customs’ appropriations committees subsequently approved the use of $188.6
million, bringing total ACE funding to $238.6 million.
On May 24, 2002, Customs submitted its third ACE expenditure plan to its appropriations committees, seeking release
of $190.2 million to finance the definition, design, development, and implementation of ACE increment 1, release 2.
Customs’ appropriations committees subsequently approved the use of $190.2 million, bringing total ACE funding to
$428.8 million.
The following figures chronicle the history of ACE appropriations, expenditure plans, and funding releases.




                                                                                                                       16




                                      Page 22                                     GAO-03-406 Customs Service Modernization
                                           Appendix I
                                           Customs’ Fourth Automated Commercial
                                           Environment (ACE) Expenditure Plan




                                                                                                                             Background
ACE appropriations, expenditure plans, and funding releases

            FY 2001 Appropriations               FY2002 Appropriations                  FY 2003 Proposed appropriations
                = $130 million                       = $300 million                              = $313 million


                        April 2001
          March 2001    $45 million                                                          Fourth expenditure plan
          $5 million    funding                                                                  for $314 million
          funding       release       March 2002
          release                     $188.6 million
                                      funding                            $1.2 million
                                      release




                        $80 million


                                                                                                 $312.8 million
                                               $108.6 million


                                                                $190.2 million




                                                        June 2002                        $0.2 million
                                                        $190.2 million
                                                        funding
                                                        release


         Source: GAO.



                                                                                                                                          17




                                           Page 23                                                       GAO-03-406 Customs Service Modernization
                                          Appendix I
                                          Customs’ Fourth Automated Commercial
                                          Environment (ACE) Expenditure Plan




                                                                                                                                       Background
Illustrated chronology of ACE expenditure plans, funding releases, and contractual events

     $5M funding            $45M                                                     $188.6M                 $190.2M
       release             funding                                                    funding                 funding
                           release                                                    release                 release
                                          Contractor
                                          authorized to
                                          proceed
                                                                      Task order 6
                               Contract                               started
                               awarded



                                            Task orders 1,       Task orders 4,                                         Task order 8
                                            2, 3 started         5, 7 started                                           started




            3/01                6/01       9/01              12/01                3/02                 6/02                9/02              12/02



              First plan                                              Second plan               Third plan                             Fourth plan
              submitted                                                submitted                submitted                               submitted


    Source: GAO.


Cumulative funding through third expenditure plan = $428.8 million.




                                                                                                                                                     18




                                          Page 24                                                     GAO-03-406 Customs Service Modernization
                                             Appendix I
                                             Customs’ Fourth Automated Commercial
                                             Environment (ACE) Expenditure Plan




                                                                                                                             Background
Conceptual View of the ACE Architecture

                                                                        Enterprise
                                                       Web &            Application    Applications
                                 Client Tier                                                           Data Tier
                                                   Interface Tier       Integration        Tier
                                                                            Tier



                                   Trade
                                  Systems              External                            ACE
                       Trade                                                                              ACE
                                               F       Interface    F
                     Community
                                               I                    I
                                  Internet     R                    R     Middleware
                                  Browser      E                    E
                                               W    Enterprise      W
                                               A      Web           A
                                   Other       L      Portal        L                     Legacy
                     Customs &                 L                    L                     (e.g. ACS)    Legacy
                                  Network
                       Other      Devices
                      Agencies


                                                                        Security
                     Source: GAO and Customs.


The Client Tier includes the user’s workstation and external systems that access ACE.
The Web & Interface Tier provides the interface required to access functionality and data, as well as the
infrastructure required to support that access.
The Enterprise Application Integration Tier provides middleware connecting business applications and users across
Customs.
The Applications Tier includes the modernized systems (i.e., ACE) and legacy systems (e.g., ACS) that provide the
automated business applications supporting execution of Customs’ business processes.
The Data Tier provides data storage resources.
                                                                                                                                          19




                                             Page 25                                                     GAO-03-406 Customs Service Modernization
                                     Appendix I
                                     Customs’ Fourth Automated Commercial
                                     Environment (ACE) Expenditure Plan




                                                                                           Results: Objective 1

Legislative Requirement                    Satisfies?   Results of our analysis
1. Plan meets OMB Capital planning             9        The expenditure plan addresses OMB capital planning and
   and investment control review                        investment control review requirements, including OMB
   requirements, including OMB                          Circular A-11, part 3. For example, the expenditure plan
   Circular A-11, part 3.                               addresses
                                                           • federal security and privacy regulations;
                                                           • Government Paperwork Elimination Act
                                                             requirements;
                                                           • Government Performance Results Act requirements;
                                                           • how the project will use a performance-based
                                                             management system; and
                                                           • cost, benefit, and schedule information.


2. Plan complies with Customs’                 9        The expenditure plan is for, among other things, the
   enterprise architecture                              definition, design, and development of ACE increment 2,
                                                        release 1 (61 percent of requested funding). Customs plans
                                                        to certify that increment 2, release 1, is compliant with
                                                        Customs’ enterprise architecture before beginning detailed
                                                        design activities.




                                                                                                                   20




                                     Page 26                                      GAO-03-406 Customs Service Modernization
                                                        Appendix I
                                                        Customs’ Fourth Automated Commercial
                                                        Environment (ACE) Expenditure Plan




                                                                                                              Results: Objective 1

     Legislative Requirement                                  Satisfies?   Results of our analysis
     3. Plan complies with federal                                9        The expenditure plan is for, among other things,
        acquisition rules, requirements,                                   continuation of Customs’ acquisition management
        guidelines, and systems acquisition                                improvement efforts based on the SEI’s SA-CMM, as well
        management practices.                                              as for implementation of OMB acquisition and investment
                                                                                                      14
                                                                           management requirements.


     4. Plan is reviewed and approved by                          9        The Customs and Treasury Investment Review Boards
        Customs Investment Review Board,                                   were consolidated to form the Joint Capital Investment
        Treasury, and OMB.                                                 Review Board, which approved the plan on October 15,
                                                                           2002. OMB approved the plan on November 12, 2002.


     5. Plan is reviewed by GAO.                                  9        Customs delivered its expenditure plan to Congress on
                                                                           November 22, 2002. Our review was completed December
                                                                           26, 2002.




14
     OMB Circular A-130 and A-11, part 3 (July 2001).
                                                                                                                                      21




                                                        Page 27                                      GAO-03-406 Customs Service Modernization
                                                       Appendix I
                                                       Customs’ Fourth Automated Commercial
                                                       Environment (ACE) Expenditure Plan




                                                                                                                                     Results: Objective 2
     Open recommendation: Justify and make investment decisions incrementally, and for each increment,
          • use disciplined investment processes;
          • prepare realistic and supportable benefit expectations;
          • require a favorable return on investment and compliance with Customs architecture before making an
            investment decision; and
          • validate actual costs and benefits once an increment is piloted.
     Customs’ actions to address this recommendation are fully consistent and in progress.


Customs has committed to and defined process controls for justifying and making investment decisions
incrementally. For ACE increment 2, Customs
         • followed its investment management process, which we determined meets OMB’s policy for management
                                             15
           of federal information resources;

         • prepared benefit expectations, which we determined are consistent with OMB’s policy for cost benefit
                                         16
           analyses of federal programs;

         • reported a favorable return on investment (benefit/cost ratio for Customs of 1.09 ) and plans to certify
                                                                                                                                17


           compliance with Customs architecture before beginning detailed design activities; however, this return
           on investment assumes a 4-year implementation schedule, and Customs is planning to extend this
           schedule—this extension could increase costs and delay benefits and thus decrease the benefit/cost ratio;
           and

         • plans to validate actual costs and benefits during post-implementation reviews.


15
   OMB Circular A-130.
16
   OMB Circular A-94.
17
   This ratio includes costs and benefits to Customs, exclusive of costs and benefits that are identified as positive to the trade community.
                                                                                                                                                       22




                                                       Page 28                                                        GAO-03-406 Customs Service Modernization
                                     Appendix I
                                     Customs’ Fourth Automated Commercial
                                     Environment (ACE) Expenditure Plan




                                                                                           Results: Objective 2
Open recommendation: Before building each ACE release (i.e., beginning detailed design and development), certify
to Customs’ House and Senate appropriations subcommittees that the enterprise architecture has been sufficiently
extended to provide the requisite enterprise design content and has been updated to ensure consistency and
integration across business areas.
Customs’ actions to address this recommendation are fully consistent and in progress.


On July 16, 2002, before beginning detailed design and development of ACE increment 1, release 1, Customs certified
to its House and Senate appropriations subcommittees that the enterprise architecture had been sufficiently
extended to provide the requisite enterprise design content and has been updated to ensure consistency and
integration across business areas. Customs plans to recertify its enterprise architecture before building ACE
increment 1, release 2 (scheduled for recertification in January 2003), and to continue doing so on all future ACE
releases.




                                                                                                                  23




                                     Page 29                                     GAO-03-406 Customs Service Modernization
                                                     Appendix I
                                                     Customs’ Fourth Automated Commercial
                                                     Environment (ACE) Expenditure Plan




                                                                                                                                Results: Objective 2
 Open recommendation: Develop and implement a rigorous and analytically verifiable cost estimating program that
 embodies the tenets of effective estimating as defined in SEI’s institutional and project-specific estimating models.
 Customs’ actions to address this recommendation are fully consistent and in progress.


On September 26, 2002, Customs hired a contractor to define and implement a cost estimating program for CMO.
                                                                                               18
Customs has specified that the program comply with industry best practices and SEI guidance. Customs expects to
initiate the program in December 2002. As a result, this program was not in place during the development of the cost
estimates contained in the fourth expenditure plan.
To validate the reliability of the fourth expenditure plan cost estimate for ACE increment 2, Customs tasked its
support and modernization contractors with comparing the estimate against SEI criteria for validating software cost
and schedule estimates. The results of this comparison are shown in the following table.


              SEI Question                           Satisfies? 19                                             Description
                                                   Fully      Partially
1. Are the objectives of the
   estimate clear and correct?
                                                     9                     The objective of the estimate was clearly stated. The tasks and
                                                                           activities to be included in the estimate were clearly identified.
2. Has the task been appropriately
   sized?
                                                     9                     The task was appropriately sized. For example, software sizing
                                                                           estimates were determined by requirements, and function point
                                                                           counts were performed by certified staff .20




18
   SEI’s institutional and project-specific estimating guidelines are defined in Checklist and Criteria for Evaluating the Cost and Schedule Estimating Capabilities
of Software Organizations and A Manager’s Checklist for Validating Software Cost and Schedule Estimates, respectively.
19
   Fully means that Customs’ actions have completely addressed the question; partially means that Customs’ actions have not completely addressed the question as
indicated in the description.
20
   The International Function Point Users’ Group maintains the recognized industry standard for function point analysis, which is a methodology for software sizing.
The group offers professional certification for practitioners of function point analysis as Certified Function Point Specialists.
                                                                                                                                                                  24




                                                     Page 30                                                      GAO-03-406 Customs Service Modernization
                                        Appendix I
                                        Customs’ Fourth Automated Commercial
                                        Environment (ACE) Expenditure Plan




                                                                                                    Results: Objective 2
          SEI Question                  Satisfies?                                    Description
                                      Fully   Partially
3. Are the estimated cost and
   schedule consistent with
                                                  9       The estimated costs are consistent with other projects. For example,
                                                          the cost and schedule models used to develop the estimate were
   demonstrated accomplishments                           calibrated to eCP historical data of other programs. However,
   on other projects?                                     sufficient project experience has not been acquired to confirm the
                                                          validity and relevance of the selected historical data.
4. Have the factors that affect the
   estimate been identified and
                                       9                  The factors that affect the estimate were identified and explained.
                                                          For example, uncertainties in software and other parameter values
   explained?                                             were identified, and estimates were developed and documented.
5. Have steps been taken to ensure
   the integrity of the estimating
                                                  9       Steps were taken to ensure the integrity of the estimating process.
                                                          For example, a second software estimating tool, Software Life Cycle
   process?                                               Model was used for validating the software cost estimate. Cost
                                                          models produced software estimates for normal and compressed
                                                          schedules. However, the cost models do not support schedules with
                                                          the extensive level of parallel development activities that are
                                                          necessary to achieve the proposed ACE schedule.
6. Is the organization’s historical               9       Historical evidence was capable of supporting a reliable estimate.
   evidence capable of supporting                         For example, eCP draws from a database that contains productivity
   a reliable estimate?                                   data and support cost percentages from a large set of completed
                                                          projects. However, the data did not support the extensive level of
                                                          parallel ACE development activities.
7. Has the situation changed since    NA                  Because the situation had not changed, this question was not
   the estimate was prepared?                             applicable to planning estimates.


We reviewed the contractors’ analysis and found no reason to question it. For example, using a different cost
estimating tool (CostXpert) with eCP’s software size estimates and assumptions, we found that the estimate for ACE
increment 2, release 1, was reasonable.



                                                                                                                            25




                                        Page 31                                          GAO-03-406 Customs Service Modernization
                                                     Appendix I
                                                     Customs’ Fourth Automated Commercial
                                                     Environment (ACE) Expenditure Plan




                                                                                                                            Results: Objective 2
     Open recommendation: Immediately develop and implement a CMO human capital management strategy that
     provides both near-term and long-term solutions to CMO’s human capital capacity limitations.
     Customs’ actions to address this recommendation are partially consistent and in progress.
We made our recommendation in March 2002, with the intent that Customs specify in its strategy how it would
accomplish its human capital goals (i.e., who would do what and by when, what resources would be needed, and
how progress would be measured).
CMO released a human capital management strategy in October 2002 that specifies high-level commitments,
including time frames, such as completing vacancy action plans by April 2003, creating an acquisition program
management career track by April 2004, and defining performance standards by June 2003.
Based on our analysis of the strategy against 10 human capital management principles drawn from commercial best
          21
practices, we found that the strategy either fully or partially recognizes each of the principles (see following table
for a comparison). However, neither the strategy nor supporting documentation identifies, for example, how the
commitments made in the strategy will be met, including what steps will be taken and what resources are needed to
execute the steps. Moreover, based on the strategy’s timeline, it will be over a year before CMO implements certain
principles.
According to Customs officials, development and implementation of its human capital management strategy has
been delayed because CMO staff have been addressing higher priority areas.




21
     U.S. General Accounting Office, Human Capital Key Principles from Nine Private Sector Organizations, GAO/GGD-00-28 (Washington, D.C.: January 2000).
                                                                                                                                                            26




                                                     Page 32                                                   GAO-03-406 Customs Service Modernization
                                                      Appendix I
                                                      Customs’ Fourth Automated Commercial
                                                      Environment (ACE) Expenditure Plan




                                                                                                                                   Results: Objective 2

Human Capital Principles                                Satisfies?22                                    Summary of Comparison
                                                      Fully      Partially
Treat human capital management as
being fundamental to strategic
                                                        9                     The human capital management strategy ties the organization’s
                                                                              human capital policies, processes, and practices to the
business management.                                                          modernization mission and strategic goals.
Integrate human capital functional
staff into management teams.
                                                                    9         The strategy does not provide for integrating human capital
                                                                              functional staff into CMO. However, a human capital staff member
                                                                              assisted in developing the strategy.
Leverage the internal human capital
function with external expertise.
                                                        9                     The CMO person responsible for human capital management is
                                                                              assisted by external expertise from its support contractor.
Hire, develop, and retain employees
according to competencies.
                                                                    9         The strategy states that CMO will hire, develop, and retain
                                                                              employees according to a planned training program and CMO-
                                                                              specific competencies that are yet to be defined. The date for
                                                                              completing competency definitions was extended from November
                                                                              25 to December 31, 2002. The date for initiating training program
                                                                              development was extended from September 30 to December 31,
                                                                              2002.
Hire, develop, and sustain leaders
according to leadership
                                                                    9         The strategy states that CMO will hire, develop, and sustain
                                                                              leaders according to leadership characteristics, which are
characteristics identified as essential                                       expected to be defined in CMO-specific competencies.
to achieving specific missions and
goals.




22
 Fully means that the strategy provides for implementing the principle and the actions associated with implementing the principle have been completed. Partially
means that (1) the strategy states that the principle will be implemented, but does not specify the actions necessary to do so; (2) the strategy describes some, but
not all, actions necessary to implement the principle, but these actions have been delayed; or (3) the strategy does not state that the principle will be implemented,
but Customs took some other action related to implementing the principle.
                                                                                                                                                                     27




                                                      Page 33                                                       GAO-03-406 Customs Service Modernization
                                       Appendix I
                                       Customs’ Fourth Automated Commercial
                                       Environment (ACE) Expenditure Plan




                                                                                                     Results: Objective 2
Human Capital Principles                 Satisfies?                             Summary of Comparison
                                       Fully   Partially
Communicate a shared vision that all
employees, working as one team, can
                                                 9         The strategy states that CMO will develop guidance that is intended
                                                           to communicate a shared vision that all employees, working as one
strive to accomplish.                                      team, can strive to accomplish. The strategy states further that this
                                                           guidance will define performance objectives linked to team
                                                           accomplishments.
Use performance management
systems, including pay and other
                                                 9         The strategy states that CMO will develop and use a performance
                                                           management system, including pay and other meaningful incentives,
meaningful incentives, to link                             to link performance to results. According to the strategy,
performance to results.                                    performance standards will be defined by June 2003 and
                                                           development goals will be set by September 2003.
Support and reward teams to achieve
high performance.
                                                 9         The strategy states that CMO will support and reward teams
                                                           through existing incentives.
Integrate employee input into the
design and implementation of human
                                                 9         The strategy states that CMO will integrate employee input into the
                                                           design and implementation of human capital policies and practices.
capital policies and practices to
achieve high performance.
Measure the effectiveness of human
capital policies and practices to
                                                 9         The strategy states that CMO will measure the effectiveness of
                                                           human capital policies and practices.
achieve high performance.


Customs’ efforts to date are not fully consistent with our recommendation, because (1) they have not been immediate
and are taking too long to accomplish, and (2) although the strategy commits to doing what we recommended,
Customs has not developed and implemented key human capital management practices to address the
recommendation.
As a result, CMO’s ability to manage its human capital resources according to these best practices, and thereby
ensure that it has the capacity to acquire ACE, is over a year away.

                                                                                                                               28




                                       Page 34                                            GAO-03-406 Customs Service Modernization
                                                    Appendix I
                                                    Customs’ Fourth Automated Commercial
                                                    Environment (ACE) Expenditure Plan




                                                                                                                              Results: Objective 2
 Open recommendation: Limit future expenditure plan requests for management reserve funds to 10 percent of the
 total funds requested for the program, or adequately justify any management reserve requests in excess of 10
 percent.
 Customs’ actions to address this recommendation are fully consistent and in progress.
Customs’ fourth expenditure plan requests $30.6 million for management reserve, which is 10.8 percent of the total
                                                          23
funds requested, excluding the management reserve itself.
Management reserves are intended to earmark funding to address project risks, including known risks that, if not
sufficiently mitigated, become actual problems that require resources to correct. Thus, estimating the amount of
funding to earmark in the reserve should be closely aligned with the project’s risk management program, including
risk identification and risk probability and impact assessment, and it should be based, in part, on the estimates of
costs to correct the potential impacts associated with known risks.
The methodology used to develop the Customs’ $30.6 million management reserve request provides for identifying
ACE risk elements, assessing risk probability and impact, and estimating the cost to correct the potential impacts
associated with known risks. Moreover, the risk elements are aligned with the risks included in the ACE risk
management program, meaning that the risk elements used in estimating the reserve are legitimate. As a result, we
found no reason to question the $30.6 million request.
The management reserve breakdown by expenditure plan activities is shown in the following table.




23
   Customs has accumulated $34.4 million in management reserve from prior funding releases. As of December 11, 2002, Customs reports that $0.7 million of the
$34.4 million has been obligated.
                                                                                                                                                                29




                                                    Page 35                                                      GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                            Results: Objective 2

Activity                                                                      Cost         Reserve       Percentage
                                                                       ($ millions)      ($ millions)    (reserve/cost)
Increment 2, releases 1 and 2, definitions                                   $17.7               $2.9             16%
Increment 2, release 1, design and development                               $168.2           $23.8               14%
Increment 3 requirements and planning                                         $14.8             $0.8               5%
Increment 1 operations and maintenance                                         $6.1             $0.2               3%
eCP ACE program management                                                    $11.9             $0.8               7%
Enterprise architecture and engineering                                       $15.7             $1.1               7%
CMO costs                                                                     $37.5             $0.9               2%
ITDS development and integration into ACE                                      $5.4             $0.0               0%
NCAP operations and maintenance and FAST implementation                        $4.5             $0.0               0%
SAP implementation support                                                     $1.5             $0.0               0%
      Total                                                                  $283.4           $30.6             10.8%




                                                                                                                    30




                                      Page 36                                     GAO-03-406 Customs Service Modernization
                                        Appendix I
                                        Customs’ Fourth Automated Commercial
                                        Environment (ACE) Expenditure Plan




                                                                                                  Results: Objective 2
Open recommendation: Develop and implement process controls for the SEI SA-CMM level 2 key process areas and
for the level 3 acquisition risk management key process area, and by September 30, 2002, conduct and report to
Customs’ House and Senate appropriation subcommittees on the results of an internal assessment of CMO’s
maturity against these key process areas.
Customs’ actions to address this recommendation are partially consistent and in progress.
The SEI, recognized for its software engineering expertise, defines key process areas that are necessary for effective
acquisition management. SEI research shows that not having effective acquisition management processes increases
the probability of systems not performing as intended and costing more and taking longer than planned. The SA-CMM
process areas are described in the following table.


Key process area                           Description
Software acquisition planning (SAP)        Ensures that reasonable planning for the software acquisition is conducted and
                                           that all elements of the project are included
Solicitation (SOL)                         Ensures that award is made to the contractor most capable of satisfying the
                                           specified requirements
Requirements development and               Establishes a common and unambiguous definition of software acquisition
management (RDM)                           requirements understood by the acquisition team, system user, and the
                                           contractor
Project office management (PM)             Manages the activities of the project office and supporting contractor(s) to
                                           ensure a timely, efficient, and effective software acquisition
Contract tracking and oversight (CTO)      Ensures that the software activities under contract are being performed in
                                           accordance with contract requirements, and that products and services will
                                           satisfy contract requirements




                                                                                                                            31




                                        Page 37                                        GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                                  Results: Objective 2
Key process area                         Description
Evaluation (EVAL)                        Determines that the acquired software products and services satisfy contract
                                         requirements before acceptance and transition to support
Transition to support (TTS)              Provides for the transition of the software products being acquired to the
                                         software support organization
Acquisition risk management (ARM)        Identifies risks as early as possible, adjusts acquisition strategy to mitigate those
                                         risks, and develops and implements a risk management process as an integral
                                         part of the acquisition process


We first recommended that Customs develop and implement the SA-CMM level two key process areas in February
1999, and in March 2002 we reiterated this recommendation, adding that Customs report to its appropriations
committees on its progress in doing so by September 2002.
On September 30, 2002, Customs reported to its House and Senate appropriations subcommittees the results of an
internal assessment of its efforts to develop and implement SA-CMM key process areas. Customs reported that its
process development effort was 90 percent complete.
However, Customs did not report on its implementation of developed processes. Progress in implementing its
processes as of November 2002 is shown in the following figure and table.
According to Customs officials, its progress in developing and implementing software acquisition processes is behind
schedule because resources (i.e., staff) are being diverted to higher priority activities.
Because Customs has not yet established effective software acquisition processes, it does not have the means to
adequately ensure that promised ACE capabilities will be delivered on time and within budget.




                                                                                                                            32




                                      Page 38                                         GAO-03-406 Customs Service Modernization
                                                                         Appendix I
                                                                         Customs’ Fourth Automated Commercial
                                                                         Environment (ACE) Expenditure Plan




                                                                                                                                       Results: Objective 2

                                                20

                                                18

                                                16
                      Number of key practices




                                                14

                                                12

                                                10

                                                 8

                                                 6

                                                 4

                                                 2

                                                 0
                                                     SAP (15)   SOL (18)     PM (16)    RDM (14)   CTO (17)   EVAL (15)   TTS (16)   ARM (15)
                                                                                                                                                Totals
      Key practices not                                  1          1              2        1          3           2          12         1        18%
      developed and not
      implemented
      Key practices                                      13         17             13       13         14         13          4          14      80%24
      developed, but not
      implemented
      Key practices                                      1          0              1        0          0           0          0          0         2%
      developed and
      implemented
Source: GAO.                                                                                                                                     100%


24
   This percentage of key practices developed is based on each key practice having equal weight; Customs’ method for determining that its process development
effort was 90 percent complete was based on a method that weighted key practices.
                                                                                                                                                                33




                                                                         Page 39                                           GAO-03-406 Customs Service Modernization
                                         Appendix I
                                         Customs’ Fourth Automated Commercial
                                         Environment (ACE) Expenditure Plan




                                                                                                              Results: Objective 2
Open recommendation: Address the risks associated with the accelerated ACE acquisition strategy.
Customs’ actions to address this recommendation are fully consistent and in progress.


Customs’ third expenditure plan contained a 4-year accelerated schedule (see figure). Before this, Customs was
following a 5-year schedule.

                                                    Fiscal        Fiscal        Fiscal      Fiscal      Fiscal
                                                    year 2001     year 2002     year 2003   year 2004   year 2005

                           Increment 1 release 1

                           Increment 1 release 2


                           Increment 2 release 1


                           Increment 2 release 2


                           Increment 3 release 1


                           Increment 3 release 2


                           Increment 4


                                 Requirements & planning
                                 Design (preliminary & detailed), develop, deploy
                          Source: GAO.




                                                                                                                                    34




                                         Page 40                                                   GAO-03-406 Customs Service Modernization
                                                      Appendix I
                                                      Customs’ Fourth Automated Commercial
                                                      Environment (ACE) Expenditure Plan




                                                                                                                                 Results: Objective 2
Customs current ACE schedule is not set, although it appears that Customs will not be pursuing the accelerated 4-
year schedule. Specifically, Customs officials told us that they began rethinking the 4-year schedule because the
funding needed to support such an accelerated schedule was not received and that they expect to follow a 5-year
         25
schedule. According to Customs, a new schedule is expected to be completed and approved by April 2003. At the
                                                                                      26
same time, Customs fourth expenditure plan includes a 6-year schedule (see figure).

                                              Fiscal         Fiscal        Fiscal      Fiscal        Fiscal          Fiscal         Fiscal
                                              year 2001      year 2002     year 2003   year 2004     year 2005       year 2006      year 2007

                     Increment 1 release 1

                     Increment 1 release 2


                     Increment 2 release 1


                     Increment 2 release 2


                     Increment 3


                     Increment 4




                            Requirements & planning
                            Design (preliminary & detailed), develop, deploy
                    Source: GAO and Customs.




25
  Actually 5 years, 7 months beginning in March 2002 and ending in September 2007.
26
  Actually 6 years, 2 months beginning in August 2001 and ending in September 2007. According to Customs, this schedule is intended to portray a high level
representation of project phases and is not intended to represent a detailed schedule.
                                                                                                                                                              35




                                                      Page 41                                                     GAO-03-406 Customs Service Modernization
                                                     Appendix I
                                                     Customs’ Fourth Automated Commercial
                                                     Environment (ACE) Expenditure Plan




                                                                                                                                Results: Objective 3
     Observation: Contractor-performed system integration and acceptance testing is conducted in accordance with
     best practices.
To be effective, test activities should be planned and conducted in a structured and disciplined fashion. Industry best
practices provide a framework for managing testing activities. This framework includes steps for establishing a
testing organization and the infrastructure for conducting test activities, and for planning and conducting incremental
phases of testing, including integration and system acceptance testing.
The contractor’s integration and system acceptance test programs are consistent with best practices. In particular,
eCP has established a test organization and built an infrastructure for test activities. Additionally, eCP has conducted
                                                                            27
system integration and acceptance tests for ACE increment 1 release 1.1, and these test activities satisfied each of
the major steps associated with test management best practices, as summarized in the following table.

                   Major Test Steps                                                                       Satisfies?
                                                                                      Integration Test             System Acceptance Test
                   Schedule and plan test                                                     9                                 9
                   Prepare test procedures                                                    9                                 9
                   Define test exit criteria                                                  9                                 9
                   Execute tests                                                              9                                 9
                   Document test results                                                      9                                 9
                                   28
                   Correct defects                                                            9                                 9
                   Ensure that test exit criteria are met                                     9                                 9




27
 ACE increment 1, release 1, consists of two sub-releases, designated releases 1.1 and 1.2.
28
 ACE increment 1, release 1.1, still had system defects at the end of both integration and acceptance testing. However, because the defects were noncritical, and
defined test exit criteria allow for noncritical defects, this step was satisfied.
                                                                                                                                                                    36




                                                     Page 42                                                      GAO-03-406 Customs Service Modernization
                                       Appendix I
                                       Customs’ Fourth Automated Commercial
                                       Environment (ACE) Expenditure Plan




                                                                                              Results: Objective 3
Observation: Contractor-performed system acceptance test results are positive.
System Acceptance Testing (SAT) is intended to show that a system performs according to the system specification
(i.e., performs specified functions within defined performance parameters). Acceptance testing is generally
conducted by the developing organization (i.e., eCP), with the participation of the acquiring organization (i.e., CMO).
The criteria for passing SAT generally provide for executing prescribed test cases without the realization of material
defects.
All but seven requirements for ACE increment 1, release 1.1, successfully passed their test cases for SAT. The seven
requirements that did not pass resulted in only noncritical system defects. Accordingly, these defects were deferred
for correction in future ACE releases. For example:
   • One of the defects related to the requirement for the portal to print search results. This defect was deemed non-
     critical because an acceptable workaround (e.g., selecting the search results and using the Web browser’s print
     function) will be used.
   • Another defect related to the requirement that the portal user sessions automatically time out if the session is
     not active after a certain amount of time. This failure resulted in identification of a defect because Customs
     policy requires users to invoke their screensavers or log off when leaving their workstations. However, the
     defect was deemed noncritical because an acceptable workaround will be used (i.e., Customs plans to reinforce
     this policy during the training that will be required before using the system).
Because all the ACE requirements for increment 1, release 1.1, either passed their tests or, in the case of the seven
failures, acceptable workarounds were identified, the first ACE release successfully completed SAT.




                                                                                                                        37




                                       Page 43                                      GAO-03-406 Customs Service Modernization
                                        Appendix I
                                        Customs’ Fourth Automated Commercial
                                        Environment (ACE) Expenditure Plan




                                                                                                 Results: Objective 3
Observation: System defect trends suggest contractor is delivering a quality product.
One measure of system maturation and quality is downward trends in unresolved system defects, particularly defects
that are categorized as material.
Defects are system problems (hardware and software) that require a solution. A defect can be due to a failure to meet
the system specifications, or it can occur because specified requirements are not complete and correct. To facilitate
the management of defects identified in testing, eCP documents defects in program trouble reports (PTRs) and
categorizes them by their severity (impact). The categories are shown in the following table.


Category           Description
Critical (1)       A problem that makes it impossible to execute a test case
Severe (2)         A problem that adversely affects the execution and completion of a test case and for which no
                   acceptable workaround can be found
Moderate (3)       A problem with an acceptable workaround that allows the test case to be executed
Minor (4)          An inconvenience or annoyance that does not affect test case execution


Data provided by eCP on all unresolved defects for ACE increment 1, release 1.1, show a recent downward trend,
which suggests that eCP is delivering a quality product (see following figure). To more thoroughly analyze defect
trends, we requested defect data on critical and severe defects, as the existence of these defect types carry material
consequences. Customs and eCP could not provide documentation on the status of critical and severe defects in time
for this briefing because it did not readily exist. Instead, they stated that there were no critical or severe defects as of
December 16, 2002.




                                                                                                                          38




                                        Page 44                                        GAO-03-406 Customs Service Modernization
               Appendix I
               Customs’ Fourth Automated Commercial
               Environment (ACE) Expenditure Plan




                                                               Results: Objective 3

                     All Unresolved PTRs by Week

    80

    70

    60

    50

    40

    30

    20

    10

       0
            02

            02

            02




            02

            02

            02




            02
              2




              2

              2

              2

              2




              2

              2




              2
            02
           /0




           /0

           /0

           /0

           /0




           /0

           /0




           /0
         2/

         9/

         6/




         6/

         3/

         0/




         4/
         7/
       31




       14

       21

       28

        /5




        /2

        /9




        /7
      /1

      /1

      /2




      /1

      /2

      /3




      /1
     9/




    10




    11

    11




    12
  8/




    9/

    9/

    9/



   10

   10

   10




   11

   11

   11




   12




Source: GAO.




                                                                                       39




               Page 45                                GAO-03-406 Customs Service Modernization
                                                      Appendix I
                                                      Customs’ Fourth Automated Commercial
                                                      Environment (ACE) Expenditure Plan




                                                                                                                                  Results: Objective 3
     Observation: Customs’ oversight of contractor testing does not provide for independent verification and validation
     (IV&V).
Industry test management best practices include steps for effective managerial oversight and control, including
employing independent verification and validation (IV&V).
IV&V involves having an independent organization conduct unbiased reviews of test management processes,
products, and results. The goal is to verify and validate that these meet stated requirements and standards.
Independence is important in ensuring that the IV&V personnel are unaffected by program management pressure.
However, Customs’ managerial oversight and control of ACE testing does not include IV&V. Customs officials stated
that they did not establish an IV&V function because they hired a “world class” contractor to develop ACE and have
                                                                                     29
required that the contractor be a Software Capability Maturity Model (SW-CMM) level 3 organization. Customs
officials also stated that they are relying on the contractor’s capabilities and reputation as a guarantor of ACE
        30
quality, and that CMO (Customs and support contractors) oversees the contractor’s testing.
These actions are not an adequate substitute for IV&V, because the oversight being performed by CMO lacks
independence from program cost and schedule pressures. For large, complex, and inherently risky programs like
ACE, IV&V is a recognized best practice. Without it, Customs is unnecessarily exposing itself to risks that ACE
increments will not perform as intended or be delivered on time and within budget.




29
   The SW-CMM identifies key process areas that are necessary to effectively develop software-intensive systems. Level 3 is the third level of the SW-CMM’s five-
level scale; achieving this level means that an organization has defined and implemented effective software engineering and management processes across all
projects.
30
   In its comments on our report, Customs modified this statement. Customs’ modified statement is reflected in the letter transmitting this briefing.
                                                                                                                                                                    40




                                                      Page 46                                                      GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                           Results: Objective 3
Observation: Potential for ACE infrastructure to support other Department of Homeland Security applications
could affect ACE cost and schedule.
Customs’ current cost and schedule commitments for its ACE program, as previously described in this briefing, are
based on a set of Customs’ enterprise infrastructure and ACE-specific requirements. As with any such program,
changes to scope will affect these commitments. Also, a meaningful understanding of the magnitude of these cost
and schedule effects requires a complete understanding of the scope changes.
Customs is one of several agencies to be merged into the Department of Homeland Security and is to support the
new department’s border and transportation security mission area. Customs has proposed to the Office of
Homeland Security that the ACE infrastructure be used by the various agencies involved in carrying out this
mission. According to Customs, its infrastructure is being designed and developed in a way to make it expandable
and extendable, and thus able to support other border and transportation security applications. However, Customs’
Chief Information Officer told us that Customs has yet to receive an official response from the Office of Homeland
Security on this proposal, and a meaningful understanding of the effect of the new department’s use of the ACE
infrastructure is not yet known, because the detailed engineering analysis needed to understand the associated
program scope changes (e.g., workload sizing, security) has not been done.
As a result, while the potential exists for the cost and schedule commitments defined in the ACE expenditure plan
to change, the magnitude of such changes is not yet known.




                                                                                                                     41




                                      Page 47                                    GAO-03-406 Customs Service Modernization
                                       Appendix I
                                       Customs’ Fourth Automated Commercial
                                       Environment (ACE) Expenditure Plan




                                                                                                       Conclusions
Customs’ fourth expenditure plan satisfies its legislative conditions and is fully consistent with most of our open
recommendations. Nevertheless, Customs has moved slowly in addressing our recommendation for immediately
correcting its CMO human capital weaknesses, thus far only having made statements calling for implementing human
capital management best practices. Compounding this, Customs has not satisfied the intent of our recommendation
for disclosing to its appropriations committees where it stands in putting in place acquisition management best
practices. Moreover, its progress in implementing these practices has been slow, and where it stands today continues
to put ACE at risk.
Both of these areas—people and processes—are fundamental components of an organization’s capability to acquire a
system like ACE. Despite this, Customs has not given either sufficient priority, instead focusing to date on expediting
eCP development activities and expressly compensating for its lagging acquisition capabilities by relying on eCP,
without the benefit of IV&V, to be the guarantor of ACE quality.
Customs’ approach of offsetting weaknesses in its acquisition process and its people management by relying on its
contractor has to date not produced visible and measurable cost, schedule, and quality shortfalls, as evidenced by
reported defect trends and system acceptance test results for ACE increment 1, release 1. However, research has
linked weaknesses in an organization’s ability to acquire software-intensive systems with such shortfalls. Thus, until
Customs fully addresses its weaknesses in its acquisition process and its people management, it will be at risk of
experiencing such shortfalls. This risk is increased by the potential changes in store for ACE should its expanded role
in the Department of Homeland Security be realized. Therefore, it is critical that Customs, as the acquirer of ACE,
treat advancing its acquisition capabilities as a priority matter.




                                                                                                                     42




                                       Page 48                                     GAO-03-406 Customs Service Modernization
                                      Appendix I
                                      Customs’ Fourth Automated Commercial
                                      Environment (ACE) Expenditure Plan




                                                                                               Recommendations
To ensure that Customs has the requisite capability to manage its ACE acquisition, we recommend that the Customs
Service Commissioner designate strengthening CMO human capital and acquisition processes as priority matters. To
this end, we recommend that the Commisssioner direct the Chief Information Officer (CIO) to immediately
  • develop and implement the missing human capital management practices discussed in this briefing, and until
    this is accomplished, report to its appropriations committees quarterly on the progress of its efforts to do so;
  • develop and implement each of the missing SEI SA-CMM practices for the key process areas discussed in this
    briefing, and until this is accomplished, report to its appropriations committees quarterly on the progress of its
    efforts to do so; and
  • establish an IV&V function to assist Customs in overseeing eCP efforts, such as testing.


Additionally, we recommend that the Commissioner take steps, as appropriate in light of Customs’ merger into the
Department of Homeland Security, to have future ACE expenditure plans specifically address any proposals or plans,
whether tentative or approved, for extending and using ACE infrastructure to support other homeland security
applications, including any impact on ACE of such proposals and plans.




                                                                                                                       43




                                      Page 49                                      GAO-03-406 Customs Service Modernization
                                     Appendix I
                                     Customs’ Fourth Automated Commercial
                                     Environment (ACE) Expenditure Plan




                                                                                            Agency Comments
We provided Customs a draft of this briefing and discussed its contents with Customs’ Assistant Commissioner for
Information and Technology (CIO) and with CMO and eCP officials. In its oral comments, Customs agreed with our
findings, conclusions, and all but one recommendation. Regarding our recommendation to employ IV&V, Customs
agreed that it does not currently have an IV&V function or agent. Customs also agreed to take the recommendation
under advisement. However, Customs stated that to date it has satisfied the intent of IV&V through various process
management controls practiced by eCP, such as risk management, metrics collection, and test management, and
through CMO oversight of eCP. We do not dispute the performance of these activities by eCP and CMO. However,
none satisfies the express purpose of IV&V, which is to provide a perspective on project performance independent of
eCP and CMO. In light of CMO’s current acquisition management capabilities, both people and processes, we strongly
believe that because of the magnitude and complexity of ACE, appropriately targeted IV&V activities are a necessary
measure to reduce acquisition management risk.




                                                                                                                 44




                                     Page 50                                    GAO-03-406 Customs Service Modernization
Appendix II

Comments from the U.S. Customs Service                             Appendx
                                                                         Ii




              Page 51       GAO-03-406 Customs Service Modernization
Appendix II
Comments from the U.S. Customs Service




Page 52                                  GAO-03-406 Customs Service Modernization
Appendix II
Comments from the U.S. Customs Service




Page 53                                  GAO-03-406 Customs Service Modernization
Appendix II
Comments from the U.S. Customs Service




Page 54                                  GAO-03-406 Customs Service Modernization
           Appendix II
           Comments from the U.S. Customs Service




(310251)   Page 55                                  GAO-03-406 Customs Service Modernization
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO
                         Mailing Lists” under “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested