Customs Service Modernization: Actions Initiated to Correct ACE Management and Technical Weaknesses

Published by the Government Accountability Office on 1999-05-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States
General Accounting Office
Washington, D.C. 20548

Accounting and Information
Management Division


May 18,1999

The Honorable Phillip M. Crane
Chairman, Subcommittee on Trade
Committee on Ways and Means
House of Representatives

Subject: Customs Service Modernization: Actions Initiated to Correct ACE Management and
         Technical Weaknesses

Dear Mr. Chairman:

The U. S. Customs Service plans to acquire and deploy the Automated Commercial
Environment (ACE) to implement the import processing modernization requirements
specified in the “Customs Modernization and Informed Compliance Act” or “Mod” Act.’ On
April 13, 1999, we testified before your Subcommittee on the actions Customs needed to take
to address the management and technical weaknesses that we identified in our February 1999
report on ACE.’ This letter responds to your office’s subsequent request for information on
Customs’ efforts to date to implement our recommendations. It is based on information that
we received from Customs in preparation for our May 13,1999, testimony before the Senate
Committee on Finance.3

Our February 1999 report described ACE-related management and technical weaknesses in
the following three areas: (1) building ACE without a complete and enforced enterprise
systems architecture, (2) investing in ACE without employing effective investment
management practices, and (3) building ACE without employing software engineering rigor
and discipline. Customs agreed with our findings and has either initiated actions to
implement or has implemented the recommendations that we made to correct the
weaknesses in each of the three areas. Nevertheless, many of the actions taken to date, while
appropriate given the time that has elapsed, are first steps, and much remains to be
accomplished. Following is a brief discussion of the three areas of ACE management and
technical weaknesses and Customs’ efforts to date to correct&em.

’Title VI of the North American Free Trade Agreement Implementation Act, Public Law 103-182,is commonly referred to as the
“Customs Modernization and Informed Compliance Act” or “Mod” Act.

’Customs Sex-we Modernization: Actions Needed to Correct Serious ACE Management and T ethnical Weaknesses (GAO/
T-AI&ID-99-141, April 13,1999) and Cwto ms Service Modernizntion: Serious Management and Twhnical WeaknessesMust Be
Corrected (GAO/ADD99-41, February 26,1999).
3j ,u ,o sStieMo      miza.‘o:                                                              Weaknesses (GAO/
T-AIMD-99-186, May l&1999).

                                                                                 GAO/AIMD-99-198R, Customs Modernization
First, we found that Customs had not been building ACE within the context of a complete and
enforced enterprise systems architecture or “blueprint.” The Clinger-Cohen Act of 19964
requires agency chief information officers to develop and maintain a system architecture. In
addition, the Office of Management and Budget (OMB) issued guidance in 1996 that, among
other things, requires agency IT investments to be consistent with system architectures.5
Architectures are critical for designing and developing large and complex information
systems because they systematically and completely describe an organization’s target
business and technology environments. Without a target architecture to guide and constrain
IT investment, there is no systematic way to preclude either inconsistent system design and
development decisions or the resulting suboptimal performance and added cost associated
with incompatible systems.

In response to recommendations that we first made in May 19986and reiterated in our
February 1999ACE report,’ Customs has been working for the past year to complete its
architecture and to establish the means for enforcing it on projects like ACE. On the basis of
a Customs-provided architecture briefing and demonstration, although some limited work
remains, Customs appears to have satisfied our recommendations to (1) complete the
architecture and (2) institute a process for ensuring that projects like ACE comply with the

Second, we found that Customs was not employing effective investment management
practices on ACE. According to the Clinger-Cohen Act and OMB, when investing in
information technology, organizations should (1) identify and analyze alternative system
solutions, (2) forecast system return on investment (ROI) and invest in the alternative
providing the highest ROI, and (3) manage large investments by breaking them into a series of
increments to ensure that each increment constitutes a wise investment. In the case of ACE,
we found that Customs did not satisfy any of these requirements. For example, Customs’
analysis of ROI was based on unreliable estimates of costs and benefits, did not consider
alternative system solutions and approaches, and was predicated on a “grand design” (i.e.,
non-incremental) approach that has proven to be ineffective on large system investments.

In response to our recommendations in this area, Customs reports that it (1) has twice
revised its estimate of ACE costs and now projects ACE 7-year life-cycle costs to be between
$1.4 billion and $1.8 billion, (2) has redone its analysis of ACE’s cost effectiveness, (3) will
perform cost/benefit and post-implementation analyses on system increments, and (4) will
have these analyses independently validated. These steps are consistent with our
recommendations. However, we cannot comment on the reliability of either the revised cost
estimate or the revised economic analyses because Customs has yet to share the supporting
analytical basis for either with us.

’Public Law 104-106,section 5126,110 Stat. 684 (1996).

5 OMB Memorandum M-97-02,Funding Information Svstems Investmen$, October 25,1996.
li Customs Service Modernizxtion: Architectwe Must Be ComDleteand Enforced to Effectivelv Build and Maintain Svstems
(GAO/AI&ID-98-70,May 5,1998).

 Page 2                                                                        GAO/AIMD-99-198R,Customs Modernization
Third, we found that Customs’processes for developing and acquiring ACE software lacked
engineering discipline and rigor. One measure of such rigor and discipline is the Software
Engineering Institute’s (SEI) capability maturity models.* We evaluated ACE software
processes against SEI’s criteria for a “repeatable” level of software maturity, which is the
second level on SEI’s five-level scale. Customs did not fully satisfy any of these criteria, and
thus its capability to either develop or acquire ACE software is not effective.

In response to our recommendations, Customs has instituted a requirement that all its
software contractors have at least SE1level 2 process capabilities and hired a federally
funded research and development center (FFRDC). The FFRDC is responsible, among other
things, for (1) developing and implementing plans for Customs to achieve SE1level 2 process
maturity, (2) assisting Customs in bringing on an SE1level 3 or higher prime integration
contractor for ACE and other specified system needs, and (3) serving as an independent
verification and validation agent in overseeing and monitoring the prime contractor’s
performance. These are reasonable fust steps to begin addressing our recommendations in
this area.

To Customs’credit, its leadership fully recognizes the seriousness of the management and
technical problems in the three areas we identified, has quickly initiated actions to either
correct or begin correcting the weaknesses, and appears committed to these actions. This
commitment must be sustained for Customs to fully implement all of our recommendations.

We provided a draft copy of this letter to Customs’Assistant Commissioner, Office of
Information and Technology, and Chief Information Officer who agreed with its contents.

We are sending copies of this letter to Representative Sander M. Levin, Ranking Minority
Member, Subcommittee on Trade, House Committee on Ways and Means; Senator William V.
Roth, Jr., Chairman, and Senator Daniel Patrick Moynihan, Ranking Minority Member, Senate
Committee on Finance; Senator Ben Nighthorse Campbell, Chairman, and Senator Herb Kohl,
Ranking Minority Member, Subcommittee on Treasury, General Government, and Civil
Service, Senate Committee on Appropriations; Representative Jim Kolbe, Chairman, and
Representative Steny H. Hoyer, Ranking Minority Member, Subcommittee on Treasury, Postal
Service, and General Government, House Committee on Appropriations; and to other
congressional committees. We are also sending copies of this letter to the Honorable Robert
E. Rubin, Secretary of the Treasury; the,Honorable Raymond W. Kelly, Commissioner of
Customs; the Honorable Jacob J. Lew, Director, Office of Management and Budget; and other
interested agency officials. Copies will also be made available to others upon request. If you

’GAO/AIMD-994, February 26,1999.

’Software Development Capability Maturity Models”‘(SW-CMM”) and Software Acquisition Capability Maturity Model”
(SA-CMM”). Capability Maturity Models”is a service mark of Carnegie Mellon University, and CMM” is registered in the U.S.
Patent and Trademark Office.

Page 3                                                                            GAOIAIMD-99-198R,Customs Modernization
have any questions about this letter, please contact me or Mark Bird, Assistant Director, at
(202) 512-6240.

Sincerely yours,

Randolph C. Hite
Associate Director, Governmentwide
  and Defense Information Systems



 Page 4                                                        GAO/AIMD-99.198R,Customs Modernization
Ordering    Information

The first copy of each GAO report and testimony is free.
Additional  copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent    of Documents, when
necessary. VISA and Mastercard     credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting    Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by caBing (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony.   To receive facsimile copies of the daily Iist or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu wiB provide information          on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:


or visit GAO’s World Wide Web Home Page at:

United States
General Accounting Office
Washington, DE. 20548-0001
                                  I   Permit   No. GlOO
Offkial Business
Penalty for Private   Use $300
Address Correction    Requested