United States General Accounting Office Washington, D.C. 20548 Accounting and Information Management Division B-282756.1 May 18,1999 The Honorable Phillip M. Crane Chairman, Subcommittee on Trade Committee on Ways and Means House of Representatives Subject: Customs Service Modernization: Actions Initiated to Correct ACE Management and Technical Weaknesses Dear Mr. Chairman: The U. S. Customs Service plans to acquire and deploy the Automated Commercial Environment (ACE) to implement the import processing modernization requirements specified in the “Customs Modernization and Informed Compliance Act” or “Mod” Act.’ On April 13, 1999, we testified before your Subcommittee on the actions Customs needed to take to address the management and technical weaknesses that we identified in our February 1999 report on ACE.’ This letter responds to your office’s subsequent request for information on Customs’ efforts to date to implement our recommendations. It is based on information that we received from Customs in preparation for our May 13,1999, testimony before the Senate Committee on Finance.3 Our February 1999 report described ACE-related management and technical weaknesses in the following three areas: (1) building ACE without a complete and enforced enterprise systems architecture, (2) investing in ACE without employing effective investment management practices, and (3) building ACE without employing software engineering rigor and discipline. Customs agreed with our findings and has either initiated actions to implement or has implemented the recommendations that we made to correct the weaknesses in each of the three areas. Nevertheless, many of the actions taken to date, while appropriate given the time that has elapsed, are first steps, and much remains to be accomplished. Following is a brief discussion of the three areas of ACE management and technical weaknesses and Customs’ efforts to date to correct&em. ’Title VI of the North American Free Trade Agreement Implementation Act, Public Law 103-182,is commonly referred to as the “Customs Modernization and Informed Compliance Act” or “Mod” Act. ’Customs Sex-we Modernization: Actions Needed to Correct Serious ACE Management and T ethnical Weaknesses (GAO/ T-AI&ID-99-141, April 13,1999) and Cwto ms Service Modernizntion: Serious Management and Twhnical WeaknessesMust Be Corrected (GAO/ADD99-41, February 26,1999). 3j ,u ,o sStieMo miza.‘o: Weaknesses (GAO/ T-AIMD-99-186, May l&1999). GAO/AIMD-99-198R, Customs Modernization First, we found that Customs had not been building ACE within the context of a complete and enforced enterprise systems architecture or “blueprint.” The Clinger-Cohen Act of 19964 requires agency chief information officers to develop and maintain a system architecture. In addition, the Office of Management and Budget (OMB) issued guidance in 1996 that, among other things, requires agency IT investments to be consistent with system architectures.5 Architectures are critical for designing and developing large and complex information systems because they systematically and completely describe an organization’s target business and technology environments. Without a target architecture to guide and constrain IT investment, there is no systematic way to preclude either inconsistent system design and development decisions or the resulting suboptimal performance and added cost associated with incompatible systems. In response to recommendations that we first made in May 19986and reiterated in our February 1999ACE report,’ Customs has been working for the past year to complete its architecture and to establish the means for enforcing it on projects like ACE. On the basis of a Customs-provided architecture briefing and demonstration, although some limited work remains, Customs appears to have satisfied our recommendations to (1) complete the architecture and (2) institute a process for ensuring that projects like ACE comply with the architecture. Second, we found that Customs was not employing effective investment management practices on ACE. According to the Clinger-Cohen Act and OMB, when investing in information technology, organizations should (1) identify and analyze alternative system solutions, (2) forecast system return on investment (ROI) and invest in the alternative providing the highest ROI, and (3) manage large investments by breaking them into a series of increments to ensure that each increment constitutes a wise investment. In the case of ACE, we found that Customs did not satisfy any of these requirements. For example, Customs’ analysis of ROI was based on unreliable estimates of costs and benefits, did not consider alternative system solutions and approaches, and was predicated on a “grand design” (i.e., non-incremental) approach that has proven to be ineffective on large system investments. In response to our recommendations in this area, Customs reports that it (1) has twice revised its estimate of ACE costs and now projects ACE 7-year life-cycle costs to be between $1.4 billion and $1.8 billion, (2) has redone its analysis of ACE’s cost effectiveness, (3) will perform cost/benefit and post-implementation analyses on system increments, and (4) will have these analyses independently validated. These steps are consistent with our recommendations. However, we cannot comment on the reliability of either the revised cost estimate or the revised economic analyses because Customs has yet to share the supporting analytical basis for either with us. ’Public Law 104-106,section 5126,110 Stat. 684 (1996). 5 OMB Memorandum M-97-02,Funding Information Svstems Investmen$, October 25,1996. li Customs Service Modernizxtion: Architectwe Must Be ComDleteand Enforced to Effectivelv Build and Maintain Svstems (GAO/AI&ID-98-70,May 5,1998). Page 2 GAO/AIMD-99-198R,Customs Modernization Third, we found that Customs’processes for developing and acquiring ACE software lacked engineering discipline and rigor. One measure of such rigor and discipline is the Software Engineering Institute’s (SEI) capability maturity models.* We evaluated ACE software processes against SEI’s criteria for a “repeatable” level of software maturity, which is the second level on SEI’s five-level scale. Customs did not fully satisfy any of these criteria, and thus its capability to either develop or acquire ACE software is not effective. In response to our recommendations, Customs has instituted a requirement that all its software contractors have at least SE1level 2 process capabilities and hired a federally funded research and development center (FFRDC). The FFRDC is responsible, among other things, for (1) developing and implementing plans for Customs to achieve SE1level 2 process maturity, (2) assisting Customs in bringing on an SE1level 3 or higher prime integration contractor for ACE and other specified system needs, and (3) serving as an independent verification and validation agent in overseeing and monitoring the prime contractor’s performance. These are reasonable fust steps to begin addressing our recommendations in this area. To Customs’credit, its leadership fully recognizes the seriousness of the management and technical problems in the three areas we identified, has quickly initiated actions to either correct or begin correcting the weaknesses, and appears committed to these actions. This commitment must be sustained for Customs to fully implement all of our recommendations. We provided a draft copy of this letter to Customs’Assistant Commissioner, Office of Information and Technology, and Chief Information Officer who agreed with its contents. We are sending copies of this letter to Representative Sander M. Levin, Ranking Minority Member, Subcommittee on Trade, House Committee on Ways and Means; Senator William V. Roth, Jr., Chairman, and Senator Daniel Patrick Moynihan, Ranking Minority Member, Senate Committee on Finance; Senator Ben Nighthorse Campbell, Chairman, and Senator Herb Kohl, Ranking Minority Member, Subcommittee on Treasury, General Government, and Civil Service, Senate Committee on Appropriations; Representative Jim Kolbe, Chairman, and Representative Steny H. Hoyer, Ranking Minority Member, Subcommittee on Treasury, Postal Service, and General Government, House Committee on Appropriations; and to other congressional committees. We are also sending copies of this letter to the Honorable Robert E. Rubin, Secretary of the Treasury; the,Honorable Raymond W. Kelly, Commissioner of Customs; the Honorable Jacob J. Lew, Director, Office of Management and Budget; and other interested agency officials. Copies will also be made available to others upon request. If you ’GAO/AIMD-994, February 26,1999. ’Software Development Capability Maturity Models”‘(SW-CMM”) and Software Acquisition Capability Maturity Model” (SA-CMM”). Capability Maturity Models”is a service mark of Carnegie Mellon University, and CMM” is registered in the U.S. Patent and Trademark Office. Page 3 GAOIAIMD-99-198R,Customs Modernization have any questions about this letter, please contact me or Mark Bird, Assistant Director, at (202) 512-6240. Sincerely yours, Randolph C. Hite Associate Director, Governmentwide and Defense Information Systems :- ‘. (511156) Page 4 GAO/AIMD-99.198R,Customs Modernization Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and Mastercard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by caBing (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily Iist or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu wiB provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with “info” in the body to: info%vww.gao.gov or visit GAO’s World Wide Web Home Page at: httpJlwww.gao.gov United States General Accounting Office Washington, DE. 20548-0001 I Permit No. GlOO Offkial Business Penalty for Private Use $300 Address Correction Requested
Customs Service Modernization: Actions Initiated to Correct ACE Management and Technical Weaknesses
Published by the Government Accountability Office on 1999-05-18.
Below is a raw (and likely hideous) rendition of the original report. (PDF)