DOD Business Systems Modernization: Longstanding Management and Oversight Weaknesses Continue to Put Investments at Risk

Published by the Government Accountability Office on 2003-03-31.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          United States General Accounting Office

GAO                       Testimony
                          Before the House Subcommittee on National
                          Security, Emerging Threats, and International
                          Relations, and Subcommittee on Technology,
                          Information Policy, Intergovernmental Relations and
                          Census, Government Reform Committee
For Release on Delivery
Expected at 1 p.m.
Monday, March 31, 2003
                          DOD BUSINESS SYSTEMS
                          Longstanding Management
                          and Oversight Weaknesses
                          Continue to Put
                          Investments at Risk
                          Statement of
                          Randolph C. Hite, Director
                          Information Technology Architecture and Systems Issues

                          Gregory D. Kutz, Director
                          Financial Management and Assurance

This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.
                                                 March 31, 2003

                                                 DOD BUSINESS SYSTEMS
Highlights of GAO-03-553T, a testimony to
House Subcommittee on National Security,
Emerging Threats, and International
                                                 Longstanding Management and
Relations, and Subcommittee on
Technology, Information Policy,
                                                 Oversight Weaknesses Continue to Put
Intergovernmental Relations and Census,
Government Reform Committee                      Investments at Risk

The Department of Defense’s                      As of October 2002, DOD reported that its business systems environment
(DOD) management of its business                 consisted of 1,731 systems and system acquisition projects spanning about 18
systems modernization program                    functional areas. This environment is the product of unrelated, stovepiped
has been an area of longstanding                 initiatives supporting nonstandard, duplicative business operations across
concern to Congress and one that                 DOD components. For fiscal year 2003, about $18 billion of DOD’s IT funding
GAO has designated as high risk                  relates to operating, maintaining, and modernizing these nonintegrated
since 1995.                                      systems. To DOD’s credit, it recognizes the need to modernize, eliminating as
Because of this concern, GAO was                 many of these systems as possible.
requested to testify on (1) DOD’s
current inventory of existing and                The future of DOD’s business systems modernization is fraught with risk
new business systems and the                     because of longstanding and pervasive modernization weaknesses, three of
amount of funding devoted to this                which are discussed below. GAO’s report on four DFAS systems highlights
inventory; (2) DOD’s modernization               some of these weaknesses, and GAO’s prior reports have identified the
management capabilities, including               others. DOD has stated its commitment to addressing each and has efforts
weaknesses and DOD’s efforts to                  under way that are intended to do so.
address them; and (3) GAO’s
collective recommendations for
correcting these weaknesses and                  Lack of departmentwide enterprise architecture: DOD does not yet have an
minimizing DOD’s exposure to risk                architecture, or blueprint, to guide and constrain its business system
until they are corrected.                        investments across the department. Nevertheless, DOD continues to spend
                                                 billions of dollars on new and modified systems based the parochial needs
In developing this testimony, GAO                and strategic direction of its component organizations. This will continue to
drew from its previously issued
                                                 result in systems that are duplicative, are not integrated, are unnecessarily
reports on DOD’s business systems
modernization efforts, including                 costly to maintain and interface, and will not adequately address
one released today on four key                   longstanding financial management problems.
Defense Finance and Accounting
Service (DFAS) projects.                         Lack of effective investment management: DOD does not yet have an
                                                 effective approach to consistently selecting and controlling its investments
                                                 as a portfolio of competing department options and within the context of an
                                                 enterprise architecture. DOD is also not ensuring that it invests in each
GAO has previously made a series of              system incrementally and on the basis of reliable economic justification. For
recommendations related to putting
in place (1) an enterprise architecture
                                                 example, for the four DFAS projects, DOD spent millions of dollars without
to guide and constrain system                    knowing whether the projects would produce value commensurate with
investments; (2) an investment                   costs and risks. Thus far, this has resulted in the termination of one of the
management structure to ensure that              projects after about $126 million and 7 years of effort was spent.
systems are aligned with the
architecture and economically                    Lack of effective oversight: DOD has not consistently overseen its system
justified and approved on an                     projects to ensure that they are delivering promised system capabilities and
incremental basis; (3) effective                 benefits on time and within budget. For example, for the four DFAS projects,
oversight to ensure that project                 oversight responsibility is shared by the DOD Comptroller, DFAS, and the
commitments are met; and (4) limited             DOD chief information officer. However, these oversight authorities have
investment spending until these                  largely allowed the four to proceed unabated, even though each was
recommendations are implemented.
                                                 experiencing significant cost increases, schedule delays, and/or capability
www.gao.gov/cgi-bin/getrpt?GAO-03-553T.          and scope reductions and none were supported by adequate economic
                                                 justification. As a result, DOD invested approximately $316 million in four
To view the full report, including the scope
and methodology, click on the link above.        projects that may not resolve the very financial management weaknesses
For more information, contact Randolph C. Hite   that they were initiated to address.
at (202) 512-5555 or hiter@gao.gov or Gregory
D. Kutz at (202) 9505 or kutzg@gao.gov.
Messrs. Chairmen and Ranking Members of the Subcommittees:

We are pleased to be here today to discuss the Department of Defense’s
(DOD) management of its business systems1 modernization program, an
area of longstanding concern to the Congress, and one that we first
designated as a high risk program in 19952 and continue to do so today.3 As
we have said,4 DOD’s existing systems cannot provide reliable financial
data to support informed decisionmaking and promote accountability,
thus leaving DOD at a high risk of fraud, waste, and abuse. In addition, we
have said that DOD’s business systems modernization will remain at risk
until the department has implemented proven modernization management
controls that are embodied in the Clinger-Cohen Act, federal guidance, and
commercial best practices. These controls include investing in new and
existing systems within the context of a departmentwide modernization
blueprint, commonly called an enterprise architecture; investing in these
systems in an incremental or modular fashion, and only when they can be
economically justified on the basis of costs, benefits, and risks; and
overseeing these system investments to ensure that they are delivering
promised system capabilities and benefits on time and within budget.

Last year, your hearing5 brought additional attention and focus to DOD’s
business systems modernization program. In our testimony at that hearing,
we highlighted the department’s modernization management weaknesses,
and the department testified that it was committed to addressing each.
Since then, DOD has begun a number of efforts to follow through on its
stated commitment. For example, it plans to issue the first version of its

 Business systems include those that are used to support civilian personnel, finance, health,
logistics, military personnel, procurement, and transportation.
 U.S. General Accounting Office, High-Risk Series: An Overview, GAO-HR-95-263
(Washington, D.C.: February 1995).
U.S. General Accounting Office, High-Risk Series: An Update, GAO-03-119 (Washington,
D.C.: January 2003).
 U.S. General Accounting Office, DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458
(Washington, D.C.: Feb. 28, 2003) and Information Technology: Architecture Needed to
Guide Modernization of DOD’s Financial Operations, GAO-01-525 (Washington, D.C.:
May 17, 2001).
 U.S. General Accounting Office, DOD Financial Management: Important Steps
Underway But Reform Will Require a Long-term Commitment, GAO-02-784T
(Washington, D. C.: June 4, 2002).

Page 1                                       GAO-03-553T Business Systems Modernization
enterprise architecture in May 2003, it is creating a new investment
governance and oversight approach, and it is revising its system
acquisition guidance. We view each of these as positive steps. However,
the fact remains that today, with but isolated exceptions, DOD’s
management and oversight of its hundreds of new and existing system
investments is largely unchanged from where it was last year. As a result,
the $18 billion that DOD has designated for business systems in fiscal year
2003 continues to be at risk. In particular, our report that you are releasing
today shows that for four key accounting system projects, DOD oversight
has been limited and has allowed hundreds of millions of dollars to be
spent without adequate economic justification.6 Thus far, this has resulted
in one of these systems being terminated after about $126 million and 7
years of effort has been spent.

As you requested, our testimony today discusses (1) DOD’s current
business systems environment, including a profile of (a) the number and
types of systems that have proliferated over the years and (b) the
enormous amounts of funding that are being spent to operate and maintain
existing systems and to introduce new systems; (2) DOD’s institutional
modernization management weaknesses, including specific system
investments that are at risk because of them, such as the above-mentioned
accounting systems, and (3) a framework for overcoming these
modernization management weaknesses and limiting DOD’s exposure to
investment risk until they are resolved, which is based on our open
recommendations to the department.

In developing this testimony, we drew from our previously issued reports
on DOD’s business systems modernization efforts, as well as the report
being released today.

 U.S. General Accounting Office, DOD Business Systems Modernization: Continued
Investment in Key Accounting Systems Needs to be Justified, GAO-03-465 (Washington,
D.C.: Mar. 28, 2003).

Page 2                                   GAO-03-553T Business Systems Modernization
                       As part of its ongoing business systems modernization program, and
DOD Is Investing       consistent with our past recommendation,7 DOD has created an inventory
Billions of Dollars    of its existing and new business system investments. As of October 2002,
                       DOD reported that this inventory consisted of 1,7318 systems and system
Annually to Operate,   acquisition projects across DOD’s functional areas. In particular, DOD
Maintain, and          reported that it had 374 separate systems to support its civilian and
                       military personnel function, 335 systems to perform finance and
Modernize Its          accounting functions, and 221 systems that support inventory
Amalgamation of        management. Table 1 presents the composition of DOD business systems
Business Systems       by functional area.

                        U.S. General Accounting Office, Financial Management: DOD Improvement Plan Needs
                       Strategic Focus, GAO-01-764 (Washington, D.C.: Aug. 17, 2001).
                        DOD continues to refine its inventory of systems. More recent data indicate that the total
                       number of systems is 2,114.

                       Page 3                                       GAO-03-553T Business Systems Modernization
Table 1: Reported DOD Business Systems by Functional Area

    Functional area                                                         Army         Navy        Air Force         DFAS          DLA           Other      Total
    Personnel                                                                266          49              13             19            0             27         374
    Finance and accounting                                                    79          61              27            131            9             28         335
    Management information                                                   156          40              50             14            4             46         310
    Inventory                                                                 98          53              40              7            7             17         222
    Acquisition                                                               18          10              22              0            5             19          74
    Budget formulation                                                        25          18              10              5            0             10          68
    Cost                                                                      19          29              8               0            1             4           61
    Logistics                                                                 12           6              22              3            7             5           55
    National defense property management                                      5           12              25              1            2             1           46
    Travel                                                                    9           13              3               2            0             5           32
    Real property management                                                  17           4              6               0            0             1           28
    Time and attendance                                                       3           14              2               2            3             1           25
    Budget execution                                                          6            4              2               7            0             3           22
    Personal property management                                              3            7              7               0            0             4           21
    Procurement                                                               7            5              1               0            3             4           20
    Vendor payment                                                            3            3              1               7            0             4           18
    Transportation                                                            5            1              4               0            0             2           12
    Other functions combined                                                  12           7              6               3            0             9           37
    Total                                                                    743          336            249            201            41           190        1,760a

Source: DOD Business Modernization Systems Integration Office.
                                                                  There are 29 reported duplications within the DOD inventory (e.g., systems shown in multiple
                                                                 functional areas). Taking this duplication into account provides the reported 1,731 business systems.

                                                                 Note: More recent DOD data indicate that the number of systems is 2,114.

                                                                 As we have previously reported,9 this systems environment is not the
                                                                 result of a systematic and coordinated departmentwide strategy, but rather
                                                                 is the product of unrelated, stovepiped initiatives to support a set of
                                                                 business operations that are nonstandard and duplicative across DOD
                                                                 components. Consequently, DOD’s amalgamation of systems is
                                                                 characterized by (1) multiple systems performing the same tasks; (2) the
                                                                 same data stored in multiple systems; (3) manual data entry and reentry


                                                                 Page 4                                           GAO-03-553T Business Systems Modernization
    into multiple systems; and (4) extensive data translations and interfaces,
    each of which increases costs and limits data integrity. Further, as we have
    reported, these systems do not produce reliable financial data to support
    managerial decisionmaking and ensure accountability. To the
    department’s credit, it recognizes the need to eliminate as many systems
    as possible and integrate and standardize those that remain. In fact, three
    of the four Defense Finance and Accounting Service (DFAS) projects that
    are the subject of the report being released today were collectively
    intended to reduce or eliminate all or part of 17 different systems that
    perform similar functions. For example,

•   the Defense Procurement Payment System (DPPS) was intended to
    consolidate eight contract and vendor pay systems;
•   the Defense Departmental Reporting System (DDRS) is intended to reduce
    the number of departmental financial reporting systems from seven to one;
•   the Defense Standard Disbursing System (DSDS) is intended to eliminate
    four different disbursing systems.

    The fourth system, the DFAS Corporate Database/Corporate Warehouse
    (DCD/DCW),10 is intended to serve as the single DFAS data store, meaning
    it would contain all DOD financial information required by DFAS and be
    the central point for all shared data within DFAS.

    For fiscal year 2003, DOD has requested approximately $26 billion in IT
    funding to support a wide range of military operations and business
    functions. This $26 billion is spread across the military services and
    defense agencies—each receiving its own allocation of IT funding. The $26
    billion supports three categories of IT—business systems, business
    systems infrastructure, and national security systems—the first two of
    which comprise the earlier cited 1,731 new and existing business systems

    At last year’s hearing, DOD was asked about the makeup of its $26 billion
    in IT funding, including what amounts relate to business systems and
    related infrastructure, at which time answers were unavailable. As we are
    providing in the report being released today and as shown in figure 1,
    approximately $18 billion—about $5.2 billion for business systems and

     Originally, these were two separate projects, the DFAS Corporate Database and
    Corporate Warehouse.

    Page 5                                    GAO-03-553T Business Systems Modernization
$12.8 billion for business systems infrastructure—relates to the operation,
maintenance, and modernization of the 1,731 business systems that DOD
reported having in October 2002. Figure 2 provides the allocation of DOD’s
business systems modernization budget for fiscal year 2003 budget by

Figure 1: Allocation of DOD’s Fiscal Year 2003 Information Technology (IT) Budget

Page 6                                 GAO-03-553T Business Systems Modernization
Figure 2: Proposed Allocation of DOD’s Fiscal Year 2003 Business Systems
Modernization Budget by Component (dollars in billions)

However, recognizing the need to modernize and making funds available
are not sufficient for improving DOD’s current systems environment. Our
research of successful modernization programs in public and private-
sector organizations, as well as our reviews of these programs in various
federal agencies, has identified a number of IT disciplines that are
necessary for successful modernization. These disciplines include having
and implementing (1) an enterprise architecture to guide and constrain
systems investments; (2) an investment management process to ensure
that systems are invested in incrementally, are aligned with the enterprise
architecture, and are justified on the basis of cost, benefits, and risks; and
(3) a project oversight process to ensure that project commitments are
being met and that needed corrective action is taken. These
institutionalized disciplines have been long missing at DOD, and their
absence is a primary reason for the system environment described above.

Page 7                                GAO-03-553T Business Systems Modernization
                          The future of DOD’s business systems modernization is fraught with risk,
Key Modernization         in part because of longstanding and pervasive modernization management
Management                weaknesses. As we have reported, these weaknesses include (1) lack of an
                          enterprise architecture; (2) inadequate institutional and project-level
Weaknesses Continue,      investment management processes; and (3) limited oversight of projects’
But DOD Plans to          delivery of promised system capabilities and benefits on time and within
                          budget. To DOD’s credit, it recognizes the need to address each of these
Correct Them              weaknesses and has committed to doing so.

DOD Is Developing, But    Effectively managing a large and complex endeavor requires, among other
Still Is Without, a       things, a well-defined and enforced blueprint for operational and
Departmentwide            technological change, commonly referred to as an enterprise architecture.
                          Developing, maintaining, and using architectures is a leading practice in
Enterprise Architecture   engineering both individual systems and entire enterprises. Government-
                          wide requirements for having and using architectures to guide and
                          constrain IT investment decisionmaking are also addressed in federal law
                          and guidance.11 Our experience has shown that attempting a major systems
                          modernization program without a complete and enforceable enterprise
                          architecture results in systems that are duplicative, are not well integrated,
                          are unnecessarily costly to maintain and interface, do not ensure basic
                          financial accountability, and do not effectively optimize mission

                          In May 2001,13 we reported that DOD had neither an enterprise architecture
                          for its financial and financial-related business operations nor the
                          management structure, processes, and controls in place to effectively
                          develop and implement one. Further, we stated that DOD’s plans to
                          continue spending billions of dollars on new and modified systems
                          independently from one another, and outside the context of a

                            Clinger-Cohen Act of 1996, P.L. 104-106; Office of Management and Budget Circular A-130,
                          Management of Federal Information Resources (Nov. 30, 2000); A Practical Guide to
                          Federal Enterprise Architectures, Version 1.0, Chief Information Officers Council
                          (February 2001); and Federal Enterprise Architecture Framework, Version 1.1, Chief
                          Information Officers Council (September 1999).
                           U.S. General Accounting Office, Air Traffic Control: Complete and Enforced
                          Architecture Needed for FAA Systems Modernization, GAO/AIMD-97-30 (Washington,
                          D.C.: Feb. 3, 1997) and Customs Service Modernization: Architecture Must Be Complete
                          and Enforced to Effectively Build and Maintain Systems, GAO/AIMD-98-70 (Washington,
                          D.C.: May 5, 1998).

                          Page 8                                     GAO-03-553T Business Systems Modernization
departmental modernization blueprint, would result in more systems that
are duplicative, noninteroperable, and unnecessarily costly to maintain
and interface; moreover, they would not address longstanding financial
management problems. To assist the department, we provided a set of
recommendations on how DOD should approach developing its enterprise

In September 2002, the Secretary of Defense designated improving
financial management operations (including such business areas as
logistics, acquisition, and personnel management) as one of the
department’s top 10 priorities. In addition, the Secretary established a
program to develop an enterprise architecture, and DOD plans to have the
architecture developed by May 2003. Subsequently, the National Defense
Authorization Act for Fiscal Year 2003 directed DOD to develop, by May 1,
2003, an enterprise architecture, including a transition plan for its
implementation.14 The act also defined the scope and content of the
enterprise architecture and directed us to submit to congressional defense
committees an assessment of DOD’s actions to develop the architecture
and transition plan no later than 60 days after their approval. Finally, the
act prohibited DOD from obligating more than $1 million on any financial
systems improvement until the DOD comptroller makes a determination
regarding the necessity or suitability of such an investment.

In our February 2003 report15 on DOD enterprise architecture efforts, we
stated our support for the Secretary’s decision to develop the architecture
and recognized that DOD’s architecture plans were challenging and
ambitious. However, we also stated that despite taking a number of
positive steps toward its architecture goals, such as establishing a program
office responsible for managing the enterprise architecture, the
department had yet to implement several key recommendations and
certain leading practices for developing and implementing architectures.
For example, DOD had yet to (1) establish the requisite architecture
development governance structure needed to ensure that ownership of
and accountability for the architecture is vested with senior leaders across
the department; (2) develop and implement a strategy to effectively
communicate the purpose and scope, approach to, and roles and
responsibilities of stakeholders in developing the enterprise architecture;
and (3) fully define and implement an independent quality assurance

 Section 1004 of Public Law 107-314.

Page 9                                 GAO-03-553T Business Systems Modernization
                           process. We concluded that not implementing these recommendations and
                           practices increased DOD’s risk of developing an architecture that would
                           be limited in scope, would be resisted by those responsible for
                           implementing it, and would not support effective systems modernization.
                           To assist the department, we made additional recommendations with
                           which DOD agreed. We plan to continue reviewing DOD’s efforts to
                           develop and implement this architecture pursuant to our mandate under
                           the fiscal year 2003 defense authorization act.

DOD Has Yet to Implement   The Clinger-Cohen Act, federal guidance, and recognized best practices
Effective Investment       provide a framework for organizations to follow to effectively manage
Management Processes       their IT investments. Collectively, this framework addresses IT investment
                           management at the institutional or corporate level, as well as the
                           individual project or system level. The former involves having a single,
                           corporate approach governing how the organization’s portfolio of IT
                           investments is selected, controlled, and evaluated across its various
                           components, including assuring that each investment is aligned with the
                           organization’s enterprise architecture. The latter involves having a
                           system/project-specific investment approach that provides for making
                           investment decisions incrementally and ensuring that these decisions are
                           economically justified on the basis of current and credible analyses.

                           Corporate investment management approach: DOD has yet to establish
                           and implement an effective departmentwide approach to managing its
                           business systems investment portfolio. In May 2001,16 we reported that
                           DOD did not have a departmentwide IT investment management process
                           through which to assure that its enterprise architecture, once developed,
                           could be effectively implemented. We therefore recommended that DOD
                           establish a system investment selection and control process that treats
                           compliance with the architecture as an explicit condition to meet at key
                           decision points in the system’s life cycle and that can be waived only if
                           justified by compelling written analysis.17

                           Subsequently, in February 2003, we reported that DOD had not yet
                           established the necessary departmental investment management structure

                             The Defense Appropriation Act for Fiscal Year 2003, P.L. 107-248, prohibits the use of
                           funds appropriated by that act for a mission-critical or mission-essential financial
                           management IT system that is not registered with the chief information officer of DOD.

                           Page 10                                     GAO-03-553T Business Systems Modernization
and process controls needed to adequately align ongoing investments with
its architectural goals and direction.18 Instead, the department continued to
allow its component organizations to make their own parochial investment
decisions, following different approaches and criteria. In particular, DOD
had not established and applied common investment criteria to its ongoing
IT system projects using a hierarchy of investment review and funding
decisionmaking bodies, each composed of representatives from across the
department. DOD also had not yet conducted a comprehensive review of
its ongoing IT investments to ensure that they were consistent with its
architecture development efforts. We concluded that until it takes these
steps, DOD will likely continue to lack effective control over the billions of
dollars it is currently spending on IT projects. To address this, we
recommended that DOD create a departmentwide investment review
board with the responsibility and authority to (1) select and control all
DOD financial management investments and (2) ensure that its investment
decisions treat compliance with the financial management enterprise
architecture as an explicit condition for investment approval that can be
waived only if justified by a compelling written analysis. DOD concurred
with our recommendations and is taking steps to address them.

Project/system-specific investment management: DOD has yet to ensure
that its investments in all individual systems or projects are economically
justified and that it is investing in each incrementally. In particular, none
of the four DFAS projects addressed in the report being issued today had
current and reliable economic justifications to demonstrate that they
would produce value commensurate with the costs and risks being
incurred. For example, we found that although DCD was initiated to
contain all DOD financial data required by DFAS systems, planned DCD
capabilities had since been drastically reduced. Despite this, DFAS
planned to continue investing in DCD/DCW without having an economic
justification showing whether its revised plans were cost effective.
Moreover, DOD planned to continue investing in the three other projects
even though none had current economic analyses that reflected material
changes to costs, schedules, and/or expected benefits since the projects’
inception. For example, the economic analysis for DSDS had not been
updated to reflect material changes in the project, such as changing the
date for full operational capability from February 2003 to December
2005—a schedule change of almost 3 years that affected delivery of
promised benefits. Similarly, the DPPS economic analysis had not been


Page 11                               GAO-03-553T Business Systems Modernization
updated to recognize an estimated cost increase of $274 million and
schedule slip of almost 4 years. After recently reviewing this project’s
change in circumstances, the DOD Comptroller terminated DPPS after 7
years of effort and an investment of over $126 million, citing poor program
performance and increasing costs. Table 2 highlights the four projects’
estimated cost increases and schedule delays.

Table 2: Reported Cost Increases and Schedule Delays for the Four Pojects (Dollars
in Millions)

     System         Original       Current            Original planned date     Current planned date
                    cost           cost               (fiscal year) of full     (fiscal year) of full
                    estimate       estimate           operational capability    operational capability
     DCD/DCWb       $229           $270               2001                      2005
     DPPS           $278           $552               2002                      2006
     DDRS           $ 52           $170               1999                      2004
     DSDS           $151           $151               2003                      2006

Source: GAO, based on information provided by DFAS.
Full operational capability means the system is deployed and operating at all intended locations.
 When DFAS initiated DCW in July 2000, a full operational capability date was not established. The
current full operational capability date applies to both DCD and DCW since they were combined into
one program in November 2000.
 DSDS began in 1997; however, a cost estimate was not developed until September 2000 and this
estimate has not been updated.

Our work on other DOD projects has shown a similar absence of current
and reliable economic justification for further system investment. For
example, we reported that DOD’s ongoing and planned investment in its
Standard Procurement System (SPS)19 was based on an outdated and
unreliable economic analysis, and even this flawed analysis did not show
that the system was cost beneficial, as defined. As a result, we
recommended that investment in future releases or major enhancements
to the system be made conditional on the department’s first demonstrating
that the system was producing benefits that exceeded costs and that
future investment decisions be made on the basis of complete and reliable
economic justifications. DOD is currently in the process of addressing this

  SPS is intended to replace 76 existing procurement systems with a single departmentwide
system to more effectively support divergent contracting processes and procedures across
its component organizations.

Page 12                                                      GAO-03-553T Business Systems Modernization
                            Beyond not having current and reliable economic analyses for its projects,
                            DOD has yet to adopt an incremental approach to economically justifying
                            and investing in all system projects. For example, we have reported that
                            although DOD had divided its multiyear, billion-dollar SPS project into a
                            series of incremental releases, it had not treated each of these increments
                            as a separate investment decision.20 Such an incremental approach to
                            system investment helps to prevent discovering too late that a given
                            project is not cost beneficial. However, rather than adopt an incremental
                            approach to SPS investment management, the department chose to treat
                            investment in SPS as one, monolithic investment decision, justified by a
                            single, all-or-nothing economic analysis. This approach to investing in
                            large systems, like SPS, has proven ineffective in other federal agencies,
                            resulting in huge sums being invested in systems that do not provide
                            commensurate value, and thus has been abandoned by successful

                            We also recently reported that while DOD’s Composite Health Care
                            System II had been structured into a series of seven increments (releases),
                            the department had not treated the releases to date as separate investment
                            decisions supported by incremental economic justification.21 In response
                            to our recommendations, DOD committed to changing its strategy for
                            future releases to include economically justifying each release before
                            investing in and verifying each release’s benefits and costs after

Effective Oversight of IT   The Clinger-Cohen Act of 1996 and federal guidance22 emphasize the need
Projects Remains an         to ensure that IT projects are being implemented at acceptable costs and
Unanswered Challenge        within reasonable and expected timeframes and that they are contributing
                            to tangible, observable improvements in mission performance (that is, that

                              U.S. General Accounting Office, DOD Systems Modernization: Continued Investment in
                            Standard Procurement System Has Not Been Justified, GAO-01-682 (Washington, D.C.:
                            July 31, 2001) and DOD’s Standard Procurement System: Continued Investment Has Yet
                            to Be Justified, GAO-02-392T (Washington, D.C.: Feb. 7, 2002).
                             U.S. General Accounting Office, Information Technology: Greater Use of Best Practices
                            Can Reduce Risks in Acquiring Defense Health Care System, GAO-02-345 (Washington,
                            D.C.: Sept. 26, 2002).
                             Clinger-Cohen Act of 1996, Public Law 104-106; Office of Management and Budget (OMB)
                            Circular A-130 (Nov. 30, 2000); U.S. General Accounting Office, Information Technology
                            Investment Management: A Framework for Assessing and Improving Process Maturity
                            (Exposure Draft) GAO/AIMD-10.1.23 (Washington, D.C.: May 2000).

                            Page 13                                   GAO-03-553T Business Systems Modernization
projects are meeting the cost, schedule, and performance commitments
upon which their approval was justified). They also emphasize the need to
regularly determine each project’s progress toward expectations and
commitments and to take appropriate action to address deviations.

Our work on specific DOD projects has shown that such oversight does
not always occur, a multi-example case in point being the four DFAS
accounting system projects that are the subject of our report being
released today.23 For these four projects, oversight responsibility was
shared by the DOD comptroller, DFAS, and the DOD chief information
officer (CIO). However, these oversight authorities have not ensured, in
each case, that the requisite analytical basis for making informed
investment decisions was prepared. Moreover, they have not regularly
monitored system progress toward expectations so that timely action
could have been taken to correct deviations, even though each case had
experienced significant cost increases and schedule delays (see table 2).
Their respective oversight activities are summarized below:

DOD Comptroller—Oversight responsibility for DFAS activities, including
system investments, rests with the DOD Comptroller. However, DOD
Comptroller officials were not only unaware of cost increases and
schedule delays on these four projects, they also told us that they do not
review DFAS system investments to ensure that they are meeting cost,
schedule, and performance commitments because this is DFAS’s

DFAS—This DOD agency has established an investment committee to,
among other things, oversee its system investments.24 However, the
committee could not provide us with any evidence demonstrating
meaningful oversight of these four projects, nor could it provide us with
any guidance describing the committee’s role, responsibilities, and
authorities, and how it oversees projects.

DOD CIO—Oversight of the department’s “major” IT projects, of which
two of the four DFAS projects (DCD/DCW and DPPS) qualify, is the
responsibility of DOD’s CIO. However, this organization did not adequately
fulfill this responsibility on either project because, according to DOD CIO

 Chief Information Officers/Business Integration Executive Council.

Page 14                                    GAO-03-553T Business Systems Modernization
officials, they have little practical authority in influencing component
agency-funded IT projects.

Thus, the bad news is that these three oversight authorities have jointly
permitted approximately $316 million to be spent on the four accounting
system projects without knowing if material changes to the projects’
scopes, costs, benefits, and risks warranted continued investment. The
good news is that the DOD Comptroller recently terminated one of the
four (DPPS), thereby avoiding throwing good money after bad, and DOD
has agreed to implement the recommendations contained in our report
released today, which calls for DOD to demonstrate that the remaining
three projects will produce benefits that exceed costs before further
investing in each.

Our work on other DOD projects has shown similar voids in oversight. For
example, we reported that SPS’s full implementation date slipped by 3 ½
years, with further delays expected, and the system’s life-cycle costs grew
by 23 percent, from $3 billion to $3.7 billion.25 However, none of the
oversight authorities responsible for this project, including the DOD CIO,
had required that the economic analysis be updated to reflect these
changes and thereby provide a basis for informed decisionmaking on the
project’s future. To address this issue, we recommended, among other
things, that the lines of oversight responsibility and accountability of the
project be clarified and that further investment in SPS be limited until
such investment could be justified. DOD has taken steps to address some
of our recommendations. For example, it has clarified organizational
accountability and responsibility for the program. However, much remains
to be done before the department will be able to make informed, data-
driven decisions about whether further investment in the system is


Page 15                              GAO-03-553T Business Systems Modernization
                    We have made numerous recommendations to DOD that collectively
Our                 provide a valuable roadmap for improvement as the department attempts
Recommendations     to create the management infrastructure needed to effectively undertake a
                    massive business systems modernization program. This collection of
Provide a Roadmap   recommendations is not without precedent, as we have provided similar
for Improving       ones to other federal agencies, such as the Federal Aviation
                    Administration, the Internal Revenue Service, and the former U.S.
Management of       Customs Service, to aid them in building their respective capacities for
Business Systems    managing modernization programs. In cases where these
Modernization       recommendations have been implemented properly, we have observed
                    improved modernization management and accountability.

                    Our framework for DOD provides for developing a well-defined and
                    enforceable DOD-wide enterprise architecture to guide and constrain the
                    department’s business system investments, including specific
                    recommendations for successfully accomplishing this, such as creating an
                    enterprise architecture executive committee whose members are
                    singularly and collectively responsible and accountable for delivery and
                    approval of the architecture and a proactive enterprise architecture
                    marketing and communication program to facilitate stakeholder
                    understanding, buy-in, and commitment to the architecture.

                    Our recommendations also provide for establishing a DOD-wide
                    investment decisionmaking structure that consists of a hierarchy of
                    investment boards that are responsible for ensuring that projects meet
                    defined threshold criteria and for reviewing and deciding on projects’
                    futures on the basis of a standard set of investment criteria, two of which
                    are alignment with the enterprise architecture and return on investment.

                    In addition, our recommendations include ensuring that return on
                    investment is analytically supported by current and reliable economic
                    analyses showing that benefits are commensurate with costs and risks,
                    and that these analyses and associated investment decisions cover
                    incremental parts of each system investment, rather than treating the
                    system as one, all-or-nothing, monolithic pursuit. Further, our
                    recommendations provide clear and explicit lines of accountability for
                    project oversight and continuous monitoring and reporting of progress
                    against commitments to ensure that promised system capabilities and
                    benefits are being delivered on time and within budget.

                    Until these recommended system modernization management capabilities
                    are in place and effectively functioning, our recommendations also
                    provide for minimizing the department’s exposure to investment risk by

                    Page 16                              GAO-03-553T Business Systems Modernization
                  limiting its investment in new and existing systems to only projects that
                  (1) have successfully completed testing and involve little additional
                  investment; (2) are “stay-in-business” in nature, meaning that they involve
                  maintenance actions needed to keep a system operational; (3) are
                  congressionally directed; or (4) are relatively small, cost-effective, low-
                  risk, and can be delivered within a short timeframe.

                  In summary, the state of DOD’s business systems environment, coupled
                  with the billions of dollars that DOD spends each year on both existing
                  and new systems, makes a compelling argument for modernizing, but only
                  in a way that ensures that the department does the right thing, and that it
                  does it the right way. Historically, the department’s approach to its
                  business systems modernization has not provided for either. Moreover,
                  while the department’s leadership has stated its commitment to improving,
                  and it has begun efforts on a number of fronts to improve, DOD still is
                  investing in systems in much the same manner that it has for years. This is
                  demonstrated by our testimony today, along with our just-released report
                  on four DFAS system investments and our recent reports on a number of
                  modernization management topics and other DOD system investments. It
                  is therefore imperative, in our view, that DOD move swiftly in
                  implementing our collective set of recommendations aimed at improving
                  its capacity to manage its business systems modernization program. While
                  DOD has largely agreed with these recommendations and has efforts
                  under way intended to implement them, until it does, it will be at high risk
                  of spending billions of dollars on systems that do not support effective and
                  efficient business operations and are unable to provide timely and reliable
                  information for decisionmaking.

                  Mr. Chairmen, this concludes our statement. We would be pleased to
                  answer any questions you or Members of the subcommittees may have at
                  this time.

                  If you or your staff have any questions on matters discussed in this
Contacts and      testimony, please contact Randolph C. Hite at (202) 512-3439 or
Acknowledgement   hiter@gao.gov or Gregory D. Kutz at (202) 512-9505 or kutzg@gao.gov.
                  Individuals making key contributions to this testimony include Beatrice
                  Alff, Sophia Harrison, Tonia L. Johnson, Darby Smith, and Jenniffer

                  Page 17                             GAO-03-553T Business Systems Modernization